Show Active Servers
Active Servers:
Closed
William Chen
3:48 PM
4:28 PM
Server: Hansohl Kim
Announcements
There are currently no announcements.
Chat
Shuhui Qu
Apr 10, 5:01 PM
Hi all, I am at room 016
Shuhui Qu
Apr 10, 5:04 PM
Please let me know where you are
Shuhui Qu
Apr 10, 5:23 PM
Hey, could you please come to room 16
Barak Oshri
Apr 11, 4:16 PM
Don where can I find you?
Guillaume Genthial
Apr 12, 6:16 PM
Clare Chen where are you?
Hansohl Kim
Apr 17, 3:30 PM
Zihua where are you?
Shuhui Qu
Apr 17, 6:04 PM
Hey Casey, where are you
Shuhui Qu
Apr 17, 6:18 PM
Hey Casey, I am in Brown
Shuhui Qu
Apr 17, 6:22 PM
Hi Huyen, where are you?
Huyen Nguyen
Apr 17, 6:24 PM
i'm in front of room 49
Huyen Nguyen
Apr 17, 6:26 PM
where are you?
Huyen Nguyen
Apr 17, 6:30 PM
hello
Shuhui Qu
Apr 17, 6:31 PM
be right there
Shuhui Qu
Apr 17, 6:42 PM
where's 49...
Shuhui Qu
Apr 17, 6:44 PM
Can you see me?
Casey Chu
Apr 17, 6:51 PM
Hi Shuhui are you still around?
Shuhui Qu
Apr 17, 6:58 PM
Still here
Casey Chu
Apr 17, 6:59 PM
Where? I am on the red couches
Casey Chu
Apr 17, 6:59 PM
in green
Casey Chu
Apr 17, 6:59 PM
wearing green
Shuhui Qu
Apr 17, 7:21 PM
I am helping ziyi
Shuhui Qu
Apr 17, 7:21 PM
I will come to your soon.
Barak Oshri
Apr 18, 5:27 PM
Nipun, where can I find you?
Barak Oshri
Apr 18, 5:29 PM
Atli, where can I find you?
Barak Oshri
Apr 18, 5:29 PM
All, please provide better descriptions of how I can find you, it's really crowded here!
Atli Kosson
Apr 18, 5:29 PM
Under the stairs
Atli Kosson
Apr 18, 5:29 PM
outside the nvidia auditorium
Barak Oshri
Apr 18, 5:36 PM
be there soon
Barak Oshri
Apr 18, 6:45 PM
Axel where can I find you?
Barak Oshri
Apr 18, 7:06 PM
Gus where can I find you?
Barak Oshri
Apr 18, 7:06 PM
Blake, are you still at the red couches?
Max Ferguson
Apr 19, 6:31 PM
Are there any other students waiting that want to discuss homework 1?
Max Ferguson
Apr 19, 6:48 PM
@Guillaume: Where are you?
Guillaume Genthial
Apr 19, 6:54 PM
where are you?
Guillaume Genthial
Apr 19, 6:55 PM
I'm standing in the middle of Huang, I wear a white shirt
Alex Kolchinski
Apr 20, 10:35 AM
hi I'm online
Connie Zeng
Apr 20, 10:37 AM
Hi Alex, how do you want to discuss your question? I've sent you a Google Hangout invite but can chat through some other method of your choice.
Alex Kolchinski
Apr 20, 10:38 AM
Google Hangouts sounds great, i'm at y.a.kolchinski@gmail.com
Alex Kolchinski
Apr 20, 10:43 AM
any luck finding me?
Anthony Kim
Apr 20, 1:33 PM
Aarush Selvan, where are you?
Saied Mehdian
Apr 21, 12:39 PM
Hi Alex,
Saied Mehdian
Apr 21, 12:39 PM
Hi Can I help you?
Alex Kolchinski
Apr 21, 12:39 PM
hi!
Alex Kolchinski
Apr 21, 12:39 PM
yes
Alex Kolchinski
Apr 21, 12:39 PM
i mostly wanted to ask if you could look at this - https://piazza.com/class/j0zg8g4ww9s63f
Alex Kolchinski
Apr 21, 12:39 PM
i've been sick all week so wondering if i could use a third late day on hw1
Alex Kolchinski
Apr 21, 12:40 PM
still working on #5
Saied Mehdian
Apr 21, 12:43 PM
Can you specify the piazza post number?
Alex Kolchinski
Apr 21, 12:44 PM
200
Saied Mehdian
Apr 21, 12:53 PM
The instructor, Professor Brunskill has decided the solutions for homework1 to be released very soon.
Alex Kolchinski
Apr 21, 12:54 PM
how soon? i only need a few more hours to finish #5
Saied Mehdian
Apr 21, 12:55 PM
Decision about extending needs to be done by the Instructor, Prof. Brunskill. I sent her a message but I have not heard back.
Alex Kolchinski
Apr 21, 12:56 PM
thanks
Saied Mehdian
Apr 21, 12:56 PM
As of now, I am not sure if we are going to have an extra extension.
Alex Kolchinski
Apr 21, 12:57 PM
i guess i'll submit what i have by 1:30 and then keep working on it
Saied Mehdian
Apr 21, 12:58 PM
I recommend to contact the instructor directly. Maybe you should send her an email.
Saied Mehdian
Apr 21, 1:00 PM
She is having an office hour today 3-4
Saied Mehdian
Apr 21, 1:01 PM
The posted about in on piazza
Saied Mehdian
Apr 21, 1:01 PM
I hope this helps.
Alex Kolchinski
Apr 21, 1:04 PM
hm thanks
Alex Kolchinski
Apr 21, 1:33 PM
hi can I catch you for a couple of questions on q5?
Saied Mehdian
Apr 21, 1:41 PM
Hi Alex,
Saied Mehdian
Apr 21, 1:41 PM
How can I help you?
Alex Kolchinski
Apr 21, 1:48 PM
sent you a gchat invite
Alex Kolchinski
Apr 21, 1:48 PM
i'm at y.a.kolchinski@gmail.com
Saied Mehdian
Apr 21, 1:55 PM
I do not use gchat.
Saied Mehdian
Apr 21, 1:55 PM
can we use chat for your question
Saied Mehdian
Apr 21, 1:55 PM
?
Alex Kolchinski
Apr 21, 1:59 PM
sure
Alex Kolchinski
Apr 21, 1:59 PM
1 is - is the .78 score goal for q5 discounted by gamma^t?
Alex Kolchinski
Apr 21, 1:59 PM
or just .78 of the time the agent reaches the goal
Alex Kolchinski
Apr 21, 2:00 PM
also, my q learning converges when i initialize it with 1s but not with 0s - is that expected?
Saied Mehdian
Apr 21, 2:02 PM
it is .78 of the times, the agent reaches the goal
Saied Mehdian
Apr 21, 2:02 PM
piazza post 188 should be helpful to clarify that.
Saied Mehdian
Apr 21, 2:04 PM
Your algorithm performance can depend on the initializatoin
Alex Kolchinski
Apr 21, 2:04 PM
gotcha, i don't see that in 188 but isn't the score for a mdp usually discounted by gamma and time?
Saied Mehdian
Apr 21, 2:04 PM
*initializatin
Saied Mehdian
Apr 21, 2:05 PM
*initialization
Alex Kolchinski
Apr 21, 2:05 PM
it's just that it gets stuck at all 0s when i initialize it that way
Alex Kolchinski
Apr 21, 2:05 PM
does that mean i have a bug? it works just fine when i initialize it to all 1s
Saied Mehdian
Apr 21, 2:05 PM
please refer to piazza post 142 for more details
Saied Mehdian
Apr 21, 2:06 PM
it does not necessarily imply you have a bug.
Alex Kolchinski
Apr 21, 2:06 PM
gotcha, thanks
Saied Mehdian
Apr 21, 2:06 PM
Your algorithm seems to have very good performance with all 1s
Saied Mehdian
Apr 21, 2:07 PM
Please keep in mind, the algorithm should not get stuck for a particular initial vector.
Saied Mehdian
Apr 21, 2:09 PM
One potential issue you might want to check is the terminal state. Make sure you update the count for the terminal state
Saied Mehdian
Apr 21, 2:16 PM
for an mdp, optimizingthe sum of discounted rewards is the objective. However, in hw1 q5, what really matters is whether you were able to reach the final cell
Saied Mehdian
Apr 21, 2:18 PM
Hi Alex,
Saied Mehdian
Apr 21, 2:18 PM
Saied Mehdian
Apr 21, 2:18 PM
Regarding late homework submission, please see this link:
Saied Mehdian
Apr 21, 2:18 PM
http://web.stanford.edu/class/cs234/grading.html
Alex Kolchinski
Apr 21, 2:20 PM
thanks Saied, what do you mean by updating the count for the terminal state?
Saied Mehdian
Apr 21, 2:22 PM
each time step, you will have (s,a,r,s')
Saied Mehdian
Apr 21, 2:23 PM
once your reach a termination state, you will also need to increment the count for it.
Saied Mehdian
Apr 21, 2:24 PM
If your are not handling this, then the problem parameters (including the reward) is not changing for termination states and this is a bug
Saied Mehdian
Apr 21, 2:33 PM
Hi Alex,
Saied Mehdian
Apr 21, 2:33 PM
My office hour is over.
Saied Mehdian
Apr 21, 2:33 PM
The next office hour today is 3-4
Saied Mehdian
Apr 21, 2:33 PM
you can also post further questions on piazza
Saied Mehdian
Apr 21, 2:34 PM
have a nice day.
Connie Zeng
Apr 24, 10:03 AM
I can't find a whiteboard marker, but I am standing at the central high tables.
Jesik Min
Apr 24, 5:51 PM
is the queue in process now?
Li Deng
Apr 26, 4:40 PM
Where is the office hour right now at Huang?
Guillaume Genthial
Apr 26, 6:17 PM
Guys, I'm going to Lathrop for the coding session
Hansohl Kim
May 1, 3:57 PM
Eric I can't find you but let me know if you still have questions/where you are
Shuhui Qu
May 1, 5:00 PM
Hey, I am at Room 013
Shuhui Qu
May 1, 5:01 PM
Please let me know if you want me to switch
Alex Kolchinski
May 1, 5:02 PM
I'll come over
Shuhui Qu
May 1, 5:18 PM
Hey David I am at Room 013
Shuhui Qu
May 1, 5:19 PM
Where are you?
Shuhui Qu
May 1, 5:21 PM
Also, guys do you mind move to Room 013?
Shuhui Qu
May 1, 5:24 PM
Hi Christopher, any hint for your location?
Shuhui Qu
May 1, 5:33 PM
Hey Raghav, I cannot see you. Did you leave?
Shuhui Qu
May 1, 5:37 PM
Hey, guys please let me know if you are still here
Shuhui Qu
May 1, 5:37 PM
I am at Room13
Shuhui Qu
May 1, 5:51 PM
where are you Alex
Shuhui Qu
May 1, 6:13 PM
Hey Li, Can you come to Room013?
Barak Oshri
May 2, 5:18 PM
hey everyone, I'm going to be migrating to the coding session at Hewlett 101 at around 6pm. I will be handling anyone left on my queue there first if you want to move there as well
Guillaume Genthial
May 3, 5:35 PM
alex?
Guillaume Genthial
May 3, 5:37 PM
I'm sorry but blue shirt is not easy to spot in Huang...
Alex Kolchinski
May 4, 1:56 PM
hi are office hours happening?
Alex Kolchinski
May 4, 1:56 PM
haven't seen any progress on the queue all hour
Alex Kolchinski
May 4, 1:57 PM
oh never mind looks like it's moving
Saied Mehdian
May 5, 1:51 PM
Hi Stephen,
Saied Mehdian
May 5, 1:51 PM
Shuhui Qu
May 15, 4:50 PM
Hi all, I am at room 19
Anthony Kim
May 16, 5:03 PM
@Clare, where are you?
Shuhui Qu
May 22, 4:57 PM
Hi all, I am at room 20
Barak Oshri
May 23, 5:33 PM
Raghav, where can I find you?
Barak Oshri
May 23, 5:34 PM
Charles, where can I find you?
Barak Oshri
May 23, 5:35 PM
Clare, where are you too!
Barak Oshri
May 23, 7:00 PM
Vin/Charles, are you still here?
Anthony Kim
May 25, 2:13 PM
@Axel, where are you?
Anthony Kim
May 25, 2:38 PM
@Max, where are you?
Max Ferguson
May 25, 2:38 PM
The same desk you have been at for a while
Anthony Kim
May 25, 2:47 PM
@Jason, where are you?
Anthony Kim
May 25, 2:49 PM
Cannot find you. Moving onto the next person
Vinaya Polamreddi
May 25, 4:06 PM
hi @hansohl, it says im in process, but I dont see you
Shuhui Qu
May 29, 5:03 PM
at sofa by nvidia
Barak Oshri
May 30, 5:07 PM
I'm on the red couch
Shuhui Qu
Jun 5, 5:13 PM
@015
Barak Oshri
Jun 6, 1:21 PM
Hey Raunaq, please come to the red couches
Hansohl Kim
Jun 8, 3:51 PM
Hi William, I had to check something quickly but I'll be there in a few minutes
William Chen
Jun 8, 3:54 PM
Sure, np!
Tesa Ho
Jan 16, 4:03 PM
hello? anyone there?
Nick Guo
Jan 14, 3:00 PM
hello :)