LFD Book Forum

LFD Book Forum (http://book.caltech.edu/bookforum/index.php)
-   Homework 1 (http://book.caltech.edu/bookforum/forumdisplay.php?f=130)
-   -   PLA: Let share nice visualizations (http://book.caltech.edu/bookforum/showthread.php?t=3831)

vakhov 01-11-2013 07:03 AM

PLA: Let share nice visualizations
 
Hi!

I think all of us plotted different pictures. Some of them are very nice. For example I get this for PLA learning on N=20. What nice visualisations do you have?

http://vakhov.me/exchange/pla-20.png

Yann222 01-11-2013 09:00 AM

Re: PLA: Let share nice visualizations
 
I don't think it's sharing answers to homework if we exchange code for plotting. Am new to Octave and not having the best luck getting nice-looking plots so far. Would be thrilled to get just the snippet of code people use for visualizing different solutions.

Anne Paulson 01-11-2013 11:40 AM

Re: PLA: Let share nice visualizations
 
I'm just going to paste the Octave code right here. Hope that's OK.

This is code to display the current state of the perceptron, displaying the true separating line, the current hypothesis and all the points. Wrongly classified points are big; rightly classified points are little. Optionally, you can specify the new hypothesis and the currently selected misclassified point, and then they too will be displayed.

When I call this function, I put a pause statement afterwards, so I can step through the algorithm watching it move the hypothesis line and reclassify all the points.

For some reason I'm getting a divide by zero warning for the statements with && in them. I ignore it.

Code:

function draw_percept(points,hypothesis, predictions, true_line, target, bad, new_h)

# points (number of points, weights) is the data points. the bias weight is LAST
# hypothesis is the hypothesis. bias term is LAST.
# predictions(1, N) is what the hypothesis predicts for the points.
# true_line is the true separating line. bias term is LAST
# target is the correct classifications for the points
# bad is optional; it's the misclassified point
# new_h is optional; it's the new hypothesis

  [hypx hypy] = pts_from_hypothesis(hypothesis);
  [truex truey] = pts_from_hypothesis(true_line);
 
  rightly_on = find(predictions == target & predictions == 1);
  wrongly_on = find(predictions != target & predictions ==1);
  rightly_off = find(predictions == target & predictions == -1);
  wrongly_off = find(predictions!= target & predictions == -1);


  hold off;
  # if we are displaying old & new hypotheses,
  # display true line, old hypothesis, new hypothesis
  if nargin >6
      [newx newy] = pts_from_hypothesis(new_h);
      plot(hypx, hypy, 'r', truex, truey,'g',newx, newy,':b');
      legend("old hypothesis red", "true line green", "new hypothesis blue", "location", "southeast");
  else 
      # otherwise plot the hypothesis & real line
      plot(hypx, hypy, 'r', truex, truey,'g');
      legend("old hypothesis", "new hypothesis", "location", "southeast");
  end
 
  axis([-1 1 -1 1]);

  hold on;
  title("Perceptron");

  # plot the points
  scatter(points(rightly_on, 1),points(rightly_on, 2), 'g');
  hold on;
  scatter(points(rightly_off, 1),points(rightly_off, 2), 'b');
  hold on;
  scatter(points(wrongly_on, 1),points(wrongly_on, 2),15, 'r');
  hold on;
  scatter(points(wrongly_off, 1),points(wrongly_off, 2),15,  'c');

  # if we're displaying the selected bad point, display it
  if nargin >3
      hold on;
      scatter(bad(1), bad(2), 20, 'b');
  end
end


function [xs ys] = pts_from_hypothesis(hypothesis)
# hypothesis is a, b, c where line is ax + by + c> 0

# so y = -(a/b)x - c/b
# and x =-(b/a)y - c/a

# we need to find the the intercepts for y=-1, x=-1, y=1 and
# x = 1. Two of those points will be in the [-1,1],[-1,1] box
 

  a = hypothesis(1);
  b = hypothesis(2);
  c= hypothesis(3);
  xm1int = (b-c)/a;
  ym1int = (a-c)/b;
  x1int = (-b-c)/a;
  y1int = (-a-c)/b;

  xs = [-1 1 xm1int x1int];
  ys = [ym1int y1int -1 1];
 
 
end


chocobo_rider 01-11-2013 12:11 PM

Re: PLA: Let share nice visualizations
 
Wow, nice figure. I didn't visualize it like that. I chose to make an animation of the process

vikram360 01-14-2013 10:31 PM

Re: PLA: Let share nice visualizations
 
Please, good sir, do share it :)

theTB 01-15-2013 10:14 AM

Re: PLA: Let share nice visualizations
 
Here is a visualization of the algorithm with n=50 points. The point in the circle is the chosen misclassified point in the current iteration. The final yellow line is the selected hypothesis.



yaser 01-15-2013 10:19 AM

Re: PLA: Let share nice visualizations
 
Quote:

Originally Posted by theTB (Post 8682)
Here is a visualization of the algorithm with n=50 points.

Cool. :)

gah44 01-15-2013 02:28 PM

Re: PLA: Let share nice visualizations
 
I never did any plots.

I did a few drawings with a few points on them to understand the geometry, and once I understood that, believed in the equations.

But sometimes plots are nice to see what it is really doing.

Yann222 01-17-2013 08:26 AM

Re: PLA: Let share nice visualizations
 
Thanks Anne for sharing your code. Helped me pick up a few things that I was able to use to produce my own plots.

Barath 01-17-2013 10:05 AM

Re: PLA: Let share nice visualizations
 
https://lh3.googleusercontent.com/-9...finaldiff2.png

clarkfitzg 04-06-2013 09:39 PM

Re: PLA: Let share nice visualizations
 
http://s23.postimg.org/98thps8dn/Perceptron_page_1.jpg

The black line is f and the green line is g.

chiraz 04-07-2013 04:46 AM

Re: PLA: Let share nice visualizations
 
Quote:

Originally Posted by theTB (Post 8682)
Here is a visualization of the algorithm with n=50 points. The point in the circle is the chosen misclassified point in the current iteration. The final yellow line is the selected hypothesis.



@theTB, cool animation, what language did you use for this animation?
I usually use Matlab but haven't attempted doing animations there..

Chiraz BenAbdelkader

chiraz 04-07-2013 04:54 AM

Re: PLA: Let share nice visualizations
 
@Barath, I like to look of your plot; what software did you use? I use Matlab and my plots are much more lame looking!

Thanks,
Chiraz BenAbdelkader

OlivierB 04-07-2013 05:45 AM

Re: PLA: Let share nice visualizations
 
I would share my graph too, though it is exactly as expected. But impossible to upload an image... No trace of the 'Managed Attachment' button mentioned in the FAQ..
And I read 'You may not post attachments' at the bottom of my page..
I would be grateful for some help on this uninteresting topic.

OlivierB 04-07-2013 06:02 AM

Re: PLA: Let share nice visualizations
 
I cannot upload images to the forum, maybe because i read 'You may not post attachments' at the bottom of my screen.... Please help.

clarkfitzg 04-07-2013 12:37 PM

Re: PLA: Let share nice visualizations
 
Quote:

Originally Posted by OlivierB (Post 10197)
I cannot upload images to the forum, maybe because i read 'You may not post attachments' at the bottom of my screen.... Please help.

Use an image hosting service like http://postimage.org/. Click on the image button and paste the URL to your image.

OlivierB 04-07-2013 03:31 PM

Re: PLA: Let share nice visualizations
 
@Clarkfitzg, thx !

Here is another graph of the PLA (N=50), plotted with octave.

http://s18.postimg.org/p9b7g9qex/PLA_convergence1.png

Remark: In terms of convergence speed, for 1000 runs, as requested in questions 7,9, I find surprisingly different results from one trial (1000 runs) to the next (I use the rand() from octave). In my view, the difference makes it debatable as to which answer is "correct" (since only one is correct officially). I would tick 2 letters if I could. Am I the only one facing this issue ? Have I made some obvious mistake ? Or is it just a consequence of the imperfect multi choice form of the homework (Let nobody misunderstand me, I find the subject very interesting, the course VERY well presented, and the professor passionate and VERY engaging. By the way THANK YOU for putting your class online ! :bow:) ?

yaser 04-07-2013 03:48 PM

Re: PLA: Let share nice visualizations
 
Quote:

Originally Posted by OlivierB (Post 10210)
Remark: In terms of convergence speed, for 1000 runs, as requested in questions 7,9, I find surprisingly different results from one trial (1000 runs) to the next (I use the rand() from octave).

Hi,

Do your 1000 runs have different target functions as well as training data?

OlivierB 04-08-2013 12:37 PM

Re: PLA: Let share nice visualizations
 
Oh ! Thank you for clarifying this point.

Actually no, I was randomly choosing one f and training set, and then running 1000 PLA on these.
The variation in convergence speed were coming from the randomly selected misclassified point at each step.

Indeed, on re-reading hw1.pdf more carefully, I see that you mention "In each run, choose a random line in the plane as your target function...".

Now for each of the 1000 runs, I randomly choose a single f and training set and then run on PLA on them.
The convergence speed is now very stable, and the questions 7 and 9 do not seem ambiguous any longer.

As I already have submitted, I know that I probably get the correct results for questions 7, 8, and 10. But not for question 9! So there is still something puzzling... If by experience you have an idea about my probable mistake, I would be grateful for your insight (In fact, I already am for your first answer).

Elroch 04-08-2013 02:19 PM

Re: PLA: Let share nice visualizations
 
Quote:

Originally Posted by OlivierB (Post 10235)
Oh ! Thank you for clarifying this point.

Actually no, I was randomly choosing one f and training set, and then running 1000 PLA on these.
The variation in convergence speed were coming from the randomly selected misclassified point at each step.

Indeed, on re-reading hw1.pdf more carefully, I see that you mention "In each run, choose a random line in the plane as your target function...".

Now for each of the 1000 runs, I randomly choose a single f and training set and then run on PLA on them.
The convergence speed is now very stable, and the questions 7 and 9 do not seem ambiguous any longer.

As I already have submitted, I know that I probably get the correct results for questions 7, 8, and 10. But not for question 9! So there is still something puzzling... If by experience you have an idea about my probable mistake, I would be grateful for your insight (In fact, I already am for your first answer).

There is a quantitative indication of one part of the variation in the number of iterations in the proof of convergence of the PLA algorithm in Problem 1.3 in the book. In some problems (i.e. two separable sets of points), a much larger range of lines will perform the classification than in others. If one randomly picked lines, the expected time to converge would vary in a predictable manner between problems. Difficult problems for such a random line picking algorithm (if it deserves the term) are also ones with a loose bound on convergence using PLA.

eg) a 100 point problem can be relatively easy sometimes.
http://i699.photobucket.com/albums/v...a/PLA100-1.png

IsidroHidalgo 04-08-2013 02:57 PM

Re: PLA: Let share nice visualizations
 
With N=10 I have many iterations lower than[*] (I have an "outlier" too:[*] !!!!)

[*] Admin Edit: I have masked the specific numbers since they reveal too much about the answer. Please start a thread whose title starts with *ANSWER* to discuss this per the announcement.

yaser 04-08-2013 03:04 PM

Re: PLA: Let share nice visualizations
 
Quote:

Originally Posted by OlivierB (Post 10235)
As I already have submitted, I know that I probably get the correct results for questions 7, 8, and 10. But not for question 9! So there is still something puzzling... If by experience you have an idea about my probable mistake, I would be grateful for your insight

I suggest starting an *ANSWER* thread and discussing this openly with other participants.

wangkexue 04-08-2013 07:38 PM

Re: PLA: Let share nice visualizations
 
everyone is cool
and here is mine

http://c.hiphotos.baidu.com/album/s%...f5e0fe7f2f.jpg

The most exciting thing is that after 1000 iteration, I get almost exactly the answer, with differences 0.6 and 3 in average iteration respectively for N = 10 and 100.

Admin edit: I am guessing this is the wrong jpg, so please let me know the right one and I'll edit it in.

pyguy 04-08-2013 10:13 PM

Re: PLA: Let share nice visualizations
 
Here's my attempt at an animation using Python and matplotlib with N=100: http://dl.dropbox.com/u/153/hw/problem_9.html.

yaser 04-08-2013 10:45 PM

Re: PLA: Let share nice visualizations
 
Quote:

Originally Posted by pyguy (Post 10258)
Here's my attempt at an animation using Python and matplotlib with N=100: http://dl.dropbox.com/u/153/hw/problem_9.html.

Cool. :)

OlivierB 04-09-2013 11:52 AM

Re: PLA: Let share nice visualizations
 
Quote:

Originally Posted by Elroch (Post 10238)
There is a quantitative indication of one part of the variation in the number of iterations in the proof of convergence of the PLA algorithm in Problem 1.3 in the book.

Thanks for pointing problem 1.3 in the book.
I have ordered it, but not received it yet. I will have a look.

oscargiraldo 04-09-2013 08:54 PM

Re: PLA: Let share nice visualizations
 
This is my contribution :)
http://sphotos-a.ak.fbcdn.net/hphoto...94353584_n.jpg

paul99 04-09-2013 09:26 PM

Re: PLA: Let share nice visualizations
 
Quote:

Originally Posted by pyguy (Post 10258)
Here's my attempt at an animation using Python and matplotlib with N=100: http://dl.dropbox.com/u/153/hw/problem_9.html.

Thanks @pyguy !! That was very cool! I would love to see how you did that -- any chance you can share your code? I want to learn! Many thanks!

wangkexue 04-09-2013 11:35 PM

Re: PLA: Let share nice visualizations
 
Dose this link work?
http://test-for-use.weebly.com/uploa...01962_orig.png

IsidroHidalgo 04-10-2013 04:38 AM

Re: PLA: Let share nice visualizations
 
Quote:

Originally Posted by IsidroHidalgo (Post 10240)
With N=10 I have many iterations lower than[*] (I have an "outlier" too:[*] !!!!)

[*] Admin Edit: I have masked the specific numbers since they reveal too much about the answer. Please start a thread whose title starts with *ANSWER* to discuss this per the announcement.

Sorry about that. I'll be more careful...

mluser 04-12-2013 12:07 PM

Re: PLA: Let share nice visualizations
 
For N = 20, green line is f and red line is g

http://s15.postimg.org/kxq3g4rvf/plot.png

jonlim 09-10-2013 10:43 PM

Re: PLA: Let share nice visualizations
 
n=20, orange line is f, green line is g
http://s22.postimg.org/gl2koxckh/hw0...ngfromdata.png

vitamintk 08-27-2015 07:09 PM

Re: PLA: Let share nice visualizations
 
https://raw.githubusercontent.com/Vi...1440717390.gif

yaser 08-29-2015 12:47 PM

Re: PLA: Let share nice visualizations
 
Thank you for sharing, and for bumping this interesting thread.


All times are GMT -7. The time now is 01:13 PM.

Powered by vBulletin® Version 3.8.3
Copyright ©2000 - 2020, Jelsoft Enterprises Ltd.
The contents of this forum are to be used ONLY by readers of the Learning From Data book by Yaser S. Abu-Mostafa, Malik Magdon-Ismail, and Hsuan-Tien Lin, and participants in the Learning From Data MOOC by Yaser S. Abu-Mostafa. No part of these contents is to be communicated or made accessible to ANY other person or entity.