| --- Log opened Sun Apr 21 00:00:03 2013 | ||
| -!- Charlie [~quassel@74.63.212.44] has joined ##hplusroadmap | 00:02 | |
| -!- Juul [~Juul@108-83-132-49.lightspeed.sntcca.sbcglobal.net] has quit [Ping timeout: 264 seconds] | 00:27 | |
| -!- yashgaroth [~ffffff@cpe-66-27-118-94.san.res.rr.com] has quit [Quit: Leaving] | 00:40 | |
| -!- superkuh [~superkuh@unaffiliated/superkuh] has joined ##hplusroadmap | 01:11 | |
| kanzure | why is my computer's clock reporting an hour slow? | 01:12 |
|---|---|---|
| kanzure | maybe i am time traveling | 01:12 |
| kanzure | greetings from the past | 01:12 |
| rigel | woo | 01:13 |
| rigel | so i finally built a basic class wrapper for selenium in order to scrape certain websites that are irritatingly javascript-intensive | 01:14 |
| kanzure | i warned you not to use selenium | 01:14 |
| rigel | i fucking hate it when people put scripts inside tags | 01:14 |
| rigel | yeah you did, but i disregarded your advice, in part because i need to be able to see the results | 01:15 |
| kanzure | there's a render() and renderBase64() method in phantomjs | 01:15 |
| rigel | and since i have a class wrapper now, and have done all the footwork i can re-implement it in a better framework | 01:15 |
| rigel | if need be | 01:15 |
| rigel | it's still not finished, but i am able to log in to my target site and flip through the various pages of search results without it throwing an error | 01:16 |
| rigel | i do need to do some work to make it error tolerant, but that in and of itself is huge | 01:16 |
| kanzure | the most helpful thing you can do for error tolerance is "restart" tolerance | 01:17 |
| kanzure | for saving at a certain point and resuming | 01:17 |
| rigel | yeah, restarting is a pain in the ass, especially when i am working with a wrapper | 01:17 |
| kanzure | scraping 2M pages and having a critical error at 1,500,000 is really annoying (especially if you don't remember to tell yourself the id you left at) | 01:17 |
| rigel | because if i make changes to the wrapper, i lose connection to my object | 01:17 |
| rigel | yeah, i'm not quite at the download-all-the-pages part yet | 01:18 |
| rigel | i am building up to that | 01:18 |
| rigel | first i am going to flip through the 80 pages of search results, and pull all the URLs i need | 01:19 |
| rigel | that way i can just navigate to them directly, save the source to a file, and move on to the next one | 01:19 |
| -!- lichen [~lichen@c-24-21-206-64.hsd1.or.comcast.net] has joined ##hplusroadmap | 01:29 | |
| -!- ThomasEgi [~thomas@185.5.8.81] has joined ##hplusroadmap | 02:34 | |
| -!- ThomasEgi [~thomas@185.5.8.81] has quit [Changing host] | 02:34 | |
| -!- ThomasEgi [~thomas@panda3d/ThomasEgi] has joined ##hplusroadmap | 02:34 | |
| -!- chris_99 [~chris_99@unaffiliated/chris-99/x-3062929] has joined ##hplusroadmap | 03:12 | |
| -!- Zarakii [~devillll@ppp-46-244-176-131.dynamic.mnet-online.de] has quit [Read error: Connection reset by peer] | 03:31 | |
| -!- Suwap [~devillll@ppp-46-244-176-131.dynamic.mnet-online.de] has joined ##hplusroadmap | 03:31 | |
| -!- Aquifex [~Aquifex@host-78-144-152-224.as13285.net] has joined ##hplusroadmap | 03:44 | |
| superkuh | paperbot: http://www.sciencemag.org/content/340/6130/367.full | 03:45 |
| paperbot | error: HTTP 300 http://diyhpl.us/~bryan/papers2/paperbot/Representation%20of%20Three-Dimensional%20Space%20in%20the%20Hippocampus%20of%20Flying%20Bats.pdf | 03:45 |
| -!- FooQuuxman [~arik@c-98-215-254-55.hsd1.il.comcast.net] has joined ##hplusroadmap | 04:13 | |
| -!- yorick [~yorick@oftn/member/yorick] has joined ##hplusroadmap | 04:15 | |
| -!- makoLime [~mako@103-9-42-32.flip.co.nz] has quit [Ping timeout: 264 seconds] | 04:23 | |
| -!- Viper168 [~Viper@unaffiliated/viper168] has quit [Ping timeout: 252 seconds] | 04:40 | |
| -!- Viper168 [~Viper@unaffiliated/viper168] has joined ##hplusroadmap | 05:02 | |
| -!- Aquifex [~Aquifex@host-78-144-152-224.as13285.net] has quit [Read error: Connection reset by peer] | 05:11 | |
| -!- Aquifex [Aquifex@host-78-148-125-12.as13285.net] has joined ##hplusroadmap | 05:15 | |
| -!- EnLilaSko [EnLilaSko@unaffiliated/enlilasko] has joined ##hplusroadmap | 05:25 | |
| -!- Suwap [~devillll@ppp-46-244-176-131.dynamic.mnet-online.de] has quit [Read error: Connection reset by peer] | 05:39 | |
| -!- Suwap [~devillll@ppp-46-244-176-131.dynamic.mnet-online.de] has joined ##hplusroadmap | 05:39 | |
| -!- ielo [~ielo@cpc9-addl4-2-0-cust229.6-3.cable.virginmedia.com] has joined ##hplusroadmap | 05:42 | |
| -!- jmil [~jmil@hive76/member/jmil] has joined ##hplusroadmap | 05:45 | |
| -!- klafka [~klafka@c-24-6-18-31.hsd1.ca.comcast.net] has joined ##hplusroadmap | 05:51 | |
| -!- ielo [~ielo@cpc9-addl4-2-0-cust229.6-3.cable.virginmedia.com] has quit [Ping timeout: 256 seconds] | 06:07 | |
| -!- panax [~panax@68.200.160.182] has quit [Ping timeout: 276 seconds] | 06:08 | |
| -!- panax [~panax@68.200.160.182] has joined ##hplusroadmap | 06:10 | |
| -!- ielo [~ielo@cpc9-addl4-2-0-cust229.6-3.cable.virginmedia.com] has joined ##hplusroadmap | 06:41 | |
| -!- klafka [~klafka@c-24-6-18-31.hsd1.ca.comcast.net] has quit [Remote host closed the connection] | 06:57 | |
| -!- jmil [~jmil@hive76/member/jmil] has quit [Quit: jmil] | 07:14 | |
| -!- eudoxia [~eudoxia@r186-52-179-59.dialup.adsl.anteldata.net.uy] has joined ##hplusroadmap | 07:40 | |
| -!- eudoxia [~eudoxia@r186-52-179-59.dialup.adsl.anteldata.net.uy] has quit [Ping timeout: 276 seconds] | 07:48 | |
| -!- ielo [~ielo@cpc9-addl4-2-0-cust229.6-3.cable.virginmedia.com] has quit [Ping timeout: 258 seconds] | 07:57 | |
| -!- Aquifex [Aquifex@host-78-148-125-12.as13285.net] has quit [Ping timeout: 246 seconds] | 08:00 | |
| -!- aristarchus [~aristarch@unaffiliated/aristarchus] has joined ##hplusroadmap | 08:01 | |
| -!- Aquifex [Aquifex@host-92-16-12-220.as13285.net] has joined ##hplusroadmap | 08:02 | |
| -!- Suwap [~devillll@ppp-46-244-176-131.dynamic.mnet-online.de] has quit [Read error: Connection reset by peer] | 08:14 | |
| -!- Suwap [~devillll@ppp-46-244-176-131.dynamic.mnet-online.de] has joined ##hplusroadmap | 08:15 | |
| -!- ielo [~ielo@cpc9-addl4-2-0-cust229.6-3.cable.virginmedia.com] has joined ##hplusroadmap | 08:15 | |
| -!- Suwap [~devillll@ppp-46-244-176-131.dynamic.mnet-online.de] has quit [Ping timeout: 264 seconds] | 08:19 | |
| -!- Suwap [~devillll@ppp-46-244-176-131.dynamic.mnet-online.de] has joined ##hplusroadmap | 08:27 | |
| -!- kmo [~kmo@unaffiliated/kmo] has joined ##hplusroadmap | 08:28 | |
| -!- Aquifex [Aquifex@host-92-16-12-220.as13285.net] has quit [] | 08:31 | |
| -!- kmo [~kmo@unaffiliated/kmo] has quit [Ping timeout: 240 seconds] | 08:36 | |
| -!- kmo [~kmo@d49-191.icpnet.pl] has joined ##hplusroadmap | 08:55 | |
| -!- kmo [~kmo@d49-191.icpnet.pl] has quit [Changing host] | 08:55 | |
| -!- kmo [~kmo@unaffiliated/kmo] has joined ##hplusroadmap | 08:55 | |
| -!- FooQuuxman [~arik@c-98-215-254-55.hsd1.il.comcast.net] has quit [Ping timeout: 264 seconds] | 09:06 | |
| -!- lichen [~lichen@c-24-21-206-64.hsd1.or.comcast.net] has quit [Ping timeout: 255 seconds] | 09:06 | |
| -!- kmo [~kmo@unaffiliated/kmo] has quit [Ping timeout: 255 seconds] | 09:17 | |
| -!- kmo [~kmo@d49-191.icpnet.pl] has joined ##hplusroadmap | 09:19 | |
| -!- kmo [~kmo@d49-191.icpnet.pl] has quit [Changing host] | 09:19 | |
| -!- kmo [~kmo@unaffiliated/kmo] has joined ##hplusroadmap | 09:19 | |
| -!- lichen [~lichen@c-24-21-206-64.hsd1.or.comcast.net] has joined ##hplusroadmap | 09:24 | |
| -!- yashgaroth [~ffffff@cpe-66-27-118-94.san.res.rr.com] has joined ##hplusroadmap | 10:06 | |
| -!- ielo [~ielo@cpc9-addl4-2-0-cust229.6-3.cable.virginmedia.com] has quit [Ping timeout: 272 seconds] | 10:15 | |
| -!- ParahSa1lin [~pwang@unaffiliated/parahsailin] has quit [Ping timeout: 256 seconds] | 10:26 | |
| -!- FooQuuxman [~arik@c-98-215-254-55.hsd1.il.comcast.net] has joined ##hplusroadmap | 10:43 | |
| -!- kmo [~kmo@unaffiliated/kmo] has quit [Ping timeout: 256 seconds] | 11:04 | |
| -!- kajetan [~kmo@d49-191.icpnet.pl] has joined ##hplusroadmap | 11:05 | |
| -!- yorick [~yorick@oftn/member/yorick] has quit [Remote host closed the connection] | 11:05 | |
| -!- ielo [~ielo@cpc9-addl4-2-0-cust229.6-3.cable.virginmedia.com] has joined ##hplusroadmap | 11:23 | |
| -!- Juul [~Juul@108-83-132-49.lightspeed.sntcca.sbcglobal.net] has joined ##hplusroadmap | 11:50 | |
| -!- FooQuuxman [~arik@c-98-215-254-55.hsd1.il.comcast.net] has quit [Ping timeout: 256 seconds] | 11:50 | |
| -!- makoLime [~mako@103-9-42-32.flip.co.nz] has joined ##hplusroadmap | 12:03 | |
| -!- Juul [~Juul@108-83-132-49.lightspeed.sntcca.sbcglobal.net] has quit [Ping timeout: 248 seconds] | 12:08 | |
| -!- FooQuuxman [~arik@c-98-215-254-55.hsd1.il.comcast.net] has joined ##hplusroadmap | 12:12 | |
| -!- Juul [~Juul@108-83-132-49.lightspeed.sntcca.sbcglobal.net] has joined ##hplusroadmap | 12:19 | |
| -!- Viper168 [~Viper@unaffiliated/viper168] has quit [Ping timeout: 256 seconds] | 13:05 | |
| -!- ParahSa1lin [~pwang@unaffiliated/parahsailin] has joined ##hplusroadmap | 13:13 | |
| -!- FooQuuxman [~arik@c-98-215-254-55.hsd1.il.comcast.net] has quit [Quit: leaving] | 13:14 | |
| -!- FooQuuxman [~arik@c-98-215-254-55.hsd1.il.comcast.net] has joined ##hplusroadmap | 13:17 | |
| -!- aristarchus [~aristarch@unaffiliated/aristarchus] has quit [Quit: Leaving] | 13:19 | |
| -!- kajetan [~kmo@d49-191.icpnet.pl] has quit [Ping timeout: 240 seconds] | 13:31 | |
| -!- Suwap [~devillll@ppp-46-244-176-131.dynamic.mnet-online.de] has quit [Read error: Connection reset by peer] | 13:38 | |
| -!- Suwap [~devillll@ppp-46-244-176-131.dynamic.mnet-online.de] has joined ##hplusroadmap | 13:38 | |
| -!- Juul [~Juul@108-83-132-49.lightspeed.sntcca.sbcglobal.net] has quit [Ping timeout: 272 seconds] | 13:39 | |
| -!- Juul [~Juul@108-83-132-49.lightspeed.sntcca.sbcglobal.net] has joined ##hplusroadmap | 13:51 | |
| -!- Juul [~Juul@108-83-132-49.lightspeed.sntcca.sbcglobal.net] has quit [Ping timeout: 256 seconds] | 14:15 | |
| -!- Juul [~Juul@108-83-132-49.lightspeed.sntcca.sbcglobal.net] has joined ##hplusroadmap | 14:26 | |
| -!- EnLilaSko [EnLilaSko@unaffiliated/enlilasko] has quit [Remote host closed the connection] | 14:29 | |
| -!- Juul [~Juul@108-83-132-49.lightspeed.sntcca.sbcglobal.net] has quit [Ping timeout: 256 seconds] | 14:49 | |
| -!- klafka [~klafka@c-24-6-18-31.hsd1.ca.comcast.net] has joined ##hplusroadmap | 14:51 | |
| -!- klafka [~klafka@c-24-6-18-31.hsd1.ca.comcast.net] has quit [Ping timeout: 248 seconds] | 14:56 | |
| -!- Juul [~Juul@108-83-132-49.lightspeed.sntcca.sbcglobal.net] has joined ##hplusroadmap | 15:02 | |
| -!- devrandom [~devrandom@50-0-206-254.dsl.dynamic.sonic.net] has quit [Ping timeout: 240 seconds] | 15:21 | |
| -!- Juul [~Juul@108-83-132-49.lightspeed.sntcca.sbcglobal.net] has quit [Ping timeout: 272 seconds] | 15:24 | |
| -!- devrandom [~devrandom@50-0-206-254.dsl.dynamic.sonic.net] has joined ##hplusroadmap | 15:29 | |
| -!- Juul [~Juul@108-83-132-49.lightspeed.sntcca.sbcglobal.net] has joined ##hplusroadmap | 15:37 | |
| -!- Viper168 [~Viper@unaffiliated/viper168] has joined ##hplusroadmap | 15:46 | |
| -!- AdrianG [~dextro@unaffiliated/amphetamine] has joined ##hplusroadmap | 15:56 | |
| -!- AdrienG [~dextro@unaffiliated/amphetamine] has quit [Ping timeout: 272 seconds] | 15:58 | |
| -!- chris_99 [~chris_99@unaffiliated/chris-99/x-3062929] has quit [Quit: Leaving] | 16:07 | |
| -!- Suwap [~devillll@ppp-46-244-176-131.dynamic.mnet-online.de] has quit [Read error: Connection reset by peer] | 16:23 | |
| -!- Suwap [~devillll@ppp-46-244-176-131.dynamic.mnet-online.de] has joined ##hplusroadmap | 16:23 | |
| -!- ielo [~ielo@cpc9-addl4-2-0-cust229.6-3.cable.virginmedia.com] has quit [Ping timeout: 272 seconds] | 16:26 | |
| -!- Juul [~Juul@108-83-132-49.lightspeed.sntcca.sbcglobal.net] has quit [Ping timeout: 252 seconds] | 16:33 | |
| -!- Juul [~Juul@108-83-132-49.lightspeed.sntcca.sbcglobal.net] has joined ##hplusroadmap | 16:44 | |
| -!- Mariu [Jimmy98@89.41.57.33] has joined ##hplusroadmap | 17:10 | |
| -!- aristarchus [~aristarch@unaffiliated/aristarchus] has joined ##hplusroadmap | 17:21 | |
| -!- soylentbomb [~k@d149-67-118-140.col.wideopenwest.com] has joined ##hplusroadmap | 17:22 | |
| -!- soylentbomb [~k@d149-67-118-140.col.wideopenwest.com] has quit [Changing host] | 17:22 | |
| -!- soylentbomb [~k@unaffiliated/soylentbomb] has joined ##hplusroadmap | 17:22 | |
| -!- Juul [~Juul@108-83-132-49.lightspeed.sntcca.sbcglobal.net] has quit [Ping timeout: 272 seconds] | 17:34 | |
| -!- Juul [~Juul@108-83-132-49.lightspeed.sntcca.sbcglobal.net] has joined ##hplusroadmap | 17:45 | |
| -!- Juul [~Juul@108-83-132-49.lightspeed.sntcca.sbcglobal.net] has quit [Ping timeout: 255 seconds] | 17:50 | |
| -!- panax [~panax@68.200.160.182] has quit [Ping timeout: 258 seconds] | 18:24 | |
| -!- panax [panax@131.247.116.49] has joined ##hplusroadmap | 18:25 | |
| -!- Viper168 [~Viper@unaffiliated/viper168] has quit [Quit: Leaving] | 18:43 | |
| -!- Viper168 [~Viper@node248.19.251.72.1dial.com] has joined ##hplusroadmap | 18:51 | |
| -!- Viper168 [~Viper@node248.19.251.72.1dial.com] has quit [Changing host] | 18:51 | |
| -!- Viper168 [~Viper@unaffiliated/viper168] has joined ##hplusroadmap | 18:51 | |
| -!- Viper168 [~Viper@unaffiliated/viper168] has quit [Read error: Connection reset by peer] | 18:52 | |
| -!- Viper168 [~Viper@node248.19.251.72.1dial.com] has joined ##hplusroadmap | 18:52 | |
| -!- Viper168 [~Viper@node248.19.251.72.1dial.com] has quit [Changing host] | 18:52 | |
| -!- Viper168 [~Viper@unaffiliated/viper168] has joined ##hplusroadmap | 18:52 | |
| -!- Viper168 [~Viper@unaffiliated/viper168] has quit [Read error: Connection reset by peer] | 18:53 | |
| -!- Viper168 [~Viper@node248.19.251.72.1dial.com] has joined ##hplusroadmap | 18:55 | |
| -!- Viper168 [~Viper@node248.19.251.72.1dial.com] has quit [Changing host] | 18:55 | |
| -!- Viper168 [~Viper@unaffiliated/viper168] has joined ##hplusroadmap | 18:55 | |
| kanzure | heh dropboxusercontent.com... guess they were worried about cookies being stolen. | 18:56 |
| kanzure | http://blog.archive.org/2013/04/11/450000-early-journal-articles-now-available/ | 18:57 |
| kanzure | "Internet Archive announces today the addition of over 450,000 journal articles from the JSTOR Early Journal Content collection. Early Journal Content is a selection of pre-1923 materials from more than 350 journals and includes articles in the arts and humanities, economics and politics, and mathematics and other sciences. This content was digitized by JSTOR and is freely available through jstor.org, and it can now also be accessed and ... | 18:57 |
| kanzure | ... downloaded via archive.org." | 18:57 |
| kanzure | "All 2 terabytes of the Early Journal Collection are available for bulk harvesting from the Internet Archive. Web search engines have been indexing the full-text contents of these materials already and, so far, people and robots have downloaded the articles over 400,000 times even before it has been announced. A data bundle including OCR text and metadata is also available from JSTOR’s Data for Research service for free downloading." | 18:58 |
| kanzure | 450k articles = 2 terabytes? | 18:58 |
| kanzure | the OCR thing being "available to download" (dunno if that's real or just "send us an email and we might agree to let you have it") makes me wonder if these 450k articles are just images.. | 18:59 |
| -!- ThomasEgi [~thomas@panda3d/ThomasEgi] has quit [Remote host closed the connection] | 19:01 | |
| -!- Suwap [~devillll@ppp-46-244-176-131.dynamic.mnet-online.de] has quit [Read error: Connection reset by peer] | 19:03 | |
| -!- Suwap [~devillll@ppp-46-244-176-131.dynamic.mnet-online.de] has joined ##hplusroadmap | 19:04 | |
| -!- FooQuuxman [~arik@c-98-215-254-55.hsd1.il.comcast.net] has quit [Ping timeout: 252 seconds] | 19:06 | |
| brownies | http://www.wired.com/wiredenterprise/2013/04/bio-fab-open-source-language | 19:30 |
| brownies | ^ thoughts? | 19:30 |
| yashgaroth | wasn't juul working on biofab | 19:31 |
| kanzure | the article is probably juul's work, but since he's just a lowly superpostdoc he can't take credit. | 19:41 |
| kanzure | (i don't know what his status is, actulaly.) | 19:41 |
| kanzure | *actually | 19:41 |
| brownies | "super postdoc" ... heh. | 19:41 |
| kanzure | Finished in 0.004 seconds | 19:46 |
| kanzure | fastest browser test suite ever | 19:46 |
| -!- aristarchus [~aristarch@unaffiliated/aristarchus] has quit [Quit: Leaving] | 20:08 | |
| -!- jmil [~jmil@hive76/member/jmil] has joined ##hplusroadmap | 21:17 | |
| -!- jmil [~jmil@hive76/member/jmil] has quit [Ping timeout: 256 seconds] | 21:23 | |
| -!- soylentbomb [~k@unaffiliated/soylentbomb] has quit [Ping timeout: 252 seconds] | 21:35 | |
| -!- shyGuy_ [~shyGuy_@CPE485b39d5ac5d-CM000a739b1451.cpe.net.cable.rogers.com] has joined ##hplusroadmap | 22:21 | |
| -!- shyGuy_ [~shyGuy_@CPE485b39d5ac5d-CM000a739b1451.cpe.net.cable.rogers.com] has left ##hplusroadmap ["Leaving"] | 22:47 | |
| --- Log closed Mon Apr 22 00:00:04 2013 | ||
Generated by irclog2html.py 2.15.0.dev0 by Marius Gedminas - find it at mg.pov.lt!