| apanda (~apanda@dhcp-44-187.EECS.Berkeley.EDU) left irc: Remote host closed the connection | 00:06 | |
| jonanin (~jonanin@24-183-50-140.dhcp.mdsn.wi.charter.com) left irc: Read error: Connection reset by peer | 00:07 | |
| Da_Blitz (~Da_Blitz@203.56.250.63) left irc: Ping timeout: 244 seconds | 00:09 | |
| harrison | this is fun. i am compiling pypy from repo using pypy from download | 00:09 |
|---|---|---|
| harrison | i tried 6 months ago but didn't have enough ram | 00:09 |
| apanda (~apanda@dhcp-44-187.EECS.Berkeley.EDU) joined #pypy. | 00:10 | |
| harrison | lo lac | 00:11 |
| Squarepy (~Squarepy@ip82-139-118-42.lijbrandt.net) left irc: Quit: Leaving | 00:15 | |
| kenaan | 12alex_gaynor numpy-dtype-refactor-complex 112d7dd6a415a4 15/pypy/module/micronumpy/: progress, can now see if 2 complexes are equal and read them out of a value | 00:16 |
| fzzzy (~donovan@76-198-130-19.lightspeed.mtvwca.sbcglobal.net) left irc: Quit: fzzzy | 00:17 | |
| apanda (~apanda@dhcp-44-187.EECS.Berkeley.EDU) left irc: Remote host closed the connection | 00:17 | |
| fzzzy (~donovan@76-198-130-19.lightspeed.mtvwca.sbcglobal.net) joined #pypy. | 00:18 | |
| apanda (~apanda@dhcp-44-187.EECS.Berkeley.EDU) joined #pypy. | 00:18 | |
| apanda (~apanda@dhcp-44-187.EECS.Berkeley.EDU) left irc: Ping timeout: 244 seconds | 00:22 | |
| rekamso (~textual@c-71-197-238-154.hsd1.wa.comcast.net) joined #pypy. | 00:32 | |
| lucian (~lucian@cpc1-newc15-2-0-cust84.gate.cable.virginmedia.com) left irc: Ping timeout: 240 seconds | 00:34 | |
| ousado_ (~ousado@hmbg-5f762bbb.pool.mediaWays.net) joined #pypy. | 00:38 | |
| fzzzy (~donovan@76-198-130-19.lightspeed.mtvwca.sbcglobal.net) left irc: Quit: fzzzy | 00:39 | |
| tilgovi (~randall@couchdb/developer/tilgovi) left irc: Ping timeout: 240 seconds | 00:41 | |
| ousado (~ousado@unaffiliated/ousado) left irc: Ping timeout: 258 seconds | 00:42 | |
| bbot2 | 4Failure: 15http://buildbot.pypy.org/builders/pypy-c-jit-linux-x86-64/builds/612 [12hakanardo, jit-targets] | 00:47 |
| tlynn (~tlynn@cpc6-cmbg14-2-0-cust121.5-4.cable.virginmedia.com) left irc: Ping timeout: 240 seconds | 00:50 | |
| derdon (~derdon@p5DE8947D.dip.t-dialin.net) left irc: Remote host closed the connection | 00:56 | |
| tilgovi (~randall@c-98-210-155-124.hsd1.ca.comcast.net) joined #pypy. | 00:59 | |
| tilgovi (~randall@c-98-210-155-124.hsd1.ca.comcast.net) left irc: Changing host | 00:59 | |
| tilgovi (~randall@couchdb/developer/tilgovi) joined #pypy. | 00:59 | |
| maxyz (~maxy@186.18.94.35) joined #pypy. | 01:12 | |
| maxyz (~maxy@186.18.94.35) left irc: Ping timeout: 252 seconds | 01:21 | |
| bbot2 | 4Failure: 15http://buildbot.pypy.org/builders/own-macosx-x86-32/builds/734 | 01:21 |
| maxyz (~maxy@186.18.94.35) joined #pypy. | 01:27 | |
| harrison (~sr@adsl-69-209-213-114.dsl.chcgil.sbcglobal.net) left irc: Remote host closed the connection | 01:34 | |
| Da_Blitz (~Da_Blitz@203.56.250.63) joined #pypy. | 01:35 | |
| bbot2 | Started: 15http://buildbot.pypy.org/builders/own-linux-x86-32/builds/1877 | 01:38 |
| bbot2 | Started: 15http://buildbot.pypy.org/builders/pypy-c-jit-macosx-x86-64/builds/294 | 01:38 |
| bbot2 | Started: 15http://buildbot.pypy.org/builders/pypy-c-jit-linux-x86-64/builds/613 | 01:38 |
| bbot2 | Started: 15http://buildbot.pypy.org/builders/pypy-c-jit-linux-x86-32/builds/1142 | 01:38 |
| bbot2 | Started: 15http://buildbot.pypy.org/builders/pypy-c-app-level-linux-x86-64/builds/632 | 01:38 |
| bbot2 | Started: 15http://buildbot.pypy.org/builders/pypy-c-Ojit-no-jit-linux-x86-32/builds/817 | 01:38 |
| bbot2 | Started: 15http://buildbot.pypy.org/builders/pypy-c-app-level-linux-x86-32/builds/1459 | 01:38 |
| bbot2 | Started: 15http://buildbot.pypy.org/builders/own-linux-x86-64/builds/738 | 01:38 |
| bbot2 | Started: 15http://buildbot.pypy.org/builders/pypy-c-jit-win-x86-32/builds/283 | 01:38 |
| maxyz (~maxy@186.18.94.35) left irc: Ping timeout: 245 seconds | 01:49 | |
| Vorpal (~AnMaster@unaffiliated/anmaster) left irc: Ping timeout: 240 seconds | 01:51 | |
| maxyz (~maxy@186.18.94.35) joined #pypy. | 01:53 | |
| bbot2 | 4Failure: 15http://buildbot.pypy.org/builders/pypy-c-jit-win-x86-32/builds/283 | 02:05 |
| asmeurer__ (~asmeurer@dhcp-baca-230.resnet.nmt.edu) left irc: Quit: asmeurer__ | 02:10 | |
| asmeurer__ (~asmeurer@70.56.139.215) joined #pypy. | 02:20 | |
| asmeurer__ (~asmeurer@70.56.139.215) left irc: Ping timeout: 252 seconds | 02:25 | |
| jcookeman (~jcookeman@78-86-161-32.zone2.bethere.co.uk) left irc: Ping timeout: 244 seconds | 02:26 | |
| bbot2 | 3Success: 15http://buildbot.pypy.org/builders/jit-benchmark-linux-x86-32/builds/964 | 02:29 |
| Kaskuka (~John@osbk-4db16b6e.pool.mediaWays.net) left irc: Ping timeout: 240 seconds | 02:42 | |
| Kaskuka (~John@osbk-4db17372.pool.mediaWays.net) joined #pypy. | 02:42 | |
| nedbat (~nedbat@python/psf/nedbat) joined #pypy. | 02:46 | |
| durin42 (~durin@173-15-14-174-Illinois.hfc.comcastbusiness.net) left irc: Ping timeout: 244 seconds | 03:04 | |
| durin42 (~durin@adium/durin42) joined #pypy. | 03:06 | |
| lac_ (~quassel@c-c4c4e055.1321-1-64736c11.cust.bredbandsbolaget.se) joined #pypy. | 03:11 | |
| lac (~quassel@c-c4c4e055.1321-1-64736c11.cust.bredbandsbolaget.se) left irc: Ping timeout: 244 seconds | 03:11 | |
| Da_Blitz (~Da_Blitz@203.56.250.63) left irc: Ping timeout: 244 seconds | 03:40 | |
| Da_Blitz (~Da_Blitz@203.56.250.63) joined #pypy. | 03:42 | |
| kenaan | 12ned nedbat-sandbox 11d06f77ba878f 15/pypy/translator/sandbox/test/test_sandlib.py: Add a test of opening lots of real files also. | 03:47 |
| kenaan | 12ned nedbat-sandbox 11ba523ca5ff05 15/pypy/translator/sandbox/test/test_sandlib.py: Add a sandlib test for lseek | 03:47 |
| mitchellh (u4018@gateway/web/irccloud.com/x-ssbxjvlfhsoopzbw) left #pypy. | 03:48 | |
| bbot2 | 4Finished ??: 15http://buildbot.pypy.org/builders/pypy-c-jit-macosx-x86-64/builds/294 | 03:50 |
| nedbat (~nedbat@python/psf/nedbat) left irc: Ping timeout: 248 seconds | 03:57 | |
| apanda (~apanda@c-24-7-101-62.hsd1.ca.comcast.net) joined #pypy. | 04:03 | |
| bbot2 | 3Success: 15http://buildbot.pypy.org/builders/jit-benchmark-linux-x86-64/builds/160 | 04:10 |
| bbot2 | 4Failure: 15http://buildbot.pypy.org/builders/own-linux-x86-32/builds/1877 | 05:11 |
| bbot2 | Started: 15http://buildbot.pypy.org/builders/pypy-c-jit-macosx-x86-64/builds/295 | 05:19 |
| asmeurer__ (~asmeurer@70.56.139.215) joined #pypy. | 05:24 | |
| rekamso (~textual@c-71-197-238-154.hsd1.wa.comcast.net) left irc: Quit: Textual IRC Client: http://www.textualapp.com/ | 05:26 | |
| gutworth (~benjamin@dhcp-128-36-84-91.central.yale.edu) left irc: Ping timeout: 252 seconds | 05:39 | |
| Da_Blitz (~Da_Blitz@203.56.250.63) left irc: Ping timeout: 252 seconds | 05:52 | |
| Da_Blitz (~Da_Blitz@203.56.250.63) joined #pypy. | 05:59 | |
| aleksi (~aleksi@85.235.191.82) joined #pypy. | 06:04 | |
| bbot2 | 4Failure: 15http://buildbot.pypy.org/builders/pypy-c-app-level-linux-x86-32/builds/1459 | 06:13 |
| neunon (~snoonan@unaffiliated/neunon) joined #pypy. | 06:13 | |
| bbot2 | 4Failure: 15http://buildbot.pypy.org/builders/pypy-c-app-level-linux-x86-64/builds/632 | 06:21 |
| kennethreitz (~kennethre@c-24-127-96-129.hsd1.va.comcast.net) left irc: Quit: Textual IRC Client: http://www.textualapp.com/ | 06:34 | |
| bbot2 | 4Failure: 15http://buildbot.pypy.org/builders/pypy-c-Ojit-no-jit-linux-x86-32/builds/817 | 06:36 |
| Nick change: Gulopine -> Gulaway | 06:36 | |
| bbot2 | 4Failure: 15http://buildbot.pypy.org/builders/pypy-c-jit-linux-x86-64/builds/613 | 06:41 |
| amaury_ (~amaury_@46-127-23-192.dynamic.hispeed.ch) joined #pypy. | 06:42 | |
| bbot2 | 4Failure: 15http://buildbot.pypy.org/builders/pypy-c-jit-linux-x86-32/builds/1142 | 06:44 |
| hruske (~Gasper@188-230-156-183.dynamic.t-2.net) joined #pypy. | 06:49 | |
| cheyuni` (~user@125.129.20.246) joined #pypy. | 06:53 | |
| cheyuni` (~user@125.129.20.246) left irc: Remote host closed the connection | 06:55 | |
| amaury_ (~amaury_@46-127-23-192.dynamic.hispeed.ch) left irc: Read error: Operation timed out | 06:57 | |
| kenaan | 12hakanardo jit-targets 1116988785671d 15/pypy/jit/backend/x86/test/test_regalloc.py: fix test | 06:58 |
| mcdonc (~mcdonc@ip72-209-213-54.dc.dc.cox.net) left irc: Remote host closed the connection | 07:07 | |
| mcdonc (~mcdonc@ip72-209-213-54.dc.dc.cox.net) joined #pypy. | 07:08 | |
| kirma (~jkirma@kekkonen.cs.hut.fi) joined #pypy. | 07:11 | |
| mcdonc (~mcdonc@ip72-209-213-54.dc.dc.cox.net) left irc: Remote host closed the connection | 07:16 | |
| mcdonc (~mcdonc@ip72-209-213-54.dc.dc.cox.net) joined #pypy. | 07:17 | |
| mvt (~mvantelli@87.213.45.85) joined #pypy. | 07:19 | |
| mcdonc (~mcdonc@ip72-209-213-54.dc.dc.cox.net) left irc: Remote host closed the connection | 07:21 | |
| mcdonc (~mcdonc@ip72-209-213-54.dc.dc.cox.net) joined #pypy. | 07:22 | |
| fijal | hi | 07:30 |
| kennethreitz (~kennethre@c-24-127-96-129.hsd1.va.comcast.net) joined #pypy. | 07:32 | |
| brutal_chaos (~chaos@francium.electricgrey.com) joined #pypy. | 07:33 | |
| brutal_chaos | What is wrong with this: python2.7 ./pypy/translator/goal/translate.py -Ojit --stackless ./pypy/translator/goal/targetpypystandalone.py --withmod-bz2 --withmod-_minimal_curses --withmod-pyexpat --withmod-_ssl | 07:37 |
| brutal_chaos | can I not have stackless and jit? | 07:38 |
| Garen | brutal_chaos: It's enabled by default now, so no need to add an option for it unless you want to exclude it. | 07:42 |
| brutal_chaos | Garen: which is default? | 07:44 |
| Garen | brutal_chaos: http://morepypy.blogspot.com/2011/11/pypy-17-widening-sweet-spot.html ... "PyPy now comes with stackless features enabled by default" | 07:48 |
| brutal_chaos | oh, very cool. thanks | 07:48 |
| brutal_chaos | has anyone had success building pypy with llvm/clang? | 07:49 |
| Garen | brutal_chaos: Somewhere I've heard it works. :) Not as cross-platform as gcc though I bet. | 07:52 |
| tilgovi (~randall@couchdb/developer/tilgovi) left irc: Remote host closed the connection | 07:55 | |
| Vorpal (~AnMaster@unaffiliated/anmaster) joined #pypy. | 07:57 | |
| ronny | brutal_chaos: it need to be built with the shadowstack gc rootfinder to work on non-gcc compilers | 07:58 |
| ronny | Garen: actually more cross-platform, just a bit slower | 07:58 |
| Garen | ronny: clang/llvm doesn't work nearly well on windows for me | 08:00 |
| Garen | ronny: Where does it work better? | 08:01 |
| ronny | Garen: the main difference between the default (asmgcc) and shaodowstack is, that one us plain c, the other is gcc magic | 08:01 |
| Garen | oic | 08:02 |
| pedronis (~pedronis@73-53.195-178.cust.bluewin.ch) left irc: Remote host closed the connection | 08:07 | |
| pedronis (~pedronis@73-53.195-178.cust.bluewin.ch) joined #pypy. | 08:09 | |
| Ademan (~dan@adsl-71-141-224-79.dsl.snfc21.pacbell.net) left irc: Quit: leaving | 08:13 | |
| etrepum (~bob@75-101-96-144.dsl.static.sonic.net) joined #pypy. | 08:14 | |
| brutal_chaos | well poo. I didn't build with shadowstack (nor did I include tests). What can I do to see if pypy is failing? | 08:17 |
| fijal | ronny: it's not gcc magic | 08:24 |
| fijal | it works on VC++ | 08:24 |
| fijal | it's just that we postprocess C-compiler generated assembler | 08:24 |
| fijal | which is not very cross-compiler | 08:24 |
| fijal | brutal_chaos: did it build with asmgcc and clang? | 08:25 |
| brutal_chaos | it is still building | 08:25 |
| brutal_chaos | just past rtyper 20% | 08:25 |
| brutal_chaos | fijal: ^ | 08:25 |
| fijal | ah no | 08:25 |
| fijal | it'll crash later on | 08:25 |
| fijal | much later on | 08:25 |
| brutal_chaos | :( | 08:25 |
| fijal | I would cancel it and start with shadowstack | 08:26 |
| fijal | we're trying to phase out asmgcc, but it's kinda hard to match the performance | 08:26 |
| brutal_chaos | so what is asmgcc? | 08:26 |
| brutal_chaos | also, at what point will the build fail? | 08:26 |
| kkris (~kris@80-123-46-92.adsl.highway.telekom.at) joined #pypy. | 08:30 | |
| brutal_chaos | fijal: ^ | 08:30 |
| fijal | at C compilation stage | 08:30 |
| brutal_chaos | fijal: what is asmgcc though? | 08:30 |
| fijal | ok, so do you know the problem of finding GC stack roots? | 08:30 |
| brutal_chaos | no | 08:30 |
| fijal | so do you know what GC roots are? | 08:32 |
| fijal | those are things that can reference alive objects | 08:32 |
| fijal | so alive objects can be referenced from globals, from stack and so on | 08:33 |
| fijal | right? | 08:33 |
| brutal_chaos | right | 08:33 |
| fijal | when you're doing garbage collection you have to start at roots and walk the entire object graph until you're done | 08:33 |
| fijal | everything else is dead | 08:33 |
| fijal | so C does not provide you a way to find all pointers on the stack | 08:33 |
| fijal | you can walk the entire stack, but you might run into an int that looks like a pointer but is actually not | 08:33 |
| fijal | and that'll keep something artificially alive for example | 08:34 |
| fijal | and causes other troubles | 08:34 |
| fijal | with me so far? | 08:34 |
| brutal_chaos | yeah | 08:34 |
| fijal | ok, so there are various strategies how to track pointers on the stack | 08:35 |
| fijal | it's also not any pointers - we want only pointers that point to GC managed structures, so they're of an appropriate type | 08:35 |
| fijal | type information is gone by the time you generate assembler, so the info is not there at all | 08:35 |
| fijal | so we have two strategies | 08:35 |
| fijal | one is called "shadowstack" where you copy the entire stack into the heap and you can look what's there | 08:36 |
| fijal | this is obviously costly | 08:36 |
| fijal | the other one is to have hints that compile to assembler hints | 08:36 |
| fijal | and postprocess assembler generated by the C compiler | 08:36 |
| fijal | like gcc -S | 08:36 |
| fijal | and then compile it only | 08:36 |
| fijal | this is obviously faster, but also obviously less cross-platform | 08:36 |
| fijal | in fact it's so much pain to maintain we would like to get rid of it | 08:36 |
| JaRoel|4d (~jaroel|4d@office.fourdigits.nl) joined #pypy. | 08:37 | |
| fijal | even at the some % of performance costs | 08:37 |
| fijal | but I guess right now it's 10% and we don't want to do it | 08:37 |
| fijal | brutal_chaos: do I make any sense? | 08:37 |
| brutal_chaos | fijal: yes, thanks. | 08:37 |
| fijal | np :) | 08:37 |
| hruske (~Gasper@188-230-156-183.dynamic.t-2.net) left irc: Quit: Leaving | 08:37 | |
| brutal_chaos | fijal: clang has a -S and generates assembly. can't that be post processed? | 08:43 |
| kenaan | 12bivab arm-backend-2 117873941fd23b 15/pypy/: (arigo, bivab): move this hack to rmmap and make it a bit cleaner | 08:43 |
| kenaan | 12bivab arm-backend-2 11b485870748d3 15/pypy/jit/backend/test/runner_test.py: (arigo, bivab): test the presence of INSTANCE_PTR_NE and INSTANCE_PTR_EQ in the backend | 08:43 |
| kenaan | 12bivab arm-backend-2 1183649b8410c7 15/pypy/jit/backend/arm/regalloc.py: (arigo, bivab): add instance_ptr_... also in the register allocator | 08:43 |
| kenaan | 12bivab arm-backend-2 118fbef9af526f 15/pypy/jit/backend/arm/: (arigo, bivab): clear CPU cache everty time instructions are written to memory and put breakpoints at the loc... | 08:43 |
| fijal | brutal_chaos: but it's different assembler | 08:44 |
| bivab (~david@fwstups.cs.uni-duesseldorf.de) joined #pypy. | 08:44 | |
| fijal | sure it can but it requires effort | 08:44 |
| brutal_chaos | ah | 08:44 |
| brutal_chaos | fijal: that makes sense | 08:44 |
| bbot2 | 4Failure: 15http://buildbot.pypy.org/builders/own-linux-x86-64/builds/738 | 08:59 |
| asmeurer__ (~asmeurer@70.56.139.215) left irc: Quit: asmeurer__ | 09:07 | |
| lucian (~lucian@93-97-174-114.zone5.bethere.co.uk) joined #pypy. | 09:15 | |
| antocuni (~antocuni@host129-122-dynamic.16-79-r.retail.telecomitalia.it) joined #pypy. | 09:16 | |
| lucian (~lucian@93-97-174-114.zone5.bethere.co.uk) left irc: Client Quit | 09:16 | |
| lucian (~lucian@93-97-174-114.zone5.bethere.co.uk) joined #pypy. | 09:17 | |
| __pv (pav@xen.ihme.org) left irc: Quit: leaving | 09:17 | |
| G2P (~G2P@fw-asn1.ornis.com) joined #pypy. | 09:17 | |
| __pv (pav@xen.ihme.org) joined #pypy. | 09:18 | |
| Garen_ (garen@69.76.18.3) joined #pypy. | 09:25 | |
| uche (~uche@70-90-116-204-BusName-summit.co.hfc.comcastbusiness.net) left irc: Ping timeout: 252 seconds | 09:25 | |
| Da_Blitz (~Da_Blitz@203.56.250.63) left irc: Ping timeout: 240 seconds | 09:26 | |
| Da_Blitz (~Da_Blitz@203.56.250.63) joined #pypy. | 09:27 | |
| brutal_chaos | how do I use the shadowstack gc? | 09:28 |
| Garen (garen@69.76.18.3) left irc: Ping timeout: 258 seconds | 09:28 | |
| brutal_chaos | where in this line do I add it?: python2.7 ./pypy/translator/goal/translate.py -Ojit ./pypy/translator/goal/targetpypystandalone.py --withmod-bz2 --withmod-_minimal_curses --withmod-pyexpat --withmod-_ssl | 09:28 |
| fijal | ok | 09:29 |
| fijal | why are you using all those options, can you tell me? | 09:29 |
| fijal | you know that everything after target is redundant? | 09:29 |
| fijal | they're on by default | 09:29 |
| fijal | it's translate.py --gcrootfinder=shadowstack I believe | 09:30 |
| brutal_chaos | fijal: ah, it's the line gentoo's ebuild generates as part of the ebuild | 09:30 |
| fijal | but this and -Ojit are the only options you're interested in | 09:30 |
| brutal_chaos | as part of the build process* with its USE flags. | 09:30 |
| fijal | I know and gentoo package maintainer claims this is because gentoo users like tinkering with options | 09:30 |
| fijal | so he passed some for a good measure | 09:30 |
| brutal_chaos | so, that is why | 09:32 |
| fijal | ok | 09:32 |
| jell | next person asking about broken old ebuild file :) | 09:33 |
| brutal_chaos | old? broken? no | 09:33 |
| jell | last week there was also such person ;) | 09:33 |
| brutal_chaos | missing some features, yes | 09:33 |
| brutal_chaos | I've emailed the maintainer to add the option, hopefully they'll do so. | 09:34 |
| fijal | brutal_chaos: hey hey hey | 09:34 |
| fijal | brutal_chaos: wait a sec | 09:34 |
| fijal | why do you want to use shadowstack? | 09:34 |
| brutal_chaos | fijal: yes? | 09:34 |
| fijal | just because you want to use clang? | 09:34 |
| fijal | it *will be* slower | 09:34 |
| brutal_chaos | fijal: llvm/clang | 09:34 |
| fijal | yeah | 09:35 |
| fijal | but why not just compile using gcc? | 09:35 |
| fijal | I mean don | 09:35 |
| jell | fijal: nice catch ;) | 09:35 |
| fijal | t change package options until you did benchmarks and made sure it works at least | 09:35 |
| fijal | we don't even have such a buildbot | 09:35 |
| brutal_chaos | fijal: lol, the only package option I am adding is the ability to use shadowstack so pypy builds with clang. | 09:36 |
| fijal | we don't even have such a buildbot | 09:36 |
| fijal | uh sorry | 09:36 |
| fijal | so how would that look like? | 09:36 |
| brutal_chaos | what do you mean? | 09:36 |
| brutal_chaos | how would I build with clang? | 09:37 |
| fijal | how does that look like? | 09:37 |
| fijal | will it still compile with asmgcc on gcc? | 09:37 |
| brutal_chaos | oh, yeah, definitely | 09:37 |
| brutal_chaos | the only way clang is used, is if the user sets an environment variable to use clang instead of gcc. | 09:37 |
| brutal_chaos | gcc is default and the package option doesn't change your compiler at all | 09:38 |
| brutal_chaos | just includes --gcrootfinder=shadowstack to the translation line | 09:38 |
| brutal_chaos | that's all | 09:38 |
| brutal_chaos | I just want to have --gcrootfinder=shadowstack available so if I change my compiler to clang, I can build pypy | 09:39 |
| brutal_chaos | fijal: btw, how can I run benchmarks? | 09:40 |
| brutal_chaos | once the build completes, I'd like to see the results. | 09:40 |
| stakkars_ (~tismer@i59F7379D.versanet.de) joined #pypy. | 09:45 | |
| fijal | brutal_chaos: check out pypy/benchmarks | 09:45 |
| fijal | and then run runner.py with options | 09:45 |
| fijal | --help is a way to find them | 09:45 |
| fijal | btw, I would like to at least emit a warning to a user | 09:45 |
| fijal | "this is going to be slower" | 09:45 |
| brutal_chaos | fijal: I am not the maintainer, but I'll let them know. | 09:46 |
| fijal | cool | 09:47 |
| fijal | I mean yes, exploding is not good | 09:47 |
| brutal_chaos | fijal: I simply created my own ebuild that ads the option in the mean time though. | 09:50 |
| brutal_chaos | (personal use, not for distribution) | 09:50 |
| arigato (~arigo@fwstups.cs.uni-duesseldorf.de) joined #pypy. | 09:51 | |
| ojii (~ojii@40-34.60-188.cust.bluewin.ch) joined #pypy. | 09:52 | |
| brutal_chaos | woot, ebuild operational. Attempting a clang/shadowstack build. | 09:58 |
| brutal_chaos | after it completes (i need sleep, so sometime tomorrow), I'll run benchmarks. fijal, how would you like me to submit them? | 09:58 |
| cpets (c358b60a@gateway/web/freenode/ip.195.88.182.10) joined #pypy. | 09:59 | |
| fijal | brutal_chaos: I think if you're aware of outcomes, it's good enough to report them to ebuild maintainer | 09:59 |
| fijal | you can also post them on pypy-dev | 09:59 |
| brutal_chaos | mailing list? | 10:01 |
| kennethreitz (~kennethre@c-24-127-96-129.hsd1.va.comcast.net) left irc: Ping timeout: 244 seconds | 10:02 | |
| fijal | yes | 10:02 |
| fijal | it would be interesting to see | 10:02 |
| _aleksi (~aleksi@85.235.191.82) joined #pypy. | 10:06 | |
| brutal_chaos | fijal: alright, the build is going and I am off to bed. I'll post the results to the ml once I get a chance. | 10:06 |
| fijal | brutal_chaos: cool, thanks! | 10:07 |
| brutal_chaos | fijal: can these be hidden? [version:WARNING] Errors getting Mercurial information: Not running from a Mercurial repository! | 10:09 |
| fijal | it would be cool if you had this info | 10:09 |
| fijal | because that renders quite a few sys attributes useless | 10:09 |
| aleksi (~aleksi@85.235.191.82) left irc: Ping timeout: 240 seconds | 10:09 | |
| brutal_chaos | https://bitbucket.org/pypy/pypy/get/release-1.7.tar.bz2 is where the source comes from | 10:10 |
| brutal_chaos | fijal: ^ | 10:11 |
| fijal | right | 10:11 |
| fijal | good point | 10:11 |
| squiddy (~squiddy@g224193176.adsl.alicedsl.de) joined #pypy. | 10:12 | |
| brutal_chaos | k, now I am off to bed. :) | 10:12 |
| lizardo (~lizardo@189.2.128.130) joined #pypy. | 10:18 | |
| fijal | arigato: hi armin | 10:19 |
| ramusara (~ramusara@220.156.210.236.user.e-catv.ne.jp) joined #pypy. | 10:20 | |
| arigato | hi | 10:21 |
| ronny | hmm | 10:27 |
| CIA-13 | 03ronny 07roundup * 10#952/try to use .hg_archival if .hg is not availiable: | 10:28 |
| CIA-13 | [new] that way people getting the bitbucket tarballs will have correct vcs bases sys | 10:28 |
| CIA-13 | attributes * 14https://bugs.pypy.org/issue952 | 10:28 |
| Da_Blitz (~Da_Blitz@203.56.250.63) left irc: Ping timeout: 252 seconds | 10:29 | |
| Da_Blitz (~Da_Blitz@203.56.250.63) joined #pypy. | 10:31 | |
| fijal | arigato: your test runs with no end in sight | 10:31 |
| fijal | arigato: the one that you commited for the missing write barrier for setinteriorfield | 10:32 |
| fijal | I wrote a fix, looks like this: | 10:32 |
| fijal | http://paste.pocoo.org/show/516603/ | 10:33 |
| kenaan | 12arigo default 111f76b745f839 15/pypy/jit/backend/x86/test/test_zrpy_gc.py: Oups, sorry. | 10:35 |
| fijal | can I commit my fix? | 10:36 |
| mkerrin (~mkerrin@wg1-nat.fwgal01.deri.ie) joined #pypy. | 10:37 | |
| arigato | fijal: yes, please do, if it passes my corrected test | 10:37 |
| Action: arigato tries to answer on irc via hg commit messages | 10:37 | |
| fijal | nice haiku :) | 10:37 |
| fijal | AttributeError: 'NoneType' object has no attribute 'foo' | 10:38 |
| fijal | you did not run the test, did you :) | 10:38 |
| Action: fijal fixes | 10:38 | |
| arigato | uh? | 10:38 |
| arigato | argh | 10:38 |
| arigato | well kill the line that says "check that..." | 10:38 |
| verte (~verte@python/site-packages/verte) joined #pypy. | 10:38 | |
| verte (~verte@python/site-packages/verte) left irc: Client Quit | 10:38 | |
| fijal | ronny: can you elaborate/come up with a patch? | 10:39 |
| ronny | fijal: its on my list, i have a basic idea, its not that different from what hgdistver does | 10:44 |
| Action: fijal keeps running | 10:44 | |
| fijal | ronny: well, sure, but the bug report does not convey your basic idea in any sense | 10:44 |
| fijal | it's of no information to anyone who is not you | 10:44 |
| fijal | when I look into bugs for release I would be inclined to close this one with "wtf" | 10:44 |
| fijal | what is .hg_archival? | 10:45 |
| fijal | a thing you just invented or something else? | 10:45 |
| fijal | a bug saying "our release tarballs give warnings about hg information not being there" | 10:45 |
| fijal | is by far more informative IMO | 10:45 |
| ronny | fijal: its not our own release tarballs, its the source tarballs from bitbucket | 10:46 |
| fijal | well whatever | 10:46 |
| ronny | fijal: and those have .hg_archival | 10:46 |
| fijal | it's the same thing | 10:46 |
| kenaan | 12fijal default 116b51800a7ece 15/pypy/jit/backend/: A test and a fix | 10:47 |
| CIA-13 | 03fijal 07roundup * 10#951/segfault in nested array/dict creation: [resolved] Fixed in 6b51800a7ece thanks for reporting! * 14https://bugs.pypy.org/issue951 | 10:48 |
| fijal | arigato: thanks for finding this :) | 10:48 |
| arigato | :-) | 10:48 |
| Action: fijal ponders what to do | 10:50 | |
| bbot2 | 4Failure: 15http://buildbot.pypy.org/builders/pypy-c-jit-macosx-x86-64/builds/295 | 10:50 |
| fijal | arigato: btw, how insane is the idea of having refcounting for really large objects? | 10:51 |
| mikefc | fijal: may i ask what the problem was for that segfault? | 10:52 |
| fijal | mikefc: sure, a missing write barrier for setinteriorfield | 10:52 |
| fijal | :-) | 10:52 |
| fijal | do you want to know what that means? | 10:52 |
| mikefc | ahh. that makes about as much sense as i thought it would :) Slowing trying to figure out how to do *anything* in pypy | 10:53 |
| mikefc | now I'll have to build my own version rather than a DLd one. | 10:53 |
| ronny | the next ngihtly will include the change | 10:54 |
| ronny | what are you trying to do? | 10:54 |
| fijal | mikefc: or wait till tomorrow | 10:55 |
| fijal | mikefc: do you know what is a write barrier? | 10:55 |
| mikefc | ronny: I just had a complex data structure holding a whole tonne of crap. basically holding a DB in memory | 10:55 |
| fijal | I can seriously explain what I meant | 10:55 |
| fijal | (if you want to know) | 10:55 |
| mikefc | i didn't realise there were nightlies. I need more hours in the day. | 10:55 |
| fijal | http://buildbot.pypy.org/nightly/trunk/ | 10:56 |
| fijal | by tomorrow EU time | 10:56 |
| mikefc | fijal: i do want to know, but i may not be able to take it in. I like the idea of python in python. and of having a good numpy. but my grasp of JITs and what-not is quite quite limited. | 10:56 |
| fijal | hahaha :) | 10:57 |
| fijal | never stop believing in yourself | 10:57 |
| arigato | fijal: to ensure quick collection? that's unlikely to work, because there are probably indirectly references from non-refcounted small objects, like the frame | 10:57 |
| fijal | arigato: not in the jitted case | 10:57 |
| fijal | also, even in the normal case | 10:58 |
| fijal | I'm usually talking about such examples: | 10:58 |
| arigato | well, then the W_XxxObject that holds the reference to the big array | 10:58 |
| fijal | http://paste.pocoo.org/show/516609/ | 10:59 |
| fijal | right | 10:59 |
| fijal | but then the big object would be | 10:59 |
| fijal | W_XxxRefcountedObject | 10:59 |
| fijal | like a numpy array | 10:59 |
| mikefc | fijal: thanks for the fix. | 10:59 |
| Action: mikefc now readies to post a speed bug | 10:59 | |
| fijal | so in this case the frame would remove the refcount just before storing a new one | 11:00 |
| fijal | arigato: still a bit too disruptive idea? | 11:00 |
| fijal | or simply won't work? | 11:00 |
| arigato | fijal: trying to understand what you exactly mean: do you really mean "W_Refcounted", as in a subclass of W_Root? | 11:01 |
| fijal | I mean something like | 11:01 |
| fijal | RefoucountedNumpyArray | 11:01 |
| fijal | which is a subclass of W_Root | 11:01 |
| fijal | which itself has a reference to large *raw* array | 11:01 |
| arigato | you can't put a refcount on a subclass of W_Root, without adding refcount-management code *everywhere* that manipulates W_Root | 11:02 |
| fijal | I don't want to have an explicit refcount | 11:02 |
| fijal | just an implicit one, like gc=refcount strategy | 11:02 |
| fijal | ah | 11:02 |
| fijal | I understand what you mean | 11:03 |
| arigato | what is an "implicit reference count"? | 11:03 |
| fijal | you mean that each place that handles W_Root objects would have to do "is this a refcounted object?" question? | 11:03 |
| fijal | right? | 11:03 |
| arigato | yes | 11:03 |
| fijal | ok | 11:03 |
| fijal | that would suck | 11:03 |
| arigato | yes :-) | 11:03 |
| fijal | you know what's the issue btw? | 11:04 |
| fijal | why am I bringing it up? | 11:04 |
| arigato | no | 11:04 |
| fijal | because you have things like this: | 11:04 |
| fijal | a = some_numpy_array() | 11:04 |
| fijal | and in a loop | 11:04 |
| CIA-13 | 03mikefc 07roundup * 10#953/defaultdict behaviour inconsistent with cpython: | 11:04 |
| CIA-13 | [new] While trying to recall how defaultdicts work, I found that | 11:04 |
| CIA-13 | collections.defaultdict([]) and collections.defaultdict({}) cause a TypeError | 11:04 |
| CIA-13 | ... * 14https://bugs.pypy.org/issue953 | 11:04 |
| fijal | a = <some expression based on a> | 11:04 |
| fijal | like | 11:04 |
| fijal | a = (a + a) * (a + a) | 11:04 |
| fijal | for example | 11:04 |
| fijal | no | 11:04 |
| fijal | w | 11:04 |
| fijal | no, it's even more often | 11:05 |
| fijal | a[:] = (a + a) * (a + a) | 11:05 |
| fijal | so you replace items of a | 11:05 |
| fijal | while you're doing that, you invalidate all the expressions that are based on a | 11:05 |
| pedronis (~pedronis@73-53.195-178.cust.bluewin.ch) left irc: Ping timeout: 240 seconds | 11:05 | |
| fijal | so you need now to compute | 11:05 |
| fijal | a + a | 11:05 |
| fijal | and (a + a) * (a + a) | 11:05 |
| fijal | while computing a+a is pointless | 11:05 |
| fijal | but you don't know | 11:05 |
| arigato | I see | 11:06 |
| CIA-13 | 03fijal 07roundup * 10#953/defaultdict behaviour inconsistent with cpython: | 11:06 |
| CIA-13 | [chatting] This should probably be reported upstream as a CPython missing test case? Do they | 11:06 |
| CIA-13 | fail silently or do they actually succeed? * 14https://bugs.pypy.org/issue953 | 11:06 |
| fijal | arigato: I don't quite know how to attack that | 11:07 |
| fijal | maybe we can do JIT-based refcounting? | 11:07 |
| arigato | maybe with the help of the JIT, indeed | 11:07 |
| arigato | if the line a[:] = (a+a)*(a+a) is itself in a JITted loop | 11:08 |
| arigato | then the JIT can notice that the object b=(a+a) never escapes | 11:08 |
| arigato | still not obvious... | 11:10 |
| pedronis (~pedronis@73-53.195-178.cust.bluewin.ch) joined #pypy. | 11:10 | |
| kenaan | 12l.diekmann type-specialized-instances 11591a337d364f 15/pypy/objspace/std/test/test_mapdict.py: fixed mapdict tests | 11:11 |
| Squarepy (~Squarepy@wcw-staff-214-150.wireless.uva.nl) joined #pypy. | 11:11 | |
| Nick change: elliottcable -> elliottcable|hat | 11:12 | |
| Squarepy (~Squarepy@wcw-staff-214-150.wireless.uva.nl) left irc: Changing host | 11:16 | |
| Squarepy (~Squarepy@unaffiliated/squarepy) joined #pypy. | 11:16 | |
| Action: arigato does pyglet today, to replace pygame in (1) bub-n-bros and (2) dotviewer | 11:16 | |
| fijal | arigato: hooray! | 11:16 |
| fijal | nice zooming.... | 11:17 |
| arigato | :-) | 11:17 |
| fijal | and pypy compatibility | 11:17 |
| arigato | yes | 11:17 |
| fijal | I'm probably surfing today :) | 11:17 |
| CIA-13 | 03mikefc 07roundup * 10#953/defaultdict behaviour inconsistent with cpython: | 11:17 |
| CIA-13 | I'm currently looking at cpython source to see if I can figure what they do. | 11:17 |
| CIA-13 | Output for both pythons included below. | 11:17 |
| CIA-13 | ================ | 11:17 |
| CIA-13 | cpyth ... * 14https://bugs.pypy.org/issue953 | 11:17 |
| djc (~djc@enrai.xavamedia.nl) left irc: Changing host | 11:19 | |
| djc (~djc@gentoo/developer/djc) joined #pypy. | 11:19 | |
| djc (djc@gentoo/developer/djc) left #pypy. | 11:20 | |
| arigato | :-) | 11:20 |
| fijal | which is one of the few outdoor sports that actually work better in bad weather | 11:21 |
| arigato | :-) | 11:22 |
| stakkars_ (~tismer@i59F7379D.versanet.de) left irc: Quit: schnarch | 11:23 | |
| ronny | oh, app_defaultdict looks kind of evil | 11:23 |
| Nick change: elliottcable|hat -> ec | 11:26 | |
| fijal | arigato: so pypy 1.7 is less popular than 1.6 in terms of downloads from bitbucket | 11:26 |
| fijal | however | 11:26 |
| fijal | I suppose it's at least partly because people ship it these days | 11:27 |
| mikefc | can i mention pypy in a cpython bug report saying "add the testcase so pypy knows how to behave"? | 11:29 |
| mikefc | not sure if that is considered rude or something... | 11:29 |
| mikefc | after living through the ffmpeg/libav split i worry too much... | 11:30 |
| fijal | mikefc: sure that's fine | 11:30 |
| Dulak | mikefc: the cpython guys are not anti-pypy really | 11:30 |
| fijal | most of them anyway | 11:30 |
| lac_ (~quassel@c-c4c4e055.1321-1-64736c11.cust.bredbandsbolaget.se) left irc: Read error: Connection reset by peer | 11:30 | |
| Dulak | Well, there are exceptions to every rule but it's not exactly the holy war you'd get from say emacs vs vi | 11:31 |
| fijal | right | 11:31 |
| fijal | I think pypy and cpython occupy niches different enough | 11:32 |
| lac (~quassel@c-c4c4e055.1321-1-64736c11.cust.bredbandsbolaget.se) joined #pypy. | 11:32 | |
| derdon (~derdon@pD9E1CEBC.dip.t-dialin.net) joined #pypy. | 11:32 | |
| Action: mikefc can't imagine trying to code python in C | 11:33 | |
| lucian | Dulak: also, the emacs/vi holy war is pretty much over :) | 11:34 |
| CIA-13 | 03mikefc 07roundup * 10#953/defaultdict behaviour inconsistent with cpython: submitted a request for a testcase to cpython http://bugs.python.org/issue13531 * 14https://bugs.pypy.org/issue953 | 11:34 |
| fijal | mikefc: thanks! | 11:35 |
| Taggnostr | mikefc, if you want to work on a patch for cpython and need help you can ping me | 11:36 |
| nedbat (~nedbat@python/psf/nedbat) joined #pypy. | 11:36 | |
| fijal | nedbat: hi, again, thanks for your work on sandbox! | 11:37 |
| nedbat | fijal: sure, no problem. | 11:37 |
| mikefc | Taggnostr: thanks. the patch would be tiny. i included a 3 line example in the bug submission. I just don't know the correct 'assertRaises' syntax. looking it up now... | 11:38 |
| Taggnostr | mikefc, you can also use it as a context manager | 11:38 |
| fijal | nedbat: I just don't want to make you feel we don't appreciate it, we totally do | 11:38 |
| nedbat | fijal: thanks | 11:39 |
| mikefc | Taggnostr: pathces have to be done through "hg qdiff"? | 11:41 |
| fijal (~fijal@197.168.178.132) left irc: Read error: Connection reset by peer | 11:42 | |
| Taggnostr | mikefc, hg diff is fine | 11:42 |
| Taggnostr | mikefc, so you just need to get a clone of cpython, add the test(s), check that the test suite passes and use hg diff > patch.diff to make the patch | 11:42 |
| Taggnostr | mikefc: http://docs.python.org/devguide/#quick-start | 11:43 |
| mikefc | Taggnostr: thanks. reading "lifecycle of a patch" | 11:43 |
| mikefc | (while doing a hg clone) | 11:43 |
| Taos (jakob@host-92-20-152-142.as13285.net) left #pypy. | 11:47 | |
| Action: mikefc goes down the rabbithole. | 11:57 | |
| amaury_ (~amaury_@74.125.57.34) joined #pypy. | 11:57 | |
| derdon (~derdon@pD9E1CEBC.dip.t-dialin.net) left irc: Remote host closed the connection | 11:58 | |
| mikefc | can't compile python on this piece of shit. osx snow leopard. | 12:04 |
| JaRoel|4d (~jaroel|4d@office.fourdigits.nl) left irc: Remote host closed the connection | 12:04 | |
| mikefc | anyone know the configure magic? | 12:04 |
| JaRoel|4d (~jaroel|4d@office.fourdigits.nl) joined #pypy. | 12:05 | |
| chronitis (~chronitis@88-104-122-204.dynamic.dsl.as9105.com) joined #pypy. | 12:17 | |
| stakkars_ (~tismer@i59F7379D.versanet.de) joined #pypy. | 12:19 | |
| lucian | mikefc: try homebrew | 12:34 |
| mikefc | compiling from source, so I can run in debug mode, so I submit a patch :) | 12:34 |
| lucian | homebrew likely has a recipe | 12:35 |
| lucian | mikefc: https://github.com/mxcl/homebrew/blob/master/Library/Formula/python.rb | 12:36 |
| Taggnostr | mikefc, there's an issue with osx | 12:36 |
| mikefc | possibly. i'm down the rabbithole. from a bug to pypy, to a bug to python. to figuring out how to write python unittests. | 12:36 |
| Taggnostr | mikefc: http://bugs.python.org/issue13241 | 12:37 |
| Taggnostr | there should be instructions there | 12:37 |
| mikefc | i have something compiled. enough to work with for now. | 12:41 |
| Taggnostr | you don't probably need the debug build for this | 12:41 |
| Squarepy (~Squarepy@unaffiliated/squarepy) left irc: Ping timeout: 268 seconds | 12:42 | |
| mikefc | yeah. i'm figuring stuff out slowly. i don't even need a debug build. | 12:42 |
| mikefc | i don't even need a build | 12:42 |
| mikefc | i can probably just use the python i have to run the tests they have. | 12:42 |
| mikefc | it's late :( | 12:43 |
| mikefc | ahhh. the indentation of my code is wrong. damn space/tabs | 12:43 |
| amaury_ (~amaury_@74.125.57.34) left irc: Ping timeout: 244 seconds | 12:44 | |
| Squarepy (~Squarepy@wcw-staff-214-150.wireless.uva.nl) joined #pypy. | 12:44 | |
| witulski (~stupsi@WLan195.urz.uni-duesseldorf.de) joined #pypy. | 12:47 | |
| witulski (stupsi@WLan195.urz.uni-duesseldorf.de) left #pypy. | 12:50 | |
| lac_ (~quassel@c-c4c4e055.1321-1-64736c11.cust.bredbandsbolaget.se) joined #pypy. | 12:51 | |
| lac (~quassel@c-c4c4e055.1321-1-64736c11.cust.bredbandsbolaget.se) left irc: Ping timeout: 252 seconds | 12:52 | |
| mikefc | first python patch submitted! | 12:54 |
| Taggnostr | mikefc, you could have used assertRaises | 12:59 |
| Taggnostr | when you use it as a context manager you can then introspect the error too | 12:59 |
| mikefc | i could have. but i did what was easiest for my brain :) | 13:00 |
| Taggnostr | (even though error messages are considered implementation details, so it's usually not necessary to test them) | 13:00 |
| mikefc | i thought so. it did seem a bit specific to test the message. | 13:00 |
| Squarepy (~Squarepy@wcw-staff-214-150.wireless.uva.nl) left irc: Ping timeout: 252 seconds | 13:00 | |
| gutworth (~benjamin@dhcp-128-36-84-91.central.yale.edu) joined #pypy. | 13:01 | |
| JaRoel (~jaroel|4d@office.fourdigits.nl) joined #pypy. | 13:02 | |
| mikefc | Taggnostr: what would the assertRaises look like? assertRaises(TypeError, defaultdict, {}) | 13:03 |
| mikefc | just the one line? | 13:03 |
| exarkun | http://docs.python.org/library/unittest.html?highlight=assertraises#unittest.TestCase.assertRaises | 13:03 |
| Taggnostr | or with self.assertRaises(TypeError): defaultdict({}) | 13:04 |
| mikefc | well, that's much simpler than my shit. | 13:04 |
| Action: mikefc works on pathc2 | 13:04 | |
| mikefc | patch2 | 13:04 |
| JaRoel|4d (~jaroel|4d@office.fourdigits.nl) left irc: Ping timeout: 245 seconds | 13:05 | |
| mikefc | Taggnostr: thanks for the assist. | 13:08 |
| Taggnostr | thanks for making cpython and pypy better | 13:08 |
| kkris (~kris@80-123-46-92.adsl.highway.telekom.at) left irc: Quit: Leaving. | 13:12 | |
| _aleksi (~aleksi@85.235.191.82) left irc: Remote host closed the connection | 13:19 | |
| Squarepy (~Squarepy@wcw-staff-214-150.wireless.uva.nl) joined #pypy. | 13:20 | |
| Squarepy (~Squarepy@wcw-staff-214-150.wireless.uva.nl) left irc: Changing host | 13:20 | |
| Squarepy (~Squarepy@unaffiliated/squarepy) joined #pypy. | 13:20 | |
| aleksi (~aleksi@85.235.191.82) joined #pypy. | 13:24 | |
| aleksi (~aleksi@85.235.191.82) left irc: Remote host closed the connection | 13:24 | |
| aleksi (~aleksi@85.235.191.82) joined #pypy. | 13:25 | |
| kenaan | 12ned nedbat-sandbox 114fe097ff1a9b 15/pypy/translator/sandbox/: Remove some pypy dependencies from sandlib. | 13:27 |
| Action: arigato is not finding the (few) things that he needs in pyglet | 13:28 | |
| ronny | arigato: what do you need? | 13:29 |
| arigato | (1) creating images from RGB data with a colorkey for transparency | 13:30 |
| arigato | (2) writing and reading from the off-screen buffer | 13:30 |
| arigato | (3) not crashing when doing partially-off-screen accesses, but I can manage that myself | 13:30 |
| ronny | hmm | 13:31 |
| ronny | generating lists of global functions and running them is faster than direcly running a tree visitor n times (pyevolve gentic programming evluation on pypy) | 13:31 |
| ronny | arigato: i think modern gtk has what you want, but probbly in a bit unpleasant api | 13:32 |
| nedbat | does anyone else get a fail response from bitbucket when pushing? "remote: Timeout, server bitbucket.org not responding. abort: unexpected response: empty string" ? | 13:33 |
| arigato | I don't want gtk, I want something that is really really portable | 13:33 |
| arigato | bub-n-bros has already a gtk version | 13:33 |
| jnoller (~jnoller@vpn.nasuni.com) joined #pypy. | 13:33 | |
| stakkars_ | hi | 13:34 |
| arigato | hi | 13:34 |
| stakkars_ | I was in a meeting with Diez Roggisch, who wants to learn PyPy. | 13:34 |
| stakkars_ | hisproject will be embedding PyPy. aehm&. | 13:35 |
| lucian | arigato: i'm sure you can do it with pyglet's GL binding | 13:35 |
| lucian | or another one | 13:35 |
| aboudreault (~alanb@osgeo/member/aboudreault) joined #pypy. | 13:35 | |
| stakkars_ | arigato: going to write a blog entry, now. (fixing stuff is next on ly list). | 13:36 |
| jell | arigato: fltk is something more portable | 13:36 |
| jell | arigato: also uip http://www.tecgraf.puc-rio.br/iup/en/toolkits.html | 13:37 |
| stakkars_ | do we have a strategy, like putting a draft somewhere into extradoc, or do we write directly? | 13:37 |
| antocuni (~antocuni@host129-122-dynamic.16-79-r.retail.telecomitalia.it) left irc: Ping timeout: 252 seconds | 13:37 | |
| arigato | lucian: yes, I'm sure too, but how exactly, that would require deep digging | 13:39 |
| lucian | well, i could do those with regular gl, but i'm not sure how it'd integrate with pyglet's 2d stuff | 13:40 |
| arigato | regular gl looks good enough | 13:40 |
| lucian | arigato: cairo might be a reasonable option too | 13:40 |
| lucian | and in fact, pyglet's 2d stuff is built on GL afaik | 13:41 |
| lucian | so it might not be that hard to figure out where it fucks up | 13:41 |
| lucian | i think pyglet has no tests, or something | 13:41 |
| arigato | what I like about pyglet is that it doesn't need me to painfully compile 7 versions for various OSes, but I could potentially point people to install more stuff | 13:41 |
| lucian | that is nice, yes | 13:42 |
| arigato | but pygame is out for now, because OS/X support is bad | 13:42 |
| lucian | meh, it's been worse | 13:42 |
| lucian | does it work on PyPy? | 13:43 |
| lucian | btw, when I was on OS X I couldn't get pyglet working on pypy | 13:43 |
| witulski (~stupsi@WLan195.urz.uni-duesseldorf.de) joined #pypy. | 13:44 | |
| witulski (stupsi@WLan195.urz.uni-duesseldorf.de) left #pypy. | 13:44 | |
| arigato | the pygtk web site offers no OS/X download, so that's out too | 13:44 |
| arigato | pyglet may work on CPython on OS/X, but you are right, I should just check | 13:45 |
| stakkars_ | what, pygame is out? | 13:45 |
| lucian | gtk(3) is only realistically usable on ubuntu | 13:45 |
| lucian | Qt works great everywhere, but it's big and i don't think it works on pypy either | 13:45 |
| stakkars_ | but it is _in_ python-2.7 on Mac Os X | 13:45 |
| lucian | for just some graphs and stuff, pyglet/gl sounds best | 13:45 |
| arigato | stakkars_: yes, mac ports, if you want to install that, but it doesn't work with the system python | 13:46 |
| arigato | as far as I know | 13:46 |
| exarkun | gtk(3) isn't even realistically usable on ubuntu | 13:46 |
| lucian | arigato: no, there's a "native" pygame package | 13:47 |
| arigato | where? | 13:47 |
| lucian | arigato: http://pygame.org/download.shtml | 13:47 |
| Fluxid | does pypy like timeit? i mean, does jit work properly when benchmarking with timeit? | 13:47 |
| lucian | the first osx dmg | 13:47 |
| lucian | it's 32bit, though ( i think) | 13:47 |
| lucian | although so is pyglet | 13:47 |
| stakkars_ | aha. Well, Diez had it, I will ask him when he comes in half an hour again | 13:47 |
| lucian | exarkun: how so? | 13:47 |
| exarkun | lucian: because of the 3 part | 13:48 |
| lucian | exarkun: really? it seems to work | 13:48 |
| arigato | ok, thanks | 13:48 |
| exarkun | lucian: yea, seems to, until it starts segfaulting. or until you notice you're using all deprecated apis that are only packaged by accident. | 13:48 |
| exarkun | sort of ot though, sorry | 13:48 |
| aboudreault (~alanb@osgeo/member/aboudreault) left irc: Quit: Leaving | 13:49 | |
| lucian | exarkun: that sounds silly. is gobject-introspection that bad? | 13:49 |
| exarkun | lucian: yes | 13:49 |
| Action: lucian is sad; i was going to write an IRC client with gtk3 and twisted | 13:49 | |
| jell | Fluxid: timeit is bad also on cpython | 13:50 |
| stakkars_ | arigato: no macports. http://pygame.org/wiki/MacCompile | 13:50 |
| stakkars_ | as it looks, it should work with the standard python. | 13:51 |
| stakkars_ | but well, no simple package installer. | 13:52 |
| arigato | stakkars_: no, you're right, there is a Lion package installer now | 13:52 |
| arigato | bivab says that there wasn't, one month ago | 13:53 |
| logan` (~loganchie@canlover.m6.ntu.edu.tw) joined #pypy. | 13:54 | |
| stakkars_ | ah. I didn't follow: are you looking for a new pygamish thing for PyPy, or deciding what to support on top of PyPy? | 13:54 |
| jell | lucian: do you have it in github? | 13:54 |
| arigato | stakkars_: that's not immediately related to pypy at all. | 13:54 |
| jell | lucian: or are you starting it? | 13:54 |
| Nick change: Gulaway -> Gulopine | 13:54 | |
| arigato | stakkars_: it's related insofar as dotviewer goes | 13:54 |
| lucian | jell: yeah, just starting. empty repo mostly | 13:55 |
| arigato | but it seems that I have nothing to do, just keep it in pygame | 13:55 |
| jell | lucian: have you ported gi to pypy? ;) | 13:55 |
| lucian | jell: no, and i won't bother either | 13:55 |
| stakkars_ | yeah. Btw, I tried the jitviewer today. A bit weird, yet. | 13:55 |
| jell | without gi - how you want to write something under gtk3? | 13:56 |
| aboudreault (~alanb@osgeo/member/aboudreault) joined #pypy. | 13:56 | |
| stakkars_ | but it worked. Do you use it, or how do you look at jit traces? | 13:56 |
| lucian | jell: it's gonna run on CPython, most likely | 13:57 |
| jell | argh, ok | 13:57 |
| mikefc | one more bug case for the road... | 13:59 |
| Squarepy_ (~Squarepy@wcw-staff-214-150.wireless.uva.nl) joined #pypy. | 13:59 | |
| Squarepy_ (~Squarepy@wcw-staff-214-150.wireless.uva.nl) left irc: Changing host | 14:00 | |
| Squarepy_ (~Squarepy@unaffiliated/squarepy) joined #pypy. | 14:00 | |
| lucian | is there any (good) UI toolkit that would work on pypy? | 14:00 |
| Squarepy (~Squarepy@unaffiliated/squarepy) left irc: Ping timeout: 252 seconds | 14:01 | |
| Nick change: Squarepy_ -> Squarepy | 14:04 | |
| Nick change: ousado_ -> ousado | 14:07 | |
| Rhy0lite (dje@nat/ibm/x-mkylutffxovkxuzo) joined #pypy. | 14:13 | |
| Nick change: Gulopine -> Gulaway | 14:14 | |
| fijal (~fijal@41.177.93.12) joined #pypy. | 14:15 | |
| fijal | Alex_Gaynor: hey | 14:22 |
| Nick change: hazmat` -> hazmat | 14:24 | |
| whitelynx|work (~whitelynx@63.241.75.144) joined #pypy. | 14:27 | |
| mattip (~chatzilla@bzq-79-177-116-180.red.bezeqint.net) joined #pypy. | 14:32 | |
| mattip | hi | 14:33 |
| fijal | hi | 14:33 |
| fijal | mattip: I'll have a look at reshape now now | 14:33 |
| mattip | OK, np. I actually wanted to talk about something else. | 14:33 |
| mattip | It goes like this: | 14:33 |
| setmeaway (~setmeaway@119.201.52.190) left irc: Quit: Leaving | 14:34 | |
| mattip | find_unaryop_result_dtype in interp_ufuncs is deciding what dtype a result should be. | 14:34 |
| fijal_ (~fijal@41.53.98.38) joined #pypy. | 14:34 | |
| mattip | But when promote_is_largest is True, it is converting an int to a float64 | 14:35 |
| mattip | s/_is_/_to_/ | 14:35 |
| fijal_ | ok | 14:35 |
| mattip | When I do arange(100).sum() I expect to get back an int (at least on numpy it works like that) | 14:36 |
| mattip | but numpypy is promoting it to a float. | 14:36 |
| mattip | End of speech :) | 14:36 |
| fijal_ | hm | 14:37 |
| fijal_ | ok | 14:37 |
| fijal_ | not too good | 14:37 |
| fijal (~fijal@41.177.93.12) left irc: Disconnected by services | 14:37 | |
| mattip | no, now quite finished yet, I didn't ask the question. sorry | 14:37 |
| Nick change: fijal_ -> fijal | 14:37 | |
| fijal | stakkars: hey? | 14:37 |
| mattip | Possible fixes: sum should not promot_to_largest, or promote_to_largest should not convert int to float. | 14:38 |
| mattip | Choose one. | 14:38 |
| mattip | Now I think I'm done... | 14:38 |
| Dulak | Is there a way to suppress this error? UserWarning: readline.read_init_file() not implemented | 14:38 |
| fijal | arigato: personally I would try webgl | 14:39 |
| fijal | you need a (decently new) browser | 14:39 |
| fijal | but it might work on every OS quite reasonably | 14:39 |
| fijal | Dulak: no idea, where does it come from? | 14:40 |
| Dulak | fijal: only under pypy, when I load ipython | 14:40 |
| fijal | ah | 14:40 |
| fijal | well, yes, by implementing readline.read_init_file() :) | 14:40 |
| antocuni (~antocuni@host129-122-dynamic.16-79-r.retail.telecomitalia.it) joined #pypy. | 14:42 | |
| setmeaway (~setmeaway@119.201.52.190) joined #pypy. | 14:42 | |
| Squarepy (~Squarepy@unaffiliated/squarepy) left irc: Ping timeout: 248 seconds | 14:42 | |
| Squarepy (~Squarepy@wcw-staff-214-150.wireless.uva.nl) joined #pypy. | 14:43 | |
| Dulak | ugh I would have to implement a lot more than just that... | 14:43 |
| Squarepy (~Squarepy@wcw-staff-214-150.wireless.uva.nl) left irc: Changing host | 14:44 | |
| Squarepy (~Squarepy@unaffiliated/squarepy) joined #pypy. | 14:44 | |
| fijal | Dulak: why? | 14:44 |
| Dulak | I looks like it uses parse_and_bind | 14:44 |
| Dulak | s/I/It/ | 14:45 |
| mattip | fijal: I am going with changing promote_to_largest. It seems all the reduce functions can overflow happily in numpy. | 14:48 |
| fijal | mattip: ok | 14:49 |
| fijal | mattip: I have no particular preference actually | 14:49 |
| fijal | (as of now at least :) | 14:49 |
| mattip | cool. | 14:50 |
| mattip | hmm. Nope. that breaks mean(). So there will be two classes of reduce functions. All this to add dot. Perhaps after this the next vector of improvement would be to use fpypy. | 14:55 |
| mattip | But then I guess that's a whole new can of worms. | 14:55 |
| arigato | fijal: about both the a[:]=(a+a)*(a+a) and the sys.exc_info() issues: | 14:56 |
| arigato | maybe it would be enough if we had a CALL that is known to not have side effects, | 14:56 |
| arigato | and can thus be killed by the optimizer if the result is not used | 14:57 |
| fijal | with sys.exc_info it's more involved | 14:57 |
| arigato | for sys.exc_info(), this would a CALL that returns the traceback part of the tuple, | 14:57 |
| arigato | and if the CALL cannot be killed, then the loop is aborted | 14:57 |
| fijal | because sys.exc_info() would force frames even above you | 14:58 |
| fijal | so if your frame is not yet being traced, you still abort some JIT above you | 14:58 |
| arigato | that's fine if this only occurs once during tracing | 14:58 |
| arigato | or, let's say, that's already better than now | 14:58 |
| fijal | yes, probably | 14:58 |
| fijal | I was thinking about something obscure | 14:59 |
| fijal | like have a special bytecode for sys.exc_info()[1] | 14:59 |
| fijal | or anything that does not read [2] | 14:59 |
| arigato | yes, but that's too obscure | 14:59 |
| fijal | no, it's actually simpler :) | 14:59 |
| fijal | it's just very unpypyish | 14:59 |
| arigato | it *is* very obscure and unsafe | 14:59 |
| fijal | why? | 14:59 |
| fijal | I mean, of course, yes I agree | 15:00 |
| fijal | I'm just arguing for the sake of argument, sorry | 15:00 |
| arigato | :-) | 15:00 |
| fijal | it's super unpypyish | 15:00 |
| fijal | because you have to check if sys is ok | 15:00 |
| fijal | if exc_info is ok | 15:00 |
| arigato | yes | 15:00 |
| fijal | and you have to have a very obscure hack (albeit a localized one) | 15:00 |
| arigato | yes | 15:00 |
| fijal | I'm fine with saying "this is slow for now" | 15:01 |
| fijal | btw | 15:01 |
| arigato | and we would need to support a few use cases like "sys.exc_info()[:2]" | 15:01 |
| fijal | right | 15:01 |
| fijal | yes, *it is obscure* | 15:01 |
| fijal | I agree :) | 15:01 |
| arigato | and even "(exc, val, _) = sys.exc_info()" would be too hard to properly support | 15:01 |
| fijal | no... | 15:01 |
| fijal | because you can reference _ later | 15:01 |
| fijal | how do you know? | 15:02 |
| arigato | yes, precisely | 15:02 |
| fijal | so anyway | 15:02 |
| fijal | re - numpy | 15:02 |
| arigato | :-) | 15:02 |
| fijal | I think we should think what we want to achieve | 15:02 |
| exarkun | is the reason tracebacks are expensive written down anywhere? | 15:02 |
| fijal | exarkun: I don't think so, although it's easy | 15:02 |
| fijal | exarkun: you just need to make all the frames existy | 15:03 |
| fijal | which are in various places on stack etc. | 15:03 |
| exarkun | all at once? you can't be lazy and only fetch data when it's used? | 15:03 |
| fijal | no | 15:04 |
| fijal | because stuff from C stack might disappear later | 15:04 |
| fijal | we were thinking about elaborate hacks like copying this stuff somewhere in the heap | 15:04 |
| fijal | so you can track that later | 15:04 |
| fijal | it's kinda hard though | 15:04 |
| fijal | [doable though] | 15:04 |
| fijal | it also would "leak" memory | 15:05 |
| fijal | as you know already | 15:05 |
| fijal | that's not different than cpython | 15:06 |
| fijal | or at least not too different | 15:06 |
| exarkun | yes, the semantics of traceback objects is to leak :) | 15:06 |
| fijal | right | 15:06 |
| exarkun | anyway, the C stack part basically explains why it's hard to be lazy, thanks | 15:07 |
| fijal | exarkun: I can add that to the FAQ if you want | 15:07 |
| fijal | I think it's written *somewhere* | 15:07 |
| fijal | https://bitbucket.org/pypy/pypy/wiki/JitFriendliness | 15:08 |
| fijal | here we go | 15:08 |
| fijal | a very brief report | 15:08 |
| fijal | but well | 15:08 |
| Fluxid | fijal: are "Generators" and "String joining" parts still actual? | 15:11 |
| fijal | it's better than it was in Feb | 15:12 |
| fijal | but still might be true for some cases | 15:12 |
| fijal | especially string join | 15:12 |
| witulski (~stupsi@WLan170.urz.uni-duesseldorf.de) joined #pypy. | 15:13 | |
| witulski (stupsi@WLan170.urz.uni-duesseldorf.de) left #pypy. | 15:13 | |
| Action: fijal surfing | 15:14 | |
| fijal (~fijal@41.53.98.38) left irc: Read error: Connection reset by peer | 15:15 | |
| bivab (~david@fwstups.cs.uni-duesseldorf.de) left irc: Read error: Connection reset by peer | 15:19 | |
| arigato | ah, bah | 15:22 |
| Action: arigato found a real-world example where recursion leads to nonsensical over-compilation from the JIT | 15:24 | |
| chronitis | I was wondering for the py3k port whether it is planned to have distinct everything (ie, including rlib) or just the interpreter/modules/objspace? | 15:24 |
| arigato | (actually it traces over and over again the same bridge, aborting it all the time with "Bridge out of guard 3364 was already compiled!") | 15:24 |
| arigato | chronitis: right now it's done as a mercurial branch | 15:26 |
| arigato | which, for now, solves the problem nicely | 15:26 |
| chronitis | I know - I meant whether there was a view on the scope of changes | 15:26 |
| arigato | no | 15:27 |
| brutal_chaos | How much ram is needed for pypy 1.7 to build now (64bit)? | 15:27 |
| arigato | around 4GB | 15:27 |
| brutal_chaos | ugh. I have (htop reports) 3699MB, enough? | 15:28 |
| chronitis | (eg, rlib oct() returns python2 style 0123 instead of 0o123, hence whether commonality or performance is more useful) | 15:28 |
| brutal_chaos | arigato: ugh. I have (htop reports) 3699MB, enough? | 15:29 |
| stakkars_ | arigato: is test_dlltool supposed to work, already? | 15:29 |
| cheyuni (~user@125.129.20.246) joined #pypy. | 15:31 | |
| chronitis | I suspect that is enough RAM, at worst case a small amount of swapping | 15:31 |
| arigato | brutal_chaos: that's probably enough with CPython, yes | 15:32 |
| deets (~deets@i59F7379D.versanet.de) joined #pypy. | 15:33 | |
| arigato | with a default PyPy I would think it's really <4.small number> GB that it needs | 15:33 |
| arigato | but you need to make sure that e.g. you don't have a OS that gobbles up one GB for itself | 15:33 |
| asmeurer__ (~asmeurer@dhcp-baca-230.resnet.nmt.edu) joined #pypy. | 15:34 | |
| bivab (~david@fwstups.cs.uni-duesseldorf.de) joined #pypy. | 15:35 | |
| stakkars_ (~tismer@i59F7379D.versanet.de) left irc: Quit: schnarch | 15:35 | |
| cheyuni (~user@125.129.20.246) left irc: Ping timeout: 245 seconds | 15:35 | |
| stakkars_ (~tismer@i59F7379D.versanet.de) joined #pypy. | 15:39 | |
| brutal_chaos | arigato: lol, running Gentoo. It was idling around 140MB ram usage. | 15:47 |
| brutal_chaos | I should at some point figure on getting that number down, but it's not too bad now. | 15:48 |
| stakkars_ | hi all, | 15:50 |
| stakkars_ | are we still building dlls? | 15:50 |
| stakkars_ | ping | 16:00 |
| kenaan | 12hager ppc-jit-backend 1122a892ccc25e 15/pypy/jit/backend/ppc/ppcgen/codebuilder.py: Add cashe flushing | 16:02 |
| aleksi (~aleksi@85.235.191.82) left irc: Remote host closed the connection | 16:03 | |
| mvt (~mvantelli@87.213.45.85) left irc: Quit: This computer has gone to sleep | 16:03 | |
| brutal_chaos | arigato: the requirement of possible swapspace explains why my build failed now. :( I just created a swap file of 4GB, so the build should pass now. | 16:04 |
| brutal_chaos | possible swapspace being used* | 16:05 |
| kushal (~kdas@114.143.162.64) joined #pypy. | 16:05 | |
| kushal (~kdas@114.143.162.64) left irc: Changing host | 16:05 | |
| kushal (~kdas@fedora/kushal) joined #pypy. | 16:05 | |
| stakkars_ (~tismer@i59F7379D.versanet.de) left irc: Quit: schnarch | 16:11 | |
| asmeurer__ (~asmeurer@dhcp-baca-230.resnet.nmt.edu) left irc: Quit: asmeurer__ | 16:12 | |
| kenaan | 12hakanardo jit-targets 11d638586fc5b1 15/pypy/jit/backend/: hg merge default | 16:14 |
| kenaan | 12hakanardo jit-targets 119f840860f2cb 15/pypy/jit/metainterp/optimizeopt/: dont crash if two equal virtuals become not equal | 16:14 |
| bbot2 | Started: 15http://buildbot.pypy.org/builders/pypy-c-jit-linux-x86-64/builds/614 [12hakanardo, jit-targets] | 16:15 |
| mat^2 (~mathias@212.130.113.35) joined #pypy. | 16:25 | |
| stakkars_ (~tismer@i59F7379D.versanet.de) joined #pypy. | 16:26 | |
| stakkars_ | arigato: do you have a minute? | 16:26 |
| Guest42179 (~raydeo@50.19.209.239) left irc: Quit: Coyote finally caught me | 16:27 | |
| raydeo (~raydeo@merickel.org) joined #pypy. | 16:27 | |
| mattip (chatzilla@bzq-79-177-116-180.red.bezeqint.net) left #pypy. | 16:29 | |
| cpets (c358b60a@gateway/web/freenode/ip.195.88.182.10) left irc: Ping timeout: 265 seconds | 16:39 | |
| arigato | stakkars_: yes? | 16:40 |
| stakkars_ | arigato: there was the abiliuty to build dlls, in the past. | 16:41 |
| arigato | stakkars_: there still is: if you are compiling with the cpyext module, you get a dll | 16:42 |
| stakkars_ | how much does this still exist? Is somebody working on embedding? | 16:42 |
| arigato | no, that's just for cpyext | 16:42 |
| arigato | nobody is working on embedding so far | 16:42 |
| stakkars_ | ah, right. ok | 16:42 |
| arigato | although a simple minimal embedding solution is probably easy (like with one function in the API, something like PyRun_Simple()) | 16:43 |
| stakkars_ | the cpyext generates a dll which gets connected to pypy? | 16:43 |
| stakkars_ | and pypy is a .exe? | 16:43 |
| arigato | the cpyext puts the whole pypy in a dll | 16:43 |
| arigato | like CPython | 16:43 |
| arigato | the .exe is a dummy stub | 16:43 |
| stakkars_ | ahhh. and why? | 16:43 |
| arigato | because it's simpler, I suppose | 16:44 |
| EnCuKou (~encukou@193.167.65.94) joined #pypy. | 16:44 | |
| arigato | no need to connect the .exe and the .dll, if everything is in the .dll | 16:44 |
| stakkars_ | I thought it builds extension modules. | 16:44 |
| stakkars_ | and those are then dlls, and pypy as well? | 16:45 |
| arigato | yes, like with CPython | 16:45 |
| stakkars_ | so it is still a big exe, which is the dll | 16:45 |
| arigato | unsure what you mean, but CPython is the same | 16:46 |
| stakkars_ | ok, so one huge pypy dll, and certain extensions as dll. | 16:46 |
| apanda (~apanda@c-24-7-101-62.hsd1.ca.comcast.net) left irc: Remote host closed the connection | 16:46 | |
| arigato | for embedding, not much is missing | 16:46 |
| arigato | mostly a way to load the pypy dll without starting to interpret a program | 16:47 |
| apanda (~apanda@c-24-7-101-62.hsd1.ca.comcast.net) joined #pypy. | 16:47 | |
| stakkars_ | ok, I understand. I looked everywhere but avoided cpyext, for certain reasons ;-) | 16:47 |
| stakkars_ | last question: | 16:48 |
| stakkars_ | test_dlltool was not working when we tried. | 16:49 |
| stakkars_ | is that supposed to work, or broken? | 16:49 |
| arigato | which directory? | 16:49 |
| stakkars_ | pypy/translator/c/test | 16:49 |
| arigato | I don't know about it | 16:50 |
| deets | it was commented out | 16:50 |
| deets | and when trying to run it, complained about annotator.annotated[block] being a bool, not a graph | 16:50 |
| stakkars_ | passing over to deets. May I introduce you? Diez Roggisch | 16:51 |
| apanda (~apanda@c-24-7-101-62.hsd1.ca.comcast.net) left irc: Ping timeout: 240 seconds | 16:51 | |
| deets | ah, sorry, I should have said who I am and that I worked with Christian on this the afternoon | 16:51 |
| G2P (~G2P@fw-asn1.ornis.com) left irc: Quit: Leaving. | 16:52 | |
| stakkars_ | arigato: does cpyext already support calling back? | 16:53 |
| EnCuKou (~encukou@193.167.65.94) left irc: Remote host closed the connection | 16:53 | |
| arigato | yes | 16:55 |
| arigato | deets: hi | 16:55 |
| stakkars_ | well, then embedded pypy is really in reach. I thought we were lightyears away from it | 16:56 |
| stakkars_ | or miles ;-) | 16:56 |
| deets | sounds great | 16:57 |
| arigato | :-) | 16:57 |
| arigato | you would get more precise information from amaury | 16:57 |
| arigato | but I really think it's not very far away | 16:57 |
| arigato | (of course with the performance hit of cpyext) | 16:57 |
| deets | is there any test / code that shows how to inject a call into the build DLL for example the PyRun_Simple()-like thing? | 16:58 |
| stakkars_ | the performance hit is refcounting etc., right | 16:58 |
| bivab (~david@fwstups.cs.uni-duesseldorf.de) left irc: Read error: Connection reset by peer | 16:58 | |
| ronny | how hard is making a performant c api? | 16:58 |
| exarkun | A new one, incompatible with CPython's? | 16:59 |
| ronny | yes | 16:59 |
| stakkars_ | deets: well, let's contact amaury, then. He is here, often. | 16:59 |
| arigato | deets: what already works is using it as extension modules do, which may include the extension module containing C code that calls back | 17:01 |
| arigato | e.g. with PyRun_Simple(), if that's implemented so far | 17:01 |
| arigato | (maybe it's not, but that's an easy fix) | 17:01 |
| arigato | or with PyObject_CallObject() or anything | 17:01 |
| reuben_ (Seta00@seta00.com) left #pypy ("Textual IRC Client: http://www.textualapp.com/"). | 17:02 | |
| deets | ok, I'll try & prod around | 17:02 |
| arigato | what I don't know is how the pypy dll will behave if its main entry point is not called | 17:02 |
| arigato | it will probably not have initialized its internal state | 17:02 |
| arigato | e.g. the GC | 17:02 |
| deets | ah, ok. | 17:03 |
| fzzzy (~donovan@76-198-130-19.lightspeed.mtvwca.sbcglobal.net) joined #pypy. | 17:03 | |
| stakkars_ | well, you mean an initialization that is done once, but then other entry points can work? | 17:05 |
| arigato | yes | 17:06 |
| exarkun | arigato: CPython has Py_Initialize and doesn't work until you call that. So PyPy's not really much different? | 17:06 |
| arigato | the problem is that the initialization is in the same function that parses the command-line and runs the script provided and eventually runs the finalization code | 17:06 |
| stakkars_ | to force a single init call is not a problem, IMHO. | 17:06 |
| arigato | so yes, it's a matter of splitting it up carefully | 17:07 |
| Action: pedronis expects this to require some cleanup of the .h files which is probably a good thing | 17:07 | |
| arigato | exarkun: yes, like this | 17:07 |
| exarkun | Oh, except CPython has fun exceptions that don't require initialization: Py_SetProgramName(), PyEval_InitThreads(), PyEval_ReleaseLock(), and PyEval_AcquireLock() | 17:07 |
| exarkun | (according to <http://docs.python.org/c-api/init.html>) | 17:07 |
| stakkars_ | ok, but nothing that looks like a huge problem | 17:08 |
| stakkars_ | and that answer was very different say, half a year ago? | 17:08 |
| stakkars_ | I would now tell CCP for instance: it is pretty doable. Months, not many. Right? | 17:09 |
| stakkars_ | ok, some refactoring what goes where when, I see | 17:10 |
| arigato | but it depends what the goal is | 17:10 |
| Action: stakkars_ tends home | 17:10 | |
| stakkars_ | an embedded application | 17:10 |
| arigato | if the goal is to embed it "a la CCP games", then no, it's going to be slooooow | 17:10 |
| JaRoel|4d (~jaroel|4d@office.fourdigits.nl) joined #pypy. | 17:11 | |
| stakkars_ | aha? because they call it much too often and with tasklets, I guess | 17:11 |
| roelbruggink (~jaroel|4d@office.fourdigits.nl) joined #pypy. | 17:12 | |
| arigato | yes, they have too many transfers from C to Python to C to Python etc. | 17:12 |
| arigato | as far as I understood | 17:12 |
| stakkars_ | ok, so are moving between the worlds all the time, which is bad. | 17:12 |
| stakkars_ | but if not much slower than they have now, it is a starting point, | 17:13 |
| JaRoel (~jaroel|4d@office.fourdigits.nl) left irc: Ping timeout: 252 seconds | 17:13 | |
| deets | and I presume due to the dynamic nature of JITting, there is no stable way of exposing the objects in their internal representation? | 17:13 |
| arigato | cpyext is opaque to the JIT | 17:14 |
| stakkars_ | and they can then turn C/C++ modules into PyPy, one after the other | 17:14 |
| arigato | stakkars_: that's a better idea :-) | 17:14 |
| arigato | but I fear a bit that it will start being 5x slower | 17:14 |
| stakkars_ | huiiii :-) | 17:14 |
| arigato | e.g. running PIL on pypy works, but is maybe 5x slower | 17:15 |
| arigato | at least in my example (an image viewer) | 17:15 |
| stakkars_ | interesting, cpyext is a complete barrier for the jit. | 17:16 |
| stakkars_ | well, all the conversions etc as well. | 17:16 |
| arigato | well you cannot JIT a loop that would be looping in C, for example | 17:16 |
| JaRoel|4d (~jaroel|4d@office.fourdigits.nl) left irc: Ping timeout: 268 seconds | 17:16 | |
| stakkars_ | yes, sure | 17:16 |
| arigato | so loops written in Python instead of C are actually faster on PyPy | 17:16 |
| arigato | which is of course completely wrong on CPython | 17:16 |
| arigato | so it may take a lot of refactoring for them to arrive at good performance with PyPy | 17:17 |
| CIA-13 | 03agaynor 07roundup * 10#954/looped string formatting of dict entry slow and hogging memory: | 17:18 |
| CIA-13 | [chatting] I'm not able to reproduce this, the falling numbers are with changing the | 17:18 |
| CIA-13 | attached script up to 10k: | 17:18 |
| CIA-13 | alex@alex-gaynor-laptop:~/Dow ... * 14https://bugs.pypy.org/issue954 | 17:18 |
| azanella (~azanella@189.6.80.207) left irc: Read error: Operation timed out | 17:18 | |
| stakkars_ | so my advice is not a solution but a migration. In the end, C looses | 17:18 |
| fzzzy (~donovan@76-198-130-19.lightspeed.mtvwca.sbcglobal.net) left irc: Quit: fzzzy | 17:18 | |
| stakkars_ | ok I must go and write my blog. Simply put it in? | 17:19 |
| deets | would the tying of C/C++-structures into the JIT work on a struct-level? | 17:19 |
| deets | it's just a very rough idea, but like exposing the underlying memory layout and types to PyPy, thus not needing to go the full route of providing PyObjec*-instances | 17:20 |
| JaRoel|4d (~jaroel|4d@office.fourdigits.nl) joined #pypy. | 17:20 | |
| deets | because I fear phasing out >3M lines of code won't sit to well with my colleagues ;) | 17:20 |
| bbot2 | 4Failure: 15http://buildbot.pypy.org/builders/pypy-c-jit-linux-x86-64/builds/614 [12hakanardo, jit-targets] | 17:20 |
| exarkun | Code is a liability | 17:21 |
| exarkun | Getting rid of 3M lines of it sounds like a massive win. | 17:21 |
| stakkars_ | I remember we had that with RPython extensions and EWT applications. | 17:21 |
| Alex_Gaynor | deets: sounds like you're talking about ctypes basically | 17:21 |
| JaRoel (~jaroel|4d@office.fourdigits.nl) joined #pypy. | 17:22 | |
| stakkars_ | It always starts "great", but then people forget that they wanted to call forth and back seldom, and suddenly "your embedding" is no good. | 17:22 |
| deets | Alex_Gaynor: to some extend, yes - if that's properly jittable. If things have to run through FFI I guess it's less sexy... | 17:23 |
| stakkars_ | RPython dies as a solution, because bummers cannot give up their C crap. | 17:23 |
| Alex_Gaynor | deets: ATM ctypes structs aren't JIT'd nicely, but they're very close to being as good at reading fields as a C compiler is. | 17:23 |
| stakkars_ (~tismer@i59F7379D.versanet.de) left irc: Quit: schnarch | 17:23 | |
| JaRoel|4_ (~jaroel|4d@office.fourdigits.nl) joined #pypy. | 17:23 | |
| roelbruggink (~jaroel|4d@office.fourdigits.nl) left irc: Ping timeout: 248 seconds | 17:24 | |
| jterrace (~jterrace@aegis.CS.Princeton.EDU) joined #pypy. | 17:24 | |
| deets | ok, thanks for the infos. gotta go now too. cya | 17:25 |
| deets (deets@i59F7379D.versanet.de) left #pypy. | 17:25 | |
| JaRoel|4d (~jaroel|4d@office.fourdigits.nl) left irc: Ping timeout: 252 seconds | 17:26 | |
| JaRoel (~jaroel|4d@office.fourdigits.nl) left irc: Ping timeout: 252 seconds | 17:27 | |
| Alex_Gaynor | arigato: Do you remember which conference the hints paper carl wrote was for? | 17:29 |
| JaRoel|4d (~jaroel|4d@office.fourdigits.nl) joined #pypy. | 17:29 | |
| arigato | no | 17:29 |
| pedronis | Alex_Gaynor, the last one, ICOOOLPS | 17:30 |
| Alex_Gaynor | pedronis: thanks | 17:30 |
| pedronis | it's a workshop | 17:30 |
| mkerrin (~mkerrin@wg1-nat.fwgal01.deri.ie) left irc: Ping timeout: 240 seconds | 17:31 | |
| arigato | ah yes | 17:31 |
| JaRoel|4_ (~jaroel|4d@office.fourdigits.nl) left irc: Ping timeout: 244 seconds | 17:33 | |
| JaRoel (~jaroel|4d@office.fourdigits.nl) joined #pypy. | 17:33 | |
| roelbruggink (~jaroel|4d@office.fourdigits.nl) joined #pypy. | 17:35 | |
| JaRoel|4d (~jaroel|4d@office.fourdigits.nl) left irc: Ping timeout: 240 seconds | 17:35 | |
| JaRoel (~jaroel|4d@office.fourdigits.nl) left irc: Ping timeout: 240 seconds | 17:37 | |
| apanda (~apanda@dhcp-44-187.EECS.Berkeley.EDU) joined #pypy. | 17:39 | |
| JaRoel|4d (~jaroel|4d@office.fourdigits.nl) joined #pypy. | 17:41 | |
| apanda (~apanda@dhcp-44-187.EECS.Berkeley.EDU) left irc: Read error: Connection reset by peer | 17:42 | |
| aurojit (~apanda@dhcp-44-187.EECS.Berkeley.EDU) joined #pypy. | 17:42 | |
| lucian (~lucian@93-97-174-114.zone5.bethere.co.uk) left irc: Ping timeout: 240 seconds | 17:43 | |
| Action: arigato -> away | 17:44 | |
| roelbruggink (~jaroel|4d@office.fourdigits.nl) left irc: Ping timeout: 268 seconds | 17:44 | |
| JaRoel (~jaroel|4d@office.fourdigits.nl) joined #pypy. | 17:44 | |
| arigato (~arigo@fwstups.cs.uni-duesseldorf.de) left irc: Quit: See you | 17:45 | |
| JaRoel|4d (~jaroel|4d@office.fourdigits.nl) left irc: Ping timeout: 252 seconds | 17:47 | |
| JaRoel|4d (~jaroel|4d@office.fourdigits.nl) joined #pypy. | 17:48 | |
| azanella (~azanella@189.6.80.207) joined #pypy. | 17:48 | |
| stakkars_ (~tismer@f052069108.adsl.alicedsl.de) joined #pypy. | 17:51 | |
| JaRoel (~jaroel|4d@office.fourdigits.nl) left irc: Ping timeout: 245 seconds | 17:52 | |
| JaRoel (~jaroel|4d@office.fourdigits.nl) joined #pypy. | 17:54 | |
| DanielHolth (~dholth@ip98-180-34-112.ga.at.cox.net) joined #pypy. | 17:54 | |
| stakkars_ | Alex_Gaynor: the C types can of course be quite quick. | 17:55 |
| stakkars_ | enforcement of a Python API is probably a large overhead. | 17:56 |
| JaRoel|4d (~jaroel|4d@office.fourdigits.nl) left irc: Ping timeout: 252 seconds | 17:56 | |
| stakkars_ | thinking of PIL, Armin talked about a 5 times speed loss against CPython. | 17:57 |
| JaRoel|4d (~jaroel|4d@office.fourdigits.nl) joined #pypy. | 17:57 | |
| stakkars_ | I guess this is converting to/from refcounted objects all the time. | 17:58 |
| roelbruggink (~jaroel|4d@office.fourdigits.nl) joined #pypy. | 17:59 | |
| stakkars_ | and probably not in single actions, but stuff like a user supplied filter function, which then goes forth and back in every loop cycle? | 17:59 |
| Alex_Gaynor | PIL is slow because it uses the CPython C-API | 18:00 |
| Alex_Gaynor | Interfacing with raw C code can be extremely efficient. | 18:00 |
| JaRoel (~jaroel|4d@office.fourdigits.nl) left irc: Ping timeout: 252 seconds | 18:00 | |
| JaRoel (~jaroel|4d@office.fourdigits.nl) joined #pypy. | 18:00 | |
| stakkars_ | Alex_Gaynor: that sounds promising. | 18:00 |
| stakkars_ | so you mean actually a ctypes like interface | 18:01 |
| Alex_Gaynor | Sure, callls through ctypes are already very fast, and anto has a branch which makes using Structs very fast | 18:02 |
| JaRoel|4d (~jaroel|4d@office.fourdigits.nl) left irc: Ping timeout: 244 seconds | 18:02 | |
| stakkars_ | what about the refcounting? Only a problem for objects to be passed in and out? | 18:02 |
| roelbruggink (~jaroel|4d@office.fourdigits.nl) left irc: Ping timeout: 252 seconds | 18:03 | |
| stakkars_ | or, does cpyext really use refcounting, or is that a fake? | 18:03 |
| dmalcolm (david@nat/redhat/x-fvsigngeczovtbqf) joined #pypy. | 18:03 | |
| kushal (~kdas@fedora/kushal) left irc: Quit: This computer has gone to sleep | 18:04 | |
| roelbruggink (~jaroel|4d@office.fourdigits.nl) joined #pypy. | 18:04 | |
| Alex_Gaynor | CPyext fakes refcounting, if you interface with raw C you don't have these problems | 18:05 |
| JaRoel|4_ (~jaroel|4d@office.fourdigits.nl) joined #pypy. | 18:06 | |
| stakkars_ | well, ould it make sense to use PIL with interfacing directly, bypassing the API? | 18:06 |
| JaRoel (~jaroel|4d@office.fourdigits.nl) left irc: Ping timeout: 248 seconds | 18:06 | |
| JaRoel (~jaroel|4d@office.fourdigits.nl) joined #pypy. | 18:07 | |
| Alex_Gaynor | If someone wants to rewrite PIL using ctypes, yes that would be faster on pypy | 18:07 |
| stakkars_ | probably makes not _soo_ much sense. Better to rewrite PIL in pure Python, right? | 18:07 |
| cpets (4f7588fa@gateway/web/freenode/ip.79.117.136.250) joined #pypy. | 18:07 | |
| Alex_Gaynor | sure, and ctypes to interface with libjpg and friends | 18:08 |
| fijal (~fijal@197.168.0.109) joined #pypy. | 18:08 | |
| stakkars_ | and I doubt it would be very hard. Just much | 18:08 |
| stakkars_ | hmm, sounds like a nice project. | 18:08 |
| etrepum (~bob@75-101-96-144.dsl.static.sonic.net) left irc: Quit: etrepum | 18:09 | |
| stakkars_ | maybe we can talk into Fredrik to "Un-C" his PIL for us ;-) | 18:09 |
| roelbruggink (~jaroel|4d@office.fourdigits.nl) left irc: Ping timeout: 252 seconds | 18:09 | |
| stakkars_ | I think that's a general migration advice, maybe: | 18:10 |
| JaRoel|4_ (~jaroel|4d@office.fourdigits.nl) left irc: Ping timeout: 245 seconds | 18:10 | |
| stakkars_ | use cpyext first for your modules, | 18:10 |
| stakkars_ | then re-port them to Python, step by step. | 18:11 |
| fijal | gttt | 18:11 |
| fijal | grrr | 18:11 |
| fijal | I want to tell armin he's not right :/ | 18:11 |
| Alex_Gaynor | fijal: hey | 18:11 |
| Alex_Gaynor | what's armin not right about? | 18:12 |
| stakkars_ | funny, developers were paid for porting Python to C, nowadays the reverse thing will happen :-) | 18:12 |
| fijal | 4.something gig of ram to build pypy on pypy | 18:12 |
| fijal | and embedding | 18:12 |
| fijal | stakkars_: I did embed pypy some time in the past | 18:12 |
| stakkars_ | fijal: yes? | 18:13 |
| MostAwesomeDude | stakkars_: PIL is several thousand lines of very dense, uncommented C. I already got started on the port, but it'll take a *long* time and everybody already has more important things to do. | 18:13 |
| fijal | stakkars_: into webkit | 18:13 |
| fijal | anyway, I made it work via dlltool, might be broken these days | 18:13 |
| fijal | what are you trying to achieve? | 18:13 |
| stakkars_ | MostAwesomeDude: therefore I meant to ask the Effbot, maybe he even has a Python implementation | 18:14 |
| fijal | stakkars_: what do you want to tell CPP? | 18:15 |
| fijal | it wasn't hard | 18:15 |
| fijal | and I did it within a month | 18:15 |
| stakkars_ | dlltool? Ah, well, we were playing around today, and some things seem to be not working | 18:15 |
| fijal | ended up with a working webkit and python <-> JS interface | 18:15 |
| fijal | might be | 18:15 |
| fijal | deets is not around anymore :( | 18:16 |
| stakkars_ | test_dlltoll maybe it was. I tried to show it to Diez, but had no clue, and we failed | 18:16 |
| mattip (~chatzilla@bzq-79-177-116-180.red.bezeqint.net) joined #pypy. | 18:16 | |
| stakkars_ | yes, he is on his way home. Maybe you reach him later. | 18:16 |
| stakkars_ | It makes pretty much sense to keep him interested. | 18:17 |
| fijal | well, I can make it work | 18:17 |
| fijal | but I have no personal interest in that | 18:17 |
| stakkars_ | just confusing if non-working cole is around. Maybe put in a warning "unsupported attic code" ;-) | 18:18 |
| stakkars_ | s/cole/code/ | 18:19 |
| hpk (~hpk@hq2.merlinux.eu) left irc: Quit: WeeChat 0.3.2 | 18:20 | |
| jnoller (~jnoller@vpn.nasuni.com) left irc: | 18:21 | |
| stakkars_ | MostAwesomeDude: a crazy old idea, which I might consider again, after the JIT is so well: | 18:22 |
| fijal | stakkars_: it's probably mostly fine | 18:23 |
| stakkars_ | can we write a script that uses a primitive mapping to let C code run as Python code? | 18:23 |
| gtaylor (~gtaylor@108-196-160-120.lightspeed.tukrga.sbcglobal.net) joined #pypy. | 18:23 | |
| fijal | stakkars_: but also I have no interest in making it suite someone's else taste | 18:23 |
| antocuni (~antocuni@host129-122-dynamic.16-79-r.retail.telecomitalia.it) left irc: Ping timeout: 252 seconds | 18:23 | |
| fijal | suit | 18:23 |
| fijal | after all, we did quite a lot to interest CPP - we made the JIT, we made stackless | 18:23 |
| fijal | if they're still not believing we're any hope, sorry, but I have no real belief I would get any money out of the deal | 18:24 |
| stakkars_ | ijyou think they still are not interested? | 18:25 |
| stakkars_ | they really should be now. And Win64 is coming -- what else do they want? | 18:25 |
| fijal | I have no idea | 18:26 |
| fijal | but look | 18:26 |
| fijal | at some point they might just use it | 18:26 |
| fijal | and be happy | 18:26 |
| fijal | but I have very little hope it'll translate into some contract for us | 18:26 |
| stakkars_ | that's always a danger | 18:26 |
| fijal | I'm just saying I'm not investing my time in parts of pypy that are of not interest to me | 18:27 |
| fijal | and are actually a maitnenance burden | 18:27 |
| stakkars_ | btw., I did not mean CCP, but Diez's company. | 18:27 |
| fijal | ok | 18:27 |
| fijal | what they're doing? | 18:27 |
| kenaan | 12arigo default 110d1f6b514b53 15/pypy/jit/metainterp/test/test_recursive.py: Add a failing test. | 18:30 |
| DanielHolth | This is not PIL or even Python, but if you just want to wrap a library for doing something with images imlib2 seems easy. http://sheddingbikes.com/posts/1292063326.html | 18:30 |
| kenaan | 12arigo default 115abc8457062e 15/pypy/jit/metainterp/: Fix the test. | 18:30 |
| mattip | There is a market for pypy you may want to consider, going in the embedded direction. | 18:30 |
| arigato (~arigo@82.113.121.170) joined #pypy. | 18:31 | |
| fijal | arigato: hi armin | 18:31 |
| arigato | hi | 18:31 |
| etrepum (~bob@accessnat4.mochimedia.net) joined #pypy. | 18:31 | |
| fijal | arigato: I successfully translate pypy using pypy on 4G of RAM | 18:31 |
| fijal | 3.7 actually | 18:31 |
| fijal | (graphics) | 18:31 |
| brodie (~brodie@sven.sf.io) left irc: Changing host | 18:31 | |
| brodie (~brodie@unaffiliated/brodie) joined #pypy. | 18:31 | |
| arigato | nice | 18:32 |
| fijal | mattip: pypy's embedded story is not that great | 18:32 |
| kushal (~kdas@fedora/kushal) joined #pypy. | 18:32 | |
| mattip | I thnk there is a considerable market for algorithm implementation that starts at a very high level and ends up with assembler | 18:32 |
| fijal | arigato: it's --source and I have to close browsers | 18:32 |
| fijal | but whatever | 18:32 |
| arigato | :-) | 18:32 |
| fijal | anyway, it's possible, although you want to close your browsers | 18:32 |
| mattip | It may not be very open-source, since one of the goals is to hide the source code :), but lots of money is there. | 18:33 |
| aboudreault (~alanb@osgeo/member/aboudreault) left irc: Quit: Leaving | 18:33 | |
| fijal | mattip: I doubt it's sellable | 18:34 |
| fijal | but hey, who am I to judge | 18:34 |
| kushal (~kdas@fedora/kushal) left irc: Client Quit | 18:35 | |
| kenaan | 12alex_gaynor numpy-dtype-refactor-complex 117a810c2bdf0d 15/pypy/module/micronumpy/: added real and imag descriptors to complex objs | 18:35 |
| kenaan | 12alex_gaynor numpy-dtype-refactor-complex 11006352005df1 15/pypy/jit/: merged default | 18:35 |
| kenaan | 12alex_gaynor numpy-dtype-refactor-complex 1122862d325dba 15/pypy/module/micronumpy/: fix, make this numpypy | 18:35 |
| mattip | Well, I know alot of people who make a living translating matlab to c | 18:36 |
| fijal | well, that's great | 18:36 |
| fijal | my plan is to make those people obsolete | 18:36 |
| arigato | bah | 18:36 |
| amaury_ (~amaury_@46-127-23-192.dynamic.hispeed.ch) joined #pypy. | 18:36 | |
| logan` (~loganchie@canlover.m6.ntu.edu.tw) left irc: Remote host closed the connection | 18:36 | |
| fijal | arigato: yes? | 18:37 |
| Action: arigato just spent 10 minutes with "hg up" figuring out how this metainterp/test/test_math.py could possibly fail since forevr | 18:37 | |
| arigato | any idea about why? :-) | 18:37 |
| fijal | no? | 18:37 |
| arigato | there is no such file in hg | 18:38 |
| fijal | bah | 18:38 |
| fijal | of course | 18:38 |
| arigato | it's a local-only file | 18:38 |
| DasIch_ (~dasich@p3E990997.dip.t-dialin.net) joined #pypy. | 18:38 | |
| mattip | along the way, maybe get their employer to pay you. Company sponsorship? | 18:40 |
| arigato | I forgot to check it in, maybe? | 18:40 |
| fijal | mattip: I don't happen to have an employer ;-) | 18:40 |
| DasIch (~dasich@p4FFDE8BF.dip.t-dialin.net) left irc: Ping timeout: 240 seconds | 18:41 | |
| mattip | But you could pitch your idea of "making algorithm implementation easy" to, say, automotive firms, so they would sponsor pypy. | 18:42 |
| DanielHolth | pypy on arduino, brillant | 18:42 |
| DanielHolth | you have to reduce the memory requirements from 6GB down to 4k | 18:42 |
| Action: Alex_Gaynor doesn't get people who hack on hardware | 18:42 | |
| Squarepy | DanielHolth: ask for a fundraiser | 18:43 |
| fijal | mattip: believe me, making anyone big to invest into pypy is not gonna fly | 18:43 |
| DanielHolth | It's refreshing to code with 32 bytes of RAM when you normally have gigabytes. Totally different challenges. | 18:44 |
| Squarepy | DanielHolth: like the old days | 18:44 |
| DanielHolth | although I would spring for 128 or 256 bytes any day | 18:45 |
| kenaan | 12arigo default 11ed550a1d0c11 15/pypy/jit/metainterp/test/test_math.py: Add a test that I wrote long ago, but apparently forgot to check in. | 18:45 |
| CIA-13 | 03afa 07roundup * 10#953/defaultdict behaviour inconsistent with cpython: An easy task for someone who wants to start hacking on pypy! * 14https://bugs.pypy.org/issue953 | 18:46 |
| aboudreault (~alanb@osgeo/member/aboudreault) joined #pypy. | 18:47 | |
| stakkars_ | arigato: fijal : are you going to PyCon US? | 18:58 |
| stakkars_ | funny: a test that always annoys you, and it is a local, non-versioned thing: | 19:00 |
| stakkars_ | testing-virus | 19:00 |
| Action: fijal is going | 19:02 | |
| stakkars_ | fijal: pycon? | 19:05 |
| lmatos | do you guys pay for hosting for the pypy.org box? | 19:05 |
| stakkars_ | I am paying for hosting my server, which runs the win64 buildbot, yes. | 19:06 |
| mattip | Sage seems to have grown a community of investors, octave has a funding page. | 19:08 |
| fijal | lmatos: nope | 19:10 |
| fijal | lmatos: but we would love more boxes :) | 19:10 |
| fijal | or maybe we actually do | 19:10 |
| kenaan | 12arigo default 1192d4f8f7bcfb 15/pypy/module/_collections/: Test and fix. | 19:10 |
| CIA-13 | 03arigo 07roundup * 10#953/defaultdict behaviour inconsistent with cpython: [resolved] Sorry, couldn't resist. 92d4f8f7bcfb * 14https://bugs.pypy.org/issue953 | 19:10 |
| lmatos | fijal: what kind of boxes and what size? www.linode.com, Don't get too greedy with your offer and I'll see what I can do :) | 19:10 |
| fijal | hehe :) | 19:13 |
| fijal | essentially if we can get one powerful box for buildslave it'll be awesome | 19:14 |
| amaury_ | 4GB RAM is a bit short :-) | 19:14 |
| fijal | but yeah, 4G is a bit short | 19:14 |
| fijal | for example our windows and mac boxes are someone's boxes who is very kind | 19:14 |
| arigato | another tannit-sized box, with today's performance instead of two years ago (almost?), would be cool :-) | 19:14 |
| arigato | but indeed, it's not the most pressing need | 19:15 |
| fijal | lmatos: I can tell you details what would be cool if you're interested | 19:15 |
| fijal | arigato: I think another tannit box would be *really* cool | 19:15 |
| amaury_ | arigato: you missed lib-pypy/_collections.py | 19:15 |
| arigato | amaury_: ba | 19:15 |
| arigato | h | 19:15 |
| fijal | lmatos: we have obscure requirements though | 19:16 |
| Action: arigato copy-pastes | 19:16 | |
| amaury_ | I saw it when testing with bin/py.py | 19:16 |
| lmatos | fijal: feel free to pm me. I can ask. A 4GB or larger box may be a stretch, but it never hurts to ask. Do you guys have any analytics data anywhere? If Linode Sponsored it, I'm sure marketing would like to see that. | 19:16 |
| fijal | we have about 10-20k visits to our blog each time we post something interesting | 19:16 |
| fijal | (all of them are geeks) | 19:17 |
| stakkars_ (~tismer@f052069108.adsl.alicedsl.de) left irc: Quit: schnarch | 19:17 | |
| lmatos | haha | 19:17 |
| fijal | not sure if it's good or bad :) | 19:17 |
| arigato | amaury_: :-) | 19:17 |
| kenaan | 12arigo default 110732486f6a76 15/lib_pypy/_collections.py: Forgot this version of the file. Thanks amaury. | 19:18 |
| fijal | lmatos: I can probably just give you access to analytics, there is nothing hidden there | 19:18 |
| ojii (~ojii@40-34.60-188.cust.bluewin.ch) left irc: Remote host closed the connection | 19:22 | |
| hakanardo | hmm, test_ajit seem to be 5 times slower on jit-targets as compared to default... | 19:23 |
| fijal | :/ | 19:23 |
| fijal | hakanardo: you mean pypy py.test test_ajit | 19:24 |
| fijal | or just test_ajit? | 19:24 |
| hakanardo | py.test test_ajit | 19:24 |
| Squarepy (~Squarepy@unaffiliated/squarepy) left irc: Quit: Leaving | 19:24 | |
| fijal | well, that's not the end of the world I suppose | 19:24 |
| fijal | but I wonder why | 19:24 |
| fijal | it might mean tracing is slower as well | 19:25 |
| hakanardo | indeed but it's a bit strange | 19:25 |
| Alex_Gaynor | fijal: well it means untranslated tracing i sslower, who knows what that means for real code | 19:26 |
| hakanardo | do we have something that would scale worse than linearly on the number of resops? | 19:26 |
| fijal | hakanardo: I hope not | 19:26 |
| fijal | if we do we should remove it | 19:26 |
| arigato | hakanardo: bah | 19:30 |
| arigato | it seems that the tests really run slower and slower | 19:31 |
| arigato | I tried to run with the following command line: | 19:31 |
| arigato | py.test test_ajit.py test_ajit.py {repeat 10x} -xv -k test_merge_guard | 19:31 |
| arigato | it takes 7.7 seconds the first time, and down to 13.8 seconds | 19:31 |
| arigato | to run the same tests | 19:31 |
| arigato | that's also the impression we get by running in parallel test_ajit in both the branch and in default | 19:32 |
| arigato | at the beginning it's almost as fast | 19:32 |
| arigato | but it gets slower and slower | 19:32 |
| hakanardo | arigato: test_ajit takes 12 min to run for me the first time I runt it in jit-targets | 19:33 |
| arigato | I'm sure that it took a while to run, say, the last 10 tests | 19:33 |
| JaRoel (~jaroel|4d@office.fourdigits.nl) left irc: Remote host closed the connection | 19:33 | |
| arigato | but if you run them again, only these last 10 tests, then it will be very fast | 19:33 |
| kkris (~kris@93-82-40-130.adsl.highway.telekom.at) joined #pypy. | 19:34 | |
| JaRoel|4d (~jaroel|4d@office.fourdigits.nl) joined #pypy. | 19:35 | |
| chronitis | does the generated bytecode depend on any objspace options? | 19:37 |
| Alex_Gaynor | Yes, there are several which enable/disable custom bytecodes | 19:38 |
| chronitis | (for PEP3147 __pycache__ directories - is it necessary to tag bytecodes with finer granularity than just "pypy-<major,minor>") | 19:38 |
| JaRoel|4d (~jaroel|4d@office.fourdigits.nl) left irc: Ping timeout: 252 seconds | 19:39 | |
| chronitis | or since non-dev users are unlikely to be changing objspace options, is this a minor enough issue to ignore | 19:39 |
| arigato | I think it's minor enough for now | 19:41 |
| chronitis | right | 19:41 |
| amaury_ (~amaury_@46-127-23-192.dynamic.hispeed.ch) left irc: Ping timeout: 252 seconds | 19:42 | |
| lucian (~lucian@cpc1-newc15-2-0-cust84.gate.cable.virginmedia.com) joined #pypy. | 19:42 | |
| amaury_ (~amaury_@74.125.122.49) joined #pypy. | 19:42 | |
| arigato | bah, how do I get py.test to give a useful result with pressing Ctrl-C ? | 19:44 |
| Alex_Gaynor | some obscure combination of flags, I'm sure | 19:45 |
| arigato | didn't find it | 19:45 |
| arigato | ok, for future reference: comment out "except KeyboardInterrupt: raise" in _pytest/runner.py:100 | 19:47 |
| Alex_Gaynor | heh | 19:48 |
| arigato | ah, bah | 19:50 |
| arigato | hakanardo: it's because of gc.collect() | 19:51 |
| arigato | actually calling it explicitly takes more and more forever on CPython when the total memory usage grows | 19:51 |
| arigato | warmstate.py:178 | 19:52 |
| hakanardo | ah! | 19:52 |
| arigato | found by writing test_x.py that does 10 times the test_merge_xxx, then one time | 19:54 |
| arigato | from pypy.translator.goal import unixcheckpoint | 19:54 |
| arigato | unixcheckpoint.restartable_point(auto='run') | 19:54 |
| arigato | then one mor time the test_merge_xxx | 19:54 |
| arigato | running with py.test -s | 19:55 |
| arigato | and commenting out the two lines in _pytest/runner.py:100 as explained above | 19:55 |
| arigato | at the end, press Ctrl-C when there is a noticable pause in the output | 19:55 |
| arigato | (you get as many tries as you want, thanks to unixcheckpoint) | 19:56 |
| arigato | most of the time, the traceback is at rgc.collect() | 19:56 |
| fijal | pffff | 19:56 |
| fijal | why it does eat more and more mem btw? | 19:56 |
| Alex_Gaynor | why do we need gc calls all over the place in teh tests? | 19:56 |
| arigato | this gc.collect() is not in "default", so I'm not sure | 19:57 |
| arigato | I guess it's to test something related to weakrefs | 19:57 |
| arigato | I already had problems that it took a bit forever | 19:57 |
| arigato | I ended up putting the relevant tests in test_memmgr.py by calling a subprocess | 19:58 |
| Alex_Gaynor | it should be an explicit rgc call ni the test then, I think, not in the JIT's code | 19:58 |
| arigato | to get some clean state | 19:58 |
| arigato | Alex_Gaynor: a bit hard, see memmgr.py:81 and the comment | 19:59 |
| Alex_Gaynor | on jit-targets? | 19:59 |
| arigato | not necessarily | 19:59 |
| fijal | Alex_Gaynor: hey, I'll refactor signatures | 20:00 |
| arigato | these are for tests that check that old loops really really go away | 20:00 |
| fijal | and change the way iterators are created | 20:00 |
| Alex_Gaynor | fijal: ok :) | 20:00 |
| fijal | Alex_Gaynor: for what is worth, we can move strides to ConcreteArray | 20:00 |
| fijal | which would be either NDimArray or NDimSlice | 20:00 |
| fijal | and keep everything else leaner | 20:00 |
| Alex_Gaynor | fijal: sounds good | 20:00 |
| fijal | I *still* think we should not go via scalars when doing numeric computations on boxes | 20:00 |
| fijal | but well | 20:01 |
| Alex_Gaynor | fijal: complex stuff is almost in a mergable state | 20:01 |
| fijal | cool | 20:01 |
| fijal | reshape apparently as well | 20:01 |
| Alex_Gaynor | fijal: btw, if you run laplace (the version I have) 2x, the second version we're basically tied with C++ | 20:01 |
| Alex_Gaynor | fijal: I guess we should blog post soon | 20:01 |
| Alex_Gaynor | fijal: is numeric verison faster than numpy yet? | 20:02 |
| etrepum (~bob@accessnat4.mochimedia.net) left irc: Remote host closed the connection | 20:07 | |
| etrepum (~bob@accessnat4.mochimedia.net) joined #pypy. | 20:07 | |
| cpets (4f7588fa@gateway/web/freenode/ip.79.117.136.250) left irc: Ping timeout: 265 seconds | 20:08 | |
| hakanardo | sorry, had to take an urgent call | 20:09 |
| hakanardo | I added a gc.collect() to jit-targets to get the memgr tests to pass | 20:10 |
| mfoord (~anonymous@python/psf/voidspace) joined #pypy. | 20:10 | |
| hakanardo | there is one there that assumes the loops are freed very quickly... | 20:11 |
| fijal | Alex_Gaynor: there are few issues I want to fix | 20:11 |
| fijal | Alex_Gaynor: is it inplace or not inplace? | 20:11 |
| fijal | because on is 2x faster on C++ | 20:12 |
| fijal | or >2x | 20:12 |
| fijal | the non-inplace one | 20:12 |
| Alex_Gaynor | fijal: did we improve there at all after you fixed the registe rallocator? | 20:12 |
| fijal | yes, by 10% | 20:12 |
| Alex_Gaynor | not nearly 2x though | 20:12 |
| fijal | no, not nearly | 20:13 |
| fijal | sharing iterators should help some | 20:13 |
| fijal | and I have some other ideas | 20:13 |
| Alex_Gaynor | sharing iterator is hard :/ | 20:13 |
| fijal | but we should wait quite a bit more with a blog post ok? | 20:13 |
| fijal | no, it's not | 20:13 |
| Alex_Gaynor | is that what you want to refactor signature for? | 20:13 |
| fijal | I've done it, it just does not make optimizer happy | 20:13 |
| fijal | yes | 20:13 |
| Alex_Gaynor | heh, the whole point is to make the optimizer happy | 20:13 |
| Alex_Gaynor | ;) | 20:13 |
| Rhy0lite | maybe you are using the term "register allocator" in a broader way | 20:14 |
| stakkars_ (~stakkars@89.204.153.195) joined #pypy. | 20:15 | |
| Rhy0lite | but it seems like the problem starts earlier with how the IR deals with addresses and temporaries | 20:15 |
| fijal | Rhy0lite: addresses and temporaries are a domain of the backend | 20:15 |
| Alex_Gaynor | Rhy0lite: no, earlier there was a bug where the regalloc would assign registers to variables that were only ever used as a live-variable for a guard. this meant in tight loops you could often have stuff not in registers for stupid reasons. | 20:15 |
| Alex_Gaynor | I was specifically asking fijal how much his fix for that helped. | 20:15 |
| amaury (~amaury_@46-127-23-192.dynamic.hispeed.ch) joined #pypy. | 20:16 | |
| stakkars_ | arigato: According to what you said about the Jit and refcounting: wouldn't that be an approach to re-invent refcounting? | 20:17 |
| JaRoel|4d (~jaroel|4d@2001:980:7133:1:94a1:1366:80f8:ee54) joined #pypy. | 20:17 | |
| Rhy0lite | Alex_Gaynor: I mean things like f53 = getarrayitem_raw(i25, i2, descr=<FloatArrayNoLengthDescr>) | 20:17 |
| stakkars_ | It would nicely get virtualized as in Psyco! | 20:17 |
| Rhy0lite | and later getarrayitem_raw | 20:17 |
| Alex_Gaynor | Rhy0lite: what's the problem with them? | 20:17 |
| Rhy0lite | where the address is i2 + offset | 20:18 |
| Alex_Gaynor | ah, you're saying the issue is we don't fold arithmatic on variables into the addressing in MOV operations? | 20:18 |
| Rhy0lite | yes | 20:18 |
| Alex_Gaynor | that's a seperate issue from the register allocator. | 20:18 |
| Alex_Gaynor | that should be a backend-specific peephole optimization. | 20:18 |
| Alex_Gaynor | (IMO) | 20:19 |
| Rhy0lite | it's expensive to expect the backend to re-discover common addresses | 20:19 |
| Rhy0lite | instead of maintaining them in a common form in the IR | 20:19 |
| Alex_Gaynor | how would you express i2 = i1 + 1; getaarrayitem_raw(some_addr, i2) in the JIT's SSA repr? | 20:20 |
| amaury_ (~amaury_@74.125.122.49) left irc: Ping timeout: 240 seconds | 20:20 | |
| Alex_Gaynor | I need to run to class, I'll be back to discuss in 20 minutes? | 20:20 |
| Rhy0lite | ttyl | 20:20 |
| ojii (~ojii@202-91.78-83.cust.bluewin.ch) joined #pypy. | 20:24 | |
| fijal (~fijal@197.168.0.109) left irc: Ping timeout: 244 seconds | 20:27 | |
| arigato | stakkars_: are you saying that we should use refcounting in pypy, because the jit can optimize it away? that's a lot of work for nothing, because refcounting is not faster anyway | 20:28 |
| amaury__ (~amaury_@46-127-23-192.dynamic.hispeed.ch) joined #pypy. | 20:30 | |
| mfoord (~anonymous@python/psf/voidspace) left irc: Quit: mfoord | 20:30 | |
| amaury (~amaury_@46-127-23-192.dynamic.hispeed.ch) left irc: Ping timeout: 240 seconds | 20:31 | |
| arigato | Rhy0lite, Alex_Gaynor: fwiw, in the op_malloc_gc branch, I lower the level of the "new_*" operations before the backend sees them | 20:31 |
| arigato | the branch is not done, but that seems to make sense | 20:31 |
| arigato | maybe something similar could be done by replacing getarrayitem with explicit asm-level index manipulations | 20:32 |
| arigato | although it would need to be done a bit earlier, to benefit from the optimizer | 20:32 |
| arigato | (it's not obvious, because we'd ideally want to avoid recomputing "p1+offset+4*i2" if it was computed before, but only if there is no intermediate gc call) | 20:34 |
| stakkars_ (~stakkars@89.204.153.195) left irc: Ping timeout: 240 seconds | 20:34 | |
| arigato | Rhy0lite: also, note that on x86, addressing like "p1+offset+4*i2" can be done within any instruction, so it's an unclear win | 20:35 |
| Rhy0lite | arigato: yes, but one needs to maintain p1 and i2 | 20:35 |
| stakkars_ (~stakkars@89.204.153.195) joined #pypy. | 20:35 | |
| arigato | yes, that's the drawback | 20:36 |
| Rhy0lite | it can be a benefit | 20:36 |
| Alex_Gaynor | Right, a MOV is the same number of cycles no matter what addressing you give it right? | 20:36 |
| Rhy0lite | but computing p1+offset+4*i2, saving it to a stack location, then loading it to dereference it | 20:36 |
| arigato | I still think there are easier problems with the way we produce code | 20:36 |
| Alex_Gaynor | the advantage is in the +1/-1 case because you can fold that into the instruction always | 20:36 |
| Rhy0lite | what is less efficient is computing the address | 20:38 |
| Rhy0lite | storing it in a stack location | 20:38 |
| Rhy0lite | referencing the array through the stack location | 20:38 |
| Rhy0lite | and then updating the value in the stack location | 20:38 |
| Alex_Gaynor | that was the register allocator bug, I think | 20:38 |
| Rhy0lite | okay, I haven't seen the latest code from the JIT for Maciej's example | 20:39 |
| Alex_Gaynor | do you have the python code for his example, I have a recent pypy I can test it against | 20:39 |
| lizardo (~lizardo@189.2.128.130) left irc: Quit: Leaving | 20:41 | |
| Rhy0lite | a link is in his twitter posting, but twitter is being a twit | 20:41 |
| Alex_Gaynor | Rhy0lite: https://bitbucket.org/fijal/hack2/src/a8eb5c9e36cd/bench/laplace/laplace.py | 20:43 |
| Alex_Gaynor | that look right? | 20:43 |
| Rhy0lite | yes | 20:43 |
| Alex_Gaynor | Rhy0lite: which version do you want the ASM for? | 20:43 |
| Rhy0lite | numeric_3_time_step | 20:43 |
| Alex_Gaynor | k | 20:43 |
| tilgovi (~randall@69.111.59.168) joined #pypy. | 20:44 | |
| gutworth | punpckldq is a gcc instruction :) | 20:44 |
| gutworth | x86 rather | 20:44 |
| tilgovi (~randall@69.111.59.168) left irc: Changing host | 20:44 | |
| tilgovi (~randall@couchdb/developer/tilgovi) joined #pypy. | 20:44 | |
| Rhy0lite | gutworth: eieio is a PowerPC instruction | 20:44 |
| Alex_Gaynor | gutworth: yeah, it's a packed.... something quadword | 20:44 |
| stakkars_ (~stakkars@89.204.153.195) left irc: Quit: Colloquy for iPhone - http://colloquy.mobi | 20:45 | |
| Alex_Gaynor | Rhy0lite: do you know which loop you want? | 20:47 |
| Rhy0lite | no | 20:47 |
| Alex_Gaynor | do you remember what arithmatic ops were in it? | 20:47 |
| Rhy0lite | it should be addsd, mulsd | 20:48 |
| Rhy0lite | four getarrayitem_raw followed by setarryitem_raw | 20:49 |
| kenaan | 12hakanardo jit-targets 11417d2384375b 15/pypy/jit/metainterp/: Dont rgc.collect() more than we have too (not working when run in a separate process by test_all) | 20:49 |
| Alex_Gaynor | ok found it I think | 20:49 |
| Action: mikefc is working on his last bug report. | 20:50 | |
| mikefc | I think it's memory thrashing rather than cpu. | 20:50 |
| mikefc | Any ideas on how to profile total memory use during a run? | 20:51 |
| Alex_Gaynor | Rhy0lite: http://dpaste.com/666636/ is what we generate now | 20:51 |
| Alex_Gaynor | err shit I screwed it up a bit, one moment | 20:52 |
| Alex_Gaynor | Rhy0lite: http://dpaste.com/666637/ need to close my laptop now | 20:52 |
| Rhy0lite | time for all the students to put their heads down in class :-) | 20:53 |
| arigato | i63 = i5 < i33 | 20:54 |
| arigato | mul r14,QWORD PTR [rbp-0x120] | 20:54 |
| arigato | mul 0xdc79e3ae | 20:54 |
| arigato | I wouldn't hesitate to say here: "uh?" | 20:54 |
| Rhy0lite | is getinteriorfield_raw new? | 20:56 |
| asmeurer__ (~asmeurer@dhcp-baca-230.resnet.nmt.edu) joined #pypy. | 20:56 | |
| Rhy0lite | r14 isn't the worst | 20:58 |
| Rhy0lite | the updates to r15, r13, r10 and r8 seem bad | 20:58 |
| arigato | what is wrong with r15? | 21:00 |
| CIA-13 | 03mikefc 07roundup * 10#954/looped string formatting of dict entry slow and hogging memory: | 21:01 |
| CIA-13 | OK. This is probably just a case where pypy version is just running out of memory | 21:01 |
| CIA-13 | and thrashing swap. * 14https://bugs.pypy.org/issue954 | 21:01 |
| arigato | ah, they are all incremented by what turns out to be the same value? | 21:01 |
| Rhy0lite | arigato: it may not be as bad | 21:01 |
| Rhy0lite | what I am looking for is a read-after-write cycle | 21:02 |
| Rhy0lite | which is the problem shown by Maciej's inplace_laplace.cxx example | 21:02 |
| Rhy0lite | if the Python loop is not as fast as laplace.cxx, there is some memory dependency being introduced | 21:02 |
| Rhy0lite | for inplace_laplace.cxx, it was the data itself | 21:03 |
| Rhy0lite | that does not appear to be the problem for the JIT code, but somewhere there is a memory dependency in the addressing code | 21:03 |
| Rhy0lite | maybe it now is with rcx and rdx | 21:05 |
| stakkars_ (~stakkars@89.204.153.195) joined #pypy. | 21:05 | |
| stakkars_ | Iphone test | 21:05 |
| gutworth (~benjamin@dhcp-128-36-84-91.central.yale.edu) left irc: Ping timeout: 252 seconds | 21:05 | |
| Rhy0lite | arigato: what is difficult to determine from the code is the addresses in the stack | 21:05 |
| Rhy0lite | mov rcx,QWORD PTR [rbp-0x110] | 21:06 |
| Rhy0lite | mov rdx,QWORD PTR [rbp-0x118] | 21:06 |
| gutworth (~benjamin@dhcp-128-36-84-91.central.yale.edu) joined #pypy. | 21:06 | |
| Rhy0lite | the offsets from rbp are independent, but the addresses may be the same as some other write | 21:07 |
| stakkars_ | arigato: About refcounting: yes, I'm saying that if is not too hard, it would be nice to offer this. | 21:07 |
| arigato | Rhy0lite: true | 21:07 |
| Rhy0lite | arigato: if the code is slow, I suspect that the previous iteration of the loop is writing to the same address | 21:08 |
| stakkars_ | That's the last bastion where people can have a "yes but". It is a psycological thing | 21:08 |
| arigato | Rhy0lite: but that should be kind of obvious from the Python sources, no? | 21:08 |
| Rhy0lite | no | 21:08 |
| stakkars_ | The cosy feeling that there is nothing different if they want, although slower. Then move to a better gc. See it? | 21:09 |
| arigato | stakkars_: feel free, but it's tons of work | 21:10 |
| arigato | not for me :-) | 21:10 |
| arigato | I mean, it's not something I want to do, at all | 21:10 |
| stakkars_ | waah, well | 21:10 |
| arigato | Rhy0lite: this said, I'm sure that gcc produces a loop that is at least half as short | 21:11 |
| Rhy0lite | arigato, the length of the loop is not the problem | 21:11 |
| Rhy0lite | x86 is really good and running cr*ppy code well | 21:12 |
| stakkars_ | I hate Windows, but I do what people want, that's my goal. Undecided about ref gc, tho | 21:12 |
| arigato | Rhy0lite: it still contains many more guards than the C++ version of the loop, which I guess only has one | 21:13 |
| arigato | for example | 21:13 |
| Rhy0lite | http://paste.pocoo.org/show/515473/ | 21:13 |
| Rhy0lite | arigato: 2x should not be due to guards | 21:14 |
| goodwill (~goodwill_@pdpc/supporter/active/goodwill) joined #pypy. | 21:14 | |
| goodwill (goodwill_@pdpc/supporter/active/goodwill) left #pypy ("+++ OK ATH OK"). | 21:15 | |
| lucian (~lucian@cpc1-newc15-2-0-cust84.gate.cable.virginmedia.com) left irc: Remote host closed the connection | 21:15 | |
| arigato | I'm kind of impressed that we are *only* 2x slower, actually, given the mess of [rbp-0x*] that we have and that is not at all in the gcc version | 21:15 |
| Rhy0lite | arigato: the top code is good (laplace.cxx) and the bottom code is worse (inplace_laplace.cxx) | 21:15 |
| Rhy0lite | arigato: x86 is good with memory reads | 21:15 |
| Rhy0lite | it can pipeline it a lot | 21:15 |
| Rhy0lite | the problem is a Load-Hit-Store | 21:15 |
| kkris (~kris@93-82-40-130.adsl.highway.telekom.at) left irc: Quit: Leaving. | 21:16 | |
| Rhy0lite | where a load from memory hits a story in the previous iteration | 21:16 |
| Rhy0lite | in the bottom code | 21:16 |
| Rhy0lite | this happens on line 71 | 21:16 |
| Rhy0lite | movsd %xmm1, 8(%rdx,%rax) | 21:16 |
| Rhy0lite | when %rax gets updated for the next iteration | 21:16 |
| Rhy0lite | it hits | 21:16 |
| Rhy0lite | movsd (%rdx,%rax), %xmm2 | 21:16 |
| Alex_Gaynor | Rhy0lite: my second paste fixes teh weird MULSD issue :) | 21:17 |
| Rhy0lite | the top one does not have the conflict | 21:17 |
| Rhy0lite | movsd %xmm1, 8(%rdi,%rax) | 21:17 |
| Rhy0lite | %rdi points to a separate array | 21:17 |
| arigato | Rhy0lite: note that the two C++ versions of the code are not doing the same thing, as far as I can tell | 21:17 |
| Rhy0lite | arigato: yes, the algorithms are different | 21:17 |
| Rhy0lite | but Maciej was playing around trying to understand the source of the pypy slowdown | 21:18 |
| arigato | I mean, they compute something really different | 21:18 |
| arigato | it's not just using a different algo to compute the same thing | 21:18 |
| Rhy0lite | they both are laplace solvers | 21:18 |
| arigato | no | 21:18 |
| arigato | I think not | 21:18 |
| Rhy0lite | but different convergence characteristics | 21:18 |
| arigato | the 2nd algo does not satisfy any sane mathematical definition that I can think of | 21:19 |
| Rhy0lite | hehe | 21:19 |
| kenaan | 12mattip matrixmath-dot 119160acf08843 15/pypy/: Merge with default | 21:19 |
| Rhy0lite | talk to Maciej :-) | 21:19 |
| kenaan | 12mattip matrixmath-dot 110f00b31fe95a 15/pypy/module/micronumpy/interp_numarray.py: fix for renamed class | 21:19 |
| kenaan | 12mattip matrixmath-dot 1155693d8b2b89 15/: Merge with default | 21:19 |
| kenaan | 12mattip matrixmath-dot 1155f149559e25 15/pypy/module/micronumpy/: bad merge | 21:19 |
| kenaan | 12mattip matrixmath-dot 1109f8917ecaf3 15/pypy/module/micronumpy/test/test_compile.py: bad merge | 21:19 |
| kenaan | 12mattip matrixmath-dot 11885d36165f89 15/pypy/module/micronumpy/: experimental approach to dot iterator problem | 21:19 |
| kenaan | 12mattip matrixmath-dot 11dc6c3373f647 15/pypy/module/micronumpy/: clean compile a bit | 21:20 |
| kenaan | 12mattip matrixmath-dot 111ae4feb97953 15/pypy/module/micronumpy/: Backed out changeset: 885d36165f89 | 21:20 |
| kenaan | 12mattip matrixmath-dot 115a5cfa32fe70 15/pypy/module/micronumpy/: merge | 21:20 |
| kenaan | 12mattip matrixmath-dot 1126e6b2238d4a 15/pypy/module/micronumpy/interp_numarray.py: translation fixes: avoid negative slices | 21:20 |
| kenaan | 12mattip matrixmath-dot 11beba9400c9dd 15/pypy/module/micronumpy/: add bin_impl_one_dim, would be nice to have some tests | 21:20 |
| kenaan | 12mattip matrixmath-dot 11699511d4d5d5 15/pypy/module/micronumpy/test/test_numarray.py: add failing test for sum return value | 21:20 |
| kenaan | 12mattip matrixmath-dot 11ee2362c0cec8 15/pypy/module/micronumpy/interp_numarray.py: need to fix failing sum test before continuing | 21:20 |
| kenaan | 12mattip matrixmath-dot 11924460a509cd 15/pypy/module/micronumpy/: expose promote_to_largest to reduce functions, fixes sum() prod() bug without ruining mean() | 21:20 |
| kenaan | 12mattip matrixmath-dot 11d28b98fc74ed 15/pypy/module/micronumpy/: dot works | 21:20 |
| kenaan | 12mattip matrixmath-dot 118ee1a24557b8 15/pypy/module/micronumpy/interp_numarray.py: dot seems to work, more of a proof-of-concept than usable | 21:20 |
| mattip | sorry about that, didn't mean to interrupt :) | 21:20 |
| arigato | :-) | 21:20 |
| Rhy0lite | arigato: I'm just using the two to highlight the performance characteristics | 21:20 |
| arigato | ok | 21:20 |
| Rhy0lite | one has a read-after-write dependency | 21:21 |
| Rhy0lite | the other does not | 21:21 |
| arigato | yes | 21:21 |
| arigato | ok | 21:21 |
| Rhy0lite | arigato: also I only am showing the core loop | 21:21 |
| Rhy0lite | for the first one, tmp[][] = u[][] for each loop | 21:21 |
| Rhy0lite | I didn't show the memcpy | 21:21 |
| arigato | yes (I already discussed this with maciej, and the 2nd version looks to me like plain buggy code, but I see your point) | 21:22 |
| Rhy0lite | I believe the dependency is the bottleneck | 21:22 |
| arigato | ah | 21:22 |
| Rhy0lite | and the reason for the slowdown of the C++ code | 21:22 |
| squiddy (~squiddy@g224193176.adsl.alicedsl.de) left irc: Quit: Leaving | 21:22 | |
| Rhy0lite | and I believe there is some dependency in the pypy code | 21:23 |
| Rhy0lite | not in the data, but in the addresses stored in the stack (rbp) | 21:23 |
| Rhy0lite | my intuition is one of the writes to memory is read to initialize rcx or rdx in each loop iteration | 21:24 |
| arigato | but there is only one setinteriorfield_raw? | 21:25 |
| Rhy0lite | ah hah! | 21:25 |
| Rhy0lite | now you see why it's difficult to read the pypy jitviewer code | 21:25 |
| arigato | wait, I didn't mean it that way | 21:26 |
| arigato | (the jitviewer code is best seen colored, too) | 21:26 |
| arigato | or else I don't understand your intuition | 21:26 |
| Rhy0lite | for instance, line 14 | 21:27 |
| aboudreault (~alanb@osgeo/member/aboudreault) left irc: Ping timeout: 255 seconds | 21:27 | |
| Rhy0lite | mov rcx,QWORD PTR [rbp-0x110] | 21:27 |
| Rhy0lite | 16: movsd xmm9,QWORD PTR [rcx+r8*8] | 21:27 |
| Rhy0lite | the jit is loading rcx from the address at that stack location | 21:27 |
| Rhy0lite | on each iteration | 21:27 |
| arigato | yes | 21:27 |
| Rhy0lite | reading isn't bad | 21:27 |
| Rhy0lite | but I suspect one of the iteration index updates writes to that same location | 21:28 |
| Rhy0lite | I cannot tell because the SSA variables are independent | 21:28 |
| Rhy0lite | and the stack offsets are different | 21:28 |
| arigato | no, nothing writes to rbp-0x110 | 21:28 |
| jterrace | numpypy only has float64 but not float32? | 21:28 |
| Rhy0lite | arigato: yes | 21:28 |
| Rhy0lite | but that does not matter | 21:28 |
| Rhy0lite | it's updating [rbp-0x110] | 21:28 |
| arigato | ? | 21:29 |
| Rhy0lite | what if [rpb-0x1d0] points to the same memory? | 21:29 |
| arigato | uh? | 21:29 |
| Rhy0lite | mov QWORD PTR [rbp-0x1d0],r15 | 21:29 |
| arigato | of course [rbp-0x110] and [rbp-0x1d0] point to different memory locations | 21:29 |
| arigato | rbp is the frame pointer | 21:29 |
| Rhy0lite | why? | 21:29 |
| Rhy0lite | stop | 21:29 |
| Rhy0lite | the values at those locations | 21:30 |
| Rhy0lite | if [rbp-0x110] and [rbp-0x1d0] both are 0x12345678 | 21:30 |
| arigato | ah, /me rereads | 21:30 |
| amaury__ | jterrace: there is a "floating" type | 21:30 |
| amaury__ | defined with "float32" things internally | 21:31 |
| mikefc | jterrace: pretty sure it has a float32 dtype=f4 | 21:31 |
| jterrace | numpypy.float64 exists, and you can set dtype='float32' with a string, but there is no numpypy.float32 | 21:31 |
| amaury__ | sorry, I don't know anything about numpy :( | 21:31 |
| amaury__ | did you try dtype='floating' | 21:31 |
| amaury__ | ? | 21:31 |
| jterrace | i think it's just missing from the global module scope | 21:31 |
| Alex_Gaynor | jterrace: err we have float32, I bet I didn't expose the type though | 21:31 |
| jterrace | yeah i think you just forgot it in the global module scope | 21:31 |
| jterrace | because it works as a string | 21:31 |
| Alex_Gaynor | jterrace: yup, that's the issue | 21:32 |
| lindsay_l (~lindsay_l@96.24.71.70) joined #pypy. | 21:32 | |
| arigato | Rhy0lite: sorry, I don't understand what you mean about the two lines you pasted above | 21:32 |
| Alex_Gaynor | jterrace: I'll fix it in a little while (in class ATM) | 21:32 |
| jterrace | thanks | 21:32 |
| Rhy0lite | arigato: remember that x86 instructions can perform lots of implicit indirections | 21:32 |
| Rhy0lite | I don't know if the exact lines I quoted are the problem | 21:33 |
| Rhy0lite | I completely agree that rbp-0x110 and rbp-0x1d0 are independent addresses | 21:33 |
| Rhy0lite | but [rbp-0x110] and [rbp-0x1d0] could be the same address | 21:33 |
| Rhy0lite | in other words the address contained at those two frame pointer locations | 21:33 |
| Rhy0lite | on line 14 and 16 | 21:34 |
| Rhy0lite | one loads rcx | 21:34 |
| Rhy0lite | and then loads relative to rcx | 21:34 |
| Rhy0lite | on line 36, one is storing | 21:34 |
| arigato | yes, but I don't see the point. are you saying that the CPU is getting lost by doing "write at [ecx]; read at [edx]" because the value in ecx and edx are the same? | 21:34 |
| Rhy0lite | I *suspect* that one of the stores is hitting the same address | 21:34 |
| Rhy0lite | not lost | 21:35 |
| Rhy0lite | but needs to wait | 21:35 |
| Rhy0lite | the CPU is loading rcx | 21:35 |
| arigato | I doubt it a bit, but I may be wrong | 21:35 |
| Rhy0lite | arigato: something is updating the array indexes | 21:35 |
| Rhy0lite | and it's not done inline | 21:35 |
| Rhy0lite | and there is no other reason to write r15, for example | 21:36 |
| arigato | r15 is saved because of the guards that follow | 21:36 |
| arigato | one of these guards has the old value of r15 as "fail_args" | 21:36 |
| Rhy0lite | ok | 21:36 |
| mattip (chatzilla@bzq-79-177-116-180.red.bezeqint.net) left #pypy. | 21:36 | |
| Rhy0lite | based on the code | 21:37 |
| stakkars_ | arigato: reminds me of instruction sequences which could even kill a Cpu :-) | 21:37 |
| Rhy0lite | I suspect that rax and rdx are the neighbors above and below | 21:37 |
| Rhy0lite | and that rcx+r13*8 and rcx+r8*8 are the neighbors left and right | 21:37 |
| `fox` (~fox@95.232.111.114) joined #pypy. | 21:38 | |
| Rhy0lite | arigato: maybe the slowdown all is due to guards | 21:39 |
| Rhy0lite | writing to the index value in memory and then reading it would cause problems | 21:40 |
| Rhy0lite | if you look back at the gcc code, it always is reading from memory to set the base addresses | 21:41 |
| arigato | we are comparing 16 asm instructions using registers with 46 asm instructions using the frame, so it really doesn't seem like a complete surprize that the latter is 2x slower | 21:41 |
| arigato | Rhy0lite: ah | 21:41 |
| arigato | sorry, I think I begin to understand what you mean | 21:42 |
| Rhy0lite | addq $8, %rbx | 21:42 |
| Rhy0lite | addq $8, %r9 | 21:42 |
| Rhy0lite | movq 8(%r11,%rbx), %rdi | 21:42 |
| Rhy0lite | movq (%r9), %rsi | 21:42 |
| arigato | you are saying that storing something at rbp-xx and then reading it next iteration is bad | 21:42 |
| Rhy0lite | arigato: not exactly | 21:42 |
| jterrace | Alex_Gaynor: is there a string type for numpypy yet? | 21:42 |
| Rhy0lite | I am saying that storing something at [rbp-xx] and then reading it next iteration is bad | 21:43 |
| jterrace | np.array(['hey'], dtype='string') -> infinite loop :) | 21:43 |
| Rhy0lite | arigato: as we agree, the offsets only are used one | 21:43 |
| Rhy0lite | once | 21:43 |
| Rhy0lite | I don't see anything read and write from [rbp-NN] | 21:43 |
| Rhy0lite | but if [rbp-NN] and [rbp-MM] are the same | 21:43 |
| derdon (~derdon@p5DE894EA.dip.t-dialin.net) joined #pypy. | 21:44 | |
| Rhy0lite | then pypy may be reading and writing to the same address in each iteration | 21:44 |
| Rhy0lite | if it reads from [rbp-NN] and writes to [rbp-MM] | 21:44 |
| arigato | sorry, what is the 4 lines of assembler you pasted above? | 21:44 |
| Rhy0lite | the four lines above are the core of the update in the gcc outer loop iteration | 21:44 |
| Rhy0lite | it's marching through an index of the addresses for each C++ vector | 21:45 |
| Rhy0lite | updating rdi, rsi, rcx, rdx | 21:45 |
| Rhy0lite | but not writing the address that is read into those registers | 21:45 |
| Rhy0lite | maybe all of the problem is due to the extra frame references by pypy | 21:46 |
| arigato | I don't really follow you. why does anything done in the outer loop have a relevance to performance? | 21:46 |
| Rhy0lite | arigato: I was trying to show that the gcc loop never writes addresses to set up the vectors that it indexes | 21:47 |
| Rhy0lite | I agree that the inner loop update is just addq $8, %rax | 21:48 |
| Alex_Gaynor | jterrace: err, no we don't have a string dtype yet | 21:49 |
| jterrace | ok | 21:49 |
| jterrace | it goes into an infinite loop and keeps using more and more memory | 21:49 |
| jterrace | if you try and use dtype='string' | 21:49 |
| Alex_Gaynor | heh, ok I'll take a look at that as well | 21:50 |
| Alex_Gaynor | jterrace: I can't for the live of me figur eout why though | 21:51 |
| dracman (~draco@212.255.33.90) joined #pypy. | 21:51 | |
| jterrace | heh | 21:51 |
| jterrace | ctrl+c fails too | 21:51 |
| jterrace | had to kill it | 21:51 |
| arigato | I really think that ignoring the frame, the memory accesses are exactly the same as gcc, and that should be good enough for the cpu --- | 21:51 |
| arigato | but it still has 3 times as many instructions to run, and maybe occasionally gets cache conflicts between ebp-xx and the matrix | 21:51 |
| arigato | sorry if you're trying to tell something different and I keep missing it | 21:52 |
| ramusara (~ramusara@220.156.210.236.user.e-catv.ne.jp) left irc: Quit: Leaving... | 21:52 | |
| stakkars_ (~stakkars@89.204.153.195) left irc: Quit: Colloquy for iPhone - http://colloquy.mobi | 21:53 | |
| Rhy0lite | arigato: I cannot say for sure because it is difficult to know what the addresses resolve to | 21:55 |
| Rhy0lite | compared to the straight-forward indexing of the GCC code | 21:55 |
| Rhy0lite | maybe the slowdown is all of the extra guard code | 21:55 |
| Rhy0lite | and we're looking at it different ways | 21:56 |
| Rhy0lite | because the guard code does require extra writes | 21:56 |
| Rhy0lite | although if the writes never are used in the critical path, they can percolate out of the store queue very slowly and not matter | 21:56 |
| Rhy0lite | arigato: the problem in modern processors is the memory wall | 21:59 |
| Rhy0lite | and, apparently because of guards, pypy performs a lot more operations with memory | 21:59 |
| arigato | yes, but still: e.g. we do 10 "add" where gcc does one, so this has a cost, surely | 22:00 |
| Rhy0lite | yes, but it won't make much of a difference | 22:00 |
| arigato | I'm not sure what you mean by guards that require memory operations | 22:01 |
| Rhy0lite | cmp r14,QWORD PTR [rbp-0x120] | 22:01 |
| Rhy0lite | jge 0xdc79e3ae | 22:01 |
| arigato | this QWORD PTR is just loading Yet Another value from the frame | 22:01 |
| Rhy0lite | as opposed to GCC performing register-register comparisons | 22:01 |
| Rhy0lite | cmpq %rbp, %rbx | 22:02 |
| arigato | it's not directly the fault of the guard | 22:02 |
| Rhy0lite | gcc's inner loop comparison is cmpq %r8, %rax | 22:02 |
| Rhy0lite | in other words, gcc's update is | 22:03 |
| Rhy0lite | addq $8, %rax | 22:03 |
| Rhy0lite | cmpq %r8, %rax | 22:03 |
| Rhy0lite | pypy is | 22:03 |
| Rhy0lite | cmp r14,QWORD PTR [rbp-0x120] | 22:03 |
| Rhy0lite | jge 0xdc79e3ae | 22:03 |
| arigato | it's just that we are using something like 20 or 25 different values in the loop, not counting the xmm ones | 22:04 |
| Rhy0lite | mov rcx,r14 | 22:04 |
| Rhy0lite | add r14,0x1 | 22:04 |
| arigato | so of course, it's actually good that all variables that are used only once in a loop at directly accessed from the frame | 22:04 |
| arigato | like the value to compare r14 to | 22:05 |
| Rhy0lite | arigato: but, for instance, there should be additional free registers | 22:05 |
| arigato | there are none | 22:05 |
| arigato | rbp is the frame pointer, r11 is used as the scratch register | 22:05 |
| arigato | all others are in use | 22:05 |
| lindsay_l (~lindsay_l@96.24.71.70) left irc: Quit: Leaving | 22:07 | |
| `fox` (~fox@95.232.111.114) left irc: Ping timeout: 252 seconds | 22:08 | |
| arigato | (r11 is not actually *used* in this loop, but I mean, it is used generally as the scratch register) | 22:08 |
| Rhy0lite | okay | 22:09 |
| Rhy0lite | but pypy seems to have a lot of induction variables | 22:09 |
| Rhy0lite | not all of which should be necessary | 22:09 |
| Rhy0lite | though that is a natural outcome of the guards for the trace | 22:10 |
| Rhy0lite | To compress the code, one needs to see that some of the guards can be combined, right? | 22:10 |
| arigato | that would be advanced by our standards, but yes | 22:11 |
| arigato | maybe it's a generally useful feature to try to combine at least the base addresses together | 22:12 |
| arigato | in this case it's 5 different values that probably all point to the very same array | 22:13 |
| Rhy0lite | yes | 22:13 |
| arigato | I'm also not sure why we get 10 adds | 22:13 |
| Rhy0lite | one may need 3 base addresses, but not 5 | 22:14 |
| Rhy0lite | anyway, I need to drive home | 22:17 |
| arigato | ok, see you | 22:18 |
| Rhy0lite (dje@nat/ibm/x-mkylutffxovkxuzo) left irc: Quit: Leaving | 22:19 | |
| Nick change: Gulaway -> Gulopine | 22:20 | |
| Nick change: amaury__ -> amaury | 22:22 | |
| tilgovi (~randall@couchdb/developer/tilgovi) left irc: Remote host closed the connection | 22:25 | |
| arigato (~arigo@82.113.121.170) left irc: Quit: See you | 22:30 | |
| raydeo (~raydeo@merickel.org) got netsplit. | 22:31 | |
| gtaylor (~gtaylor@108-196-160-120.lightspeed.tukrga.sbcglobal.net) left irc: Quit: Konversation terminated! | 22:33 | |
| bbot2 | Started: 15http://buildbot.pypy.org/builders/jit-benchmark-linux-x86-64/builds/161 | 22:37 |
| bbot2 | Started: 15http://buildbot.pypy.org/builders/own-macosx-x86-32/builds/735 | 22:37 |
| bbot2 | Started: 15http://buildbot.pypy.org/builders/jit-benchmark-linux-x86-32/builds/965 | 22:37 |
| chronitis (~chronitis@88-104-122-204.dynamic.dsl.as9105.com) left irc: Quit: Lost terminal | 22:41 | |
| raydeo (~raydeo@merickel.org) got lost in the net-split. | 22:42 | |
| aurojit (~apanda@dhcp-44-187.EECS.Berkeley.EDU) left irc: Read error: Connection reset by peer | 22:42 | |
| apanda (~apanda@dhcp-44-187.EECS.Berkeley.EDU) joined #pypy. | 22:42 | |
| tilgovi (~randall@c-98-210-155-124.hsd1.ca.comcast.net) joined #pypy. | 22:42 | |
| tilgovi (~randall@c-98-210-155-124.hsd1.ca.comcast.net) left irc: Changing host | 22:42 | |
| tilgovi (~randall@couchdb/developer/tilgovi) joined #pypy. | 22:42 | |
| dracman (~draco@212.255.33.90) left irc: Ping timeout: 252 seconds | 22:45 | |
| kenaan | 12alex_gaynor default 1124a17a8610e1 15/pypy/module/micronumpy/: expose numpy.float32, thanks to jterrance for pointing it out | 22:50 |
| Alex_Gaynor | jterrace: ^ | 22:53 |
| jterrace | nice | 22:53 |
| jterrace | but got my name wrong :) | 22:53 |
| jterrace | but don't worry, people mispell it all the time | 22:53 |
| Action: Alex_Gaynor shifts awkwardly | 22:54 | |
| jterrace | hehe | 22:54 |
| pjenvey | nobody believes the spelling of anything in alex's commit messages anyway | 22:56 |
| Alex_Gaynor | pjenvey: this is the truth | 22:57 |
| jterrace | up to 19/46 tests passing with pypy: https://twitter.com/#!/jterrace/status/143827324247355393 | 22:58 |
| jterrace | i think reshape branch will bump that up by a lot | 22:58 |
| Alex_Gaynor | jterrace: are the tests easy to run? | 22:58 |
| Alex_Gaynor | i.e. you I go run them | 22:58 |
| jterrace | yeah | 22:58 |
| jterrace | it's just, download nightly pypy, create a virtualenv, and run setup.py | 22:59 |
| Alex_Gaynor | err i.e. *should* I go run them | 22:59 |
| jterrace | hmm? | 22:59 |
| jterrace | oh you mean on reshape branch? | 22:59 |
| Alex_Gaynor | err, link to the repo? | 23:00 |
| Alex_Gaynor | dunno, I just want to validate that there's no actual stuff to do besides reshape | 23:00 |
| jterrace | https://github.com/pycollada/pycollada/tree/pypy | 23:00 |
| jterrace | if you just run venv/bin/pypy collada, it runs the test suite | 23:01 |
| Alex_Gaynor | k, gotta translate first | 23:01 |
| jterrace | setup.py will install the only deps it needs - unittest2 and python-dateutil | 23:01 |
| jterrace | does the buildbot build branches? | 23:01 |
| Alex_Gaynor | if you ask it to | 23:02 |
| jterrace | not sure how useful the tests would be, because other things will probably fail | 23:03 |
| jterrace | i could help write small reshape tests | 23:03 |
| jterrace | which branch is reshape on? none of the numpy branch names jump out at me | 23:03 |
| Alex_Gaynor | matrixmath-reshape-merge I think | 23:04 |
| jterrace | do i have to do a full build to try out the branch? | 23:04 |
| jterrace | or can i run the tests without building the full translator? | 23:05 |
| Alex_Gaynor | you can run it, it'll be excruciatiatelingly slow though, like 2000x slower | 23:05 |
| jterrace | hmm if it's just one test module, maybe x isn't that bad though? | 23:06 |
| Alex_Gaynor | dunno, only one way to find out | 23:06 |
| jterrace | these reshape tests look pretty good | 23:07 |
| apanda (~apanda@dhcp-44-187.EECS.Berkeley.EDU) left irc: Read error: Connection reset by peer | 23:08 | |
| aurojit (~apanda@dhcp-44-187.EECS.Berkeley.EDU) joined #pypy. | 23:08 | |
| lac_ | snow! SNOW! WE HAVE snow! :-) :-) (wish armin was around) | 23:10 |
| mikefc | snow. pffft | 23:12 |
| jterrace | raining here (NJ,US) | 23:12 |
| whitelynx|work (~whitelynx@63.241.75.144) left irc: Quit: Ex-Chat | 23:13 | |
| Alex_Gaynor | just dark here (NY, US) | 23:14 |
| mikefc | a nice overcast 21C (Bris, Aus) | 23:15 |
| harrison (~quassel@adsl-69-209-194-12.dsl.chcgil.sbcglobal.net) joined #pypy. | 23:15 | |
| jterrace | Alex_Gaynor: is this right? ./pytest.py pypy/module/micronumpy/test/test_numarray.py | 23:17 |
| Alex_Gaynor | yup | 23:17 |
| jterrace | it didnt take that long at all | 23:17 |
| jterrace | 107 passes, 1 skipped | 23:17 |
| Alex_Gaynor | sorry, I thought you meant running *your* tests with the untralsated pypy | 23:17 |
| jterrace | oh | 23:17 |
| jterrace | i can even do that? | 23:17 |
| raydeo (~raydeo@merickel.org) joined #pypy. | 23:17 | |
| Alex_Gaynor | sure ./pypy/bin/py.py <your code> | 23:17 |
| Rhyolite | lac_: If it were easy, I would be happy to send snow from my home | 23:18 |
| dracman (~draco@212.255.26.64) joined #pypy. | 23:18 | |
| jterrace | oh that's super cool | 23:20 |
| jterrace | did not know that was possible | 23:20 |
| mcdonc (~mcdonc@ip72-209-213-54.dc.dc.cox.net) left irc: Read error: Operation timed out | 23:21 | |
| mfoord (~anonymous@87-194-212-65.bethere.co.uk) joined #pypy. | 23:21 | |
| mfoord (~anonymous@87-194-212-65.bethere.co.uk) left irc: Changing host | 23:21 | |
| mfoord (~anonymous@python/psf/voidspace) joined #pypy. | 23:21 | |
| mwhudson (~mwh@linaro/mwhudson) left irc: Quit: Leaving | 23:21 | |
| fzzzy (~donovan@76-198-130-19.lightspeed.mtvwca.sbcglobal.net) joined #pypy. | 23:21 | |
| mwhudson_ (~mwh@120.136.5.22) joined #pypy. | 23:22 | |
| mcdonc (~mcdonc@ip72-209-213-54.dc.dc.cox.net) joined #pypy. | 23:23 | |
| Nick change: mwhudson_ -> mwhudson | 23:26 | |
| ojii (~ojii@202-91.78-83.cust.bluewin.ch) left irc: Quit: Leaving | 23:31 | |
| tlynn (~tlynn@cpc6-cmbg14-2-0-cust121.5-4.cable.virginmedia.com) joined #pypy. | 23:39 | |
| jterrace | Alex_Gaynor: played around with adding a few more tests of my own to matrixmath-reshape-merge branch, and all worked correctly | 23:40 |
| jterrace | looks awesome | 23:40 |
| Alex_Gaynor | cool | 23:41 |
| jterrace | my tests: http://pastebin.com/raw.php?i=89ukxJMG | 23:42 |
| Vorpal (~AnMaster@unaffiliated/anmaster) left irc: Ping timeout: 248 seconds | 23:43 | |
| nedbat (~nedbat@python/psf/nedbat) left irc: Ping timeout: 260 seconds | 23:45 | |
| Action: mikefc is excited about the numpypy work. | 23:45 | |
| Action: jterrace agrees | 23:45 | |
| tlynn_ (~tlynn@cpc6-cmbg14-2-0-cust121.5-4.cable.virginmedia.com) joined #pypy. | 23:46 | |
| mikefc | hope to delve into it soon. (still working on starting my pickle numpy problem) | 23:46 |
| jterrace | did you look at the npy format? | 23:47 |
| jterrace | looks not that complicated: https://github.com/numpy/numpy/blob/master/doc/neps/npy-format.txt | 23:47 |
| jterrace | i think that's what's used for pickling | 23:47 |
| mikefc | nope. I'm only slowly wrapping my head around how pypy works. | 23:48 |
| JaRoel|4d (~jaroel|4d@2001:980:7133:1:94a1:1366:80f8:ee54) left irc: Remote host closed the connection | 23:48 | |
| jterrace (~jterrace@aegis.CS.Princeton.EDU) left irc: Quit: Leaving. | 23:49 | |
| tlynn (~tlynn@cpc6-cmbg14-2-0-cust121.5-4.cable.virginmedia.com) left irc: Ping timeout: 240 seconds | 23:49 | |
| mikefc | and what the hell is entailed in pickling. I know nothing of python internals :( | 23:49 |
| Rhyolite | first you create a Python brine | 23:49 |
| exarkun | pickling is just turning an object into some bytes and vice versa | 23:50 |
| cwillu_at_work (~cwillu@cwillu-1-pt.tunnel.tserv13.ash1.ipv6.he.net) left irc: Ping timeout: 252 seconds | 23:51 | |
| mikefc | exark: yeah, I know what pickling does, but not how it actually does it internally in python. when is __reduce__ used, when should get/setstate and getinitargs be used. I have to find time to figure these out etc. | 23:52 |
| mikefc | afk | 23:52 |
| Alex_Gaynor | I'd be lieing if I said there are more than about 50 people in the world who properly understand the answers to those questions. | 23:53 |
| exarkun | but you're just implementing numpy right? so copy whatever numpy does | 23:53 |
| mikefc | exarkun: possibly. but i had a look at at what pypy was doing in micronumpy and didn't understand any of it :) | 23:54 |
| mikefc | ufuncs and spaces | 23:54 |
| mikefc | like i said - learning | 23:55 |
| Alex_Gaynor | ufuncs are a numpy concept, objspace is a pypy one | 23:55 |
| mikefc | the comments in pypy code can be quite ... um... absent? :) | 23:55 |
| Alex_Gaynor | I'm not a good commenter :/ | 23:56 |
| mfoord (~anonymous@python/psf/voidspace) left irc: Quit: mfoord | 23:59 | |
| --- Tue Dec 6 2011 | 00:00 | |
Generated by irclog2html.py 2.9.2 by Marius Gedminas - find it at mg.pov.lt!