#pypy IRC log for Monday, 2011-11-28

asmeurer_ (~asmeurer@dhcp-baca-230.resnet.nmt.edu) joined #pypy.00:04
tilgovi (~randall@couchdb/developer/tilgovi) joined #pypy.00:07
jimbaker` (~jbaker@c-75-71-80-146.hsd1.co.comcast.net) left irc: Ping timeout: 248 seconds00:09
nedbat (~nedbat@python/psf/nedbat) joined #pypy.00:39
thinker (~thinker@123-194-52-90.dynamic.kbronet.com.tw) joined #pypy.00:41
derdon (~derdon@p5DE8B220.dip.t-dialin.net) left irc: Remote host closed the connection00:49
bbot23Success: 15http://buildbot.pypy.org/builders/jit-benchmark-linux-x86-32/builds/95700:55
tav (~tav@host-2-96-35-150.as13285.net) joined #pypy.00:56
mfoord (~anonymous@python/psf/voidspace) left irc: Quit: mfoord01:14
mp (maciej@lair.hell.cx) left #pypy.01:20
bbot24Failure: 15http://buildbot.pypy.org/builders/own-macosx-x86-32/builds/72601:24
Nick change: Guest81536 -> masquerade01:33
Vorpal (~AnMaster@unaffiliated/anmaster) left irc: Ping timeout: 244 seconds01:36
bbot2Started: 15http://buildbot.pypy.org/builders/own-linux-x86-32/builds/186501:40
bbot2Started: 15http://buildbot.pypy.org/builders/pypy-c-jit-linux-x86-64/builds/59201:40
bbot2Started: 15http://buildbot.pypy.org/builders/pypy-c-jit-macosx-x86-64/builds/28501:40
bbot2Started: 15http://buildbot.pypy.org/builders/pypy-c-app-level-linux-x86-64/builds/62501:40
bbot2Started: 15http://buildbot.pypy.org/builders/pypy-c-jit-linux-x86-32/builds/112601:40
bbot2Started: 15http://buildbot.pypy.org/builders/pypy-c-Ojit-no-jit-linux-x86-32/builds/81001:40
bbot2Started: 15http://buildbot.pypy.org/builders/pypy-c-app-level-linux-x86-32/builds/145201:40
bbot2Started: 15http://buildbot.pypy.org/builders/own-linux-x86-64/builds/72701:40
bbot2Started: 15http://buildbot.pypy.org/builders/pypy-c-jit-win-x86-32/builds/27601:40
Nick change: Gulopine -> Gulaway01:53
stakkars (~tismer@p5DDB7E68.dip.t-dialin.net) left irc: Quit: schnarch01:55
ixokai (~ixokai@rrcs-24-199-8-246.west.biz.rr.com) joined #pypy.02:08
ixokai (~ixokai@rrcs-24-199-8-246.west.biz.rr.com) left irc: Changing host02:08
ixokai (~ixokai@pdpc/supporter/student/ixokai) joined #pypy.02:08
stakkars (~tismer@p5DDB7E68.dip.t-dialin.net) joined #pypy.02:10
stakkars (~tismer@p5DDB7E68.dip.t-dialin.net) left irc: Quit: schnarch02:17
Nick change: Gulaway -> Gulopine02:22
mwhudson (~mwh@linaro/mwhudson) left irc: Ping timeout: 240 seconds02:24
bbot23Success: 15http://buildbot.pypy.org/builders/own-linux-x86-32/builds/186502:39
ixokai (~ixokai@pdpc/supporter/student/ixokai) left irc: Quit: Bye!02:43
Shanita (~John@osbk-4db15cc6.pool.mediaWays.net) joined #pypy.02:43
Kaskuka (~John@osbk-4d087e26.pool.mediaWays.net) left irc: Ping timeout: 244 seconds02:44
apanda (~apanda@c-24-7-101-62.hsd1.ca.comcast.net) joined #pypy.02:54
mwhudson (~mwh@linaro/mwhudson) joined #pypy.03:02
mwhudson (~mwh@linaro/mwhudson) left irc: Ping timeout: 258 seconds03:19
chronitis (~chronitis@79-68-126-165.dynamic.dsl.as9105.com) left irc: Quit: Lost terminal03:20
bbot24Failure: 15http://buildbot.pypy.org/builders/pypy-c-app-level-linux-x86-32/builds/145203:35
bbot24Failure: 15http://buildbot.pypy.org/builders/pypy-c-Ojit-no-jit-linux-x86-32/builds/81003:37
nedbati'm trying to understand why starting pypy-sandbox invokes the C compiler, and what can be done about it.  Anyone?03:40
bbot24Failure: 15http://buildbot.pypy.org/builders/pypy-c-app-level-linux-x86-64/builds/62503:45
bbot24Failure: 15http://buildbot.pypy.org/builders/pypy-c-jit-linux-x86-32/builds/112603:51
bbot24Failure: 15http://buildbot.pypy.org/builders/pypy-c-jit-linux-x86-64/builds/59204:08
nedbat (~nedbat@python/psf/nedbat) left irc: Ping timeout: 260 seconds04:12
mat^2 (mathias@87.55.0.198) joined #pypy.04:27
mat^2 (mathias@87.55.0.198) left irc: Ping timeout: 244 seconds04:56
bbot23Success: 15http://buildbot.pypy.org/builders/own-linux-x86-64/builds/72705:07
mwhudson (~mwh@linaro/mwhudson) joined #pypy.05:12
fijal (~fijal@197.106.33.226) joined #pypy.05:33
fijalhi05:33
bbot24Failure: 15http://buildbot.pypy.org/builders/pypy-c-jit-macosx-x86-64/builds/28505:38
fijal (~fijal@197.106.33.226) left irc: Read error: Connection reset by peer05:40
mwhudson (~mwh@linaro/mwhudson) left irc: Ping timeout: 258 seconds05:42
mattip (c1a9466d@gateway/web/freenode/ip.193.169.70.109) joined #pypy.05:42
mattiphi05:42
mitchellh (~mitchellh@c-98-245-152-34.hsd1.co.comcast.net) joined #pypy.05:45
fijal (~fijal@41.48.154.2) joined #pypy.05:45
mattipfijal: ping05:46
fijalmattip: pong05:46
mattipI tried to find out what wasn't working with arange, even found a 64 bit linux box to test it.05:46
fijaldid you write a test with floats?05:47
fijalit won't translate05:47
fijalfor one, you can't pass int-or-float to such a function05:47
mattipAhh, translate.05:47
mattipI have been writing tests only for app-level.05:47
mattipWhere is an example of a translation test?05:48
fijalhave you written a test for floats?05:49
fijallike arange(0, 0.8, 0.1)05:50
fijal?05:50
mattipI cannot look at code now, limited internet access, but I thought I had a test like arange(0.1, 3.2)05:51
mattipand also arange(3.0)05:51
mattipwhich returned an array with type float05:51
fijalyes, but arange(3.0) would not be a problem05:51
fijaleven if 3 gets truncated to int05:52
fijalsee, this is a problem with partial branch merges05:52
fijalI have a bit no idea where can I commit a failing test05:52
fijalthat's why shape & friends should go to a different branch05:52
mattipOK, I put reshape, shape onto a different branch now.05:52
fijalI'll commit to matrimath anyway05:53
mattipSo the head of matrixmath is only for arange, transpose, flat.05:53
fijalok05:53
fijalso, there is no such arange tests05:53
mattip:(05:53
fijalanyway05:54
fijalit might work by chance05:54
mattipIf you could commit some more tests I will look at it later.05:55
fijalah!05:55
fijalsorry sorry05:55
fijalarange is applevel?05:55
fijalthen it would work indeed05:55
mattipYes, it seemed to make more sense to keep it out of RPython05:55
mattipOr did I just say something stupid?05:55
mitchellh (~mitchellh@c-98-245-152-34.hsd1.co.comcast.net) left irc: Quit: Computer has gone to sleep05:56
fijalno, it probably makes sense05:56
fijalok, cool, so arange works :)05:56
mattipI would like to implement most of the things like determinant, matrixmultiply, inverse in applevel, if possible.05:56
fijalit's better than not at all05:57
fijalmatrixmath has two heads05:58
mattipKeep going, I later commited a branch name.05:58
fijalyou used --force05:58
fijal?05:58
fijalstill, matrixmath has 3 heads05:58
mattiparghh.05:59
fijal2 + mine05:59
fijalI'll sort out the mess05:59
fijalbut never ever use --force05:59
mattipThanks.05:59
fijalfor anything05:59
mattipOK, but --newbranch is OK, right?05:59
Nick change: Gulopine -> Gulaway06:00
mattipWould you prefer things in interplevel?06:00
fijal--newbranch is ok06:02
fijalinterp vs applevel - I don't know06:02
fijalit depends06:02
kenaan12fijal matrixmath 1116a56eaf2ca0 15/pypy/module/micronumpy/: oops, did not see arange is applevel. Commit a test and remove a REVIEW item06:02
kenaan12fijal matrixmath 11c6d928727402 15/pypy/module/micronumpy/REVIEW: merge (one)06:02
kenaan12fijal matrixmath 117091e3500552 15/pypy/module/micronumpy/: merge06:02
kenaan12fijal matrixmath 11bf8cd0d22563 15/pypy/module/micronumpy/interp_numarray.py: Simply kill descr_new_flatiter06:03
fijalmattip: so the head fails tests06:03
fijalthe reshape tests06:03
fijalbut that's not supposed to be there right?06:03
fijalshould I remove all mentions of reshape in this branch?06:04
mattipRight.06:04
mattipPlease.06:04
mattipThat's "Yes please"06:04
fijalok :)06:05
fijalmattip: note that descr_set_shape and descr_reshape should share most of the code06:05
mattipI also could not get numpypy.flatiter to throw the exception that numpy.flatiter does06:05
fijalwhich one?06:06
Action: mattip gotta go do something else now - will come back later.06:07
mattip (c1a9466d@gateway/web/freenode/ip.193.169.70.109) left irc: Quit: Page closed06:08
kenaan12fijal matrixmath 11e03f83291da1 15/pypy/module/micronumpy/: Remove the notion of reshape/set_shape and clean up the branch06:09
fijalAlex_Gaynor: can you review this branch now?06:09
kenaan12fijal matrixmath 11bb4b4da8518c 15/pypy/module/micronumpy/REVIEW: remove, all items are done06:11
kenaan12fijal matrixmath 11643c52f23a6d 15/pypy/module/micronumpy/interp_dtype.py: leave branch-unrelated comment06:13
kenaan12fijal matrixmath 1171542b62e8e8 15/pypy/module/micronumpy/interp_numarray.py: find_size no longer raises ValueError06:14
CIA-1303fijal 07roundup * 10#941/15000+ matrix filling instructions kill pypy-c: [resolved] closing then * 14https://bugs.pypy.org/issue94106:15
CIA-1303fijal 07roundup * 10#935/Possible typing error in micronumpy: [chatting] Hey, this should be fixed by now, can you comment? * 14https://bugs.pypy.org/issue93506:17
Alex_Gaynorfijal: in like 12 hours, yes06:17
CIA-1303fijal 07roundup * 10#919/implement multi-dimension arrays in micronumpy: [resolved] Done :) * 14https://bugs.pypy.org/issue91906:17
fijalAlex_Gaynor: ok, I can merge it06:17
CIA-1303fijal 07roundup * 10#846/TypeError: type 'basestring' is not an acceptable base class: 06:19
CIA-13[wontfix] Since the upstream patch has been applied, I think it's ok to close this as06:19
CIA-13wontfix. Anyone who feel we should support this obscure us ... * 14https://bugs.pypy.org/issue84606:19
fijalok, a bit of gardening06:21
fijalAlex_Gaynor: are django tests still failing?06:21
Alex_Gaynorfijal: no, and we have a buildbot for 1.7 now06:21
fijalcan I close the ticket then?06:21
Alex_Gaynorfijal: did you see the obscure test failure we had for a bit though?06:21
Alex_Gaynorfijal: go for it06:21
fijalyou showed me something06:22
Alex_Gaynorfijal: http://ci.django-cms.org/job/Django/database=sqlite3,python=pypy1.7/501/testReport/django.contrib.humanize.tests/HumanizeTests/test_naturaltime/06:22
CIA-1303fijal 07roundup * 10#908/Django tests failing: [resolved] Apparently fixed. * 14https://bugs.pypy.org/issue90806:22
Alex_Gaynorit failed because datetime wasn't datetime due to monkey patches :)06:22
fijalright06:22
Alex_Gaynorfix: https://code.djangoproject.com/changeset/1714406:23
mattip (c1a9466d@gateway/web/freenode/ip.193.169.70.109) joined #pypy.06:23
fijalI don't know what to do with some bugs06:23
fijalmagcius: I cleaned up the branch, I think I'll merge it06:24
fijalnice06:24
magciushuh?06:24
Alex_Gaynorfijal: here, let me take 10 seconds to review, branch is matrixmath?06:24
fijalAlex_Gaynor: yes06:25
fijalmagcius: sorry :)06:25
fijalwas mattip 06:25
Alex_Gaynorfijal: can we just remove maskna argument if we don't support it?  we haven't put it in for other missing args06:25
fijalok06:25
mattipfijal: sorry for the rudeness. I'm at work now, and not really free to chat. Anyhow I will try to finish shape/reshape within 24 hours.06:26
fijalmattip: thanks and no rush06:26
Alex_Gaynorfijal: numpy.arange(False, True, True) returns  a bool array, instead of an int one :)06:26
mattipgood luck with the merge.06:26
mattipGotta run.06:27
fijalAlex_Gaynor: numpy.arange(True) returns an int one though06:27
Circlefusion (~cf@74-138-199-109.dhcp.insightbb.com) left irc: Read error: Connection reset by peer06:27
Alex_Gaynorfijal: I'm saying it's a bug, it should return an int one06:27
mattip (c1a9466d@gateway/web/freenode/ip.193.169.70.109) left irc: Client Quit06:27
fijalbesides06:27
fijalI don't care06:27
fijalI think06:27
Alex_Gaynorwell, please fix, it's kind of easy06:27
fijalbecause it's about app vs interp level of handling the fact that bool is a subclass of int06:27
fijalcan you fix it?06:28
Alex_Gaynoralso, def setshape it still in there06:28
Alex_Gaynoryes, I can06:28
fijalcause I don't know exactly what you're after06:28
Alex_Gaynorfijal: should def setshape be removed?06:30
fijalyes06:30
Circlefusion (~cf@74-138-199-109.dhcp.insightbb.com) joined #pypy.06:34
kenaan12alex_gaynor matrixmath 114876b3c92797 15/pypy/: merged default in06:40
kenaan12alex_gaynor matrixmath 11248307654a59 15/pypy/module/micronumpy/: Some cleanup: remove parameters that's never used, rename class, delete some more setshape code, fix a cas...06:40
Alex_Gaynorfijal: ok, done06:41
fijalcool06:47
fijalcan I/you merge it?06:47
Alex_Gaynorgo for it06:48
fijaldoes anyone have an admin password for pypy-issue?06:48
asmeurer_ (~asmeurer@dhcp-baca-230.resnet.nmt.edu) left irc: Quit: asmeurer_06:50
kenaan12fijal default 113998a7e1ed9b 15/pypy/module/micronumpy/: (mattip) merge matrixmath, adds arange, transpose, flatiter06:50
kenaan12fijal matrixmath 1178795591bf65 15/: close merged branch06:50
Alex_Gaynorfijal: is there anything left before we can run that benhcmark?06:50
fijaltranslation :)06:51
Action: fijal is about to translate on tannit06:51
Alex_Gaynorjust hit the buildbot06:51
Alex_Gaynorthen we can all use it06:51
bbot2Started: 15http://buildbot.pypy.org/builders/pypy-c-jit-linux-x86-64/builds/593 [12fijal]06:51
fijaldo we have numpy.dot?06:53
Alex_Gaynorno, we have ndarray.dot though, I guess numpy.dot just calls this06:54
fijal (~fijal@41.48.154.2) left irc: Ping timeout: 260 seconds06:59
asmeurer__ (~asmeurer@dhcp-baca-230.resnet.nmt.edu) joined #pypy.07:02
Nick change: Seta00 -> reuben_07:03
Vorpal (~AnMaster@unaffiliated/anmaster) joined #pypy.07:10
fijal_ (~fijal@197.173.2.175) joined #pypy.07:11
fijal_ (~fijal@197.173.2.175) left irc: Read error: Connection reset by peer07:12
fijal (~fijal@41.51.204.9) joined #pypy.07:26
Alex_Gaynorfijal: so I ran the benchmark under CPython, using the numeric version, and it runs it .7s, while the wiki page says it runs in 30s, running the "slow" version is about the same speed as they say07:28
fijal_ (~fijal@197.105.185.169) joined #pypy.07:30
fijal (~fijal@41.51.204.9) left irc: Read error: Connection reset by peer07:30
Nick change: fijal_ -> fijal07:30
fijalgeez07:30
Alex_Gaynor?07:30
fijalinternet07:31
fijalI met project manager of free software innovation center the other day, while surfing07:31
fijalfrom a local uni07:31
Alex_Gaynorby chance, or aranged?07:31
fijalby chance07:32
fijalworld is small I guess07:33
Alex_Gaynorhehe, indeed07:33
kkris (~kris@80-123-37-238.adsl.highway.telekom.at) joined #pypy.07:36
fijalwhat is mraleph so pissed off about?07:39
Alex_Gaynorfijal: dunno, but if you ever hear me say, "don't use try/except" hit me with a hammer07:40
fijala bit far, but I'll try07:40
fijalAlex_Gaynor: how busy you're in Feb vs April?07:40
Alex_Gaynorprobably about the same, but it's a bit of a guess07:40
fijaland - do you remember the blog post from mozilla dudes abotu sweet spot07:40
fijalI wonder if I should come to states before or after pycon07:40
Alex_Gaynorthat blog post was from mozilla?07:41
Alex_Gaynorthe one comparing array of stuff in C vs JS?07:41
fijalthey were talking how doing this is not hitting the sweet spot, while doing that is hitting the sweet spot07:41
fijalfor apparently not very good reasons07:41
Alex_Gaynoreh, link?07:41
fijalyes, link :)07:42
Action: Alex_Gaynor is confused07:42
ronan (~ronan@host86-142-249-87.range86-142.btcentralplus.com) left irc: Ping timeout: 248 seconds07:44
fijalhm07:44
fijalit was even mr aleph himself07:44
Alex_Gaynorso not mozilla, you're saying07:44
fijalhttp://blog.mrale.ph/post/12396216081/the-trap-of-the-performance-sweet-spot?3d04d97007:44
Alex_Gaynoryes, I saw this post07:45
arigato (~arigo@89.204.138.33) joined #pypy.07:45
fijalarigato: hi armin07:45
arigatohi07:45
fijalAlex_Gaynor: I think they just hit some sort of VM complexity bottleneck or something07:46
fijaland they can't do like list-strategies07:46
Alex_Gaynorthey have a lot of hard coded ASM, so I'm not surprised07:46
fijalgood thing I did not start working for them ;-)07:47
Alex_GaynorPyPy lets me understand how x86 works without needing to write it, it's nice :)07:47
mwhudson (~mwh@linaro/mwhudson) joined #pypy.07:49
bbot24Failure: 15http://buildbot.pypy.org/builders/pypy-c-jit-linux-x86-64/builds/593 [12fijal]07:49
fijalpypy is cooked :)07:49
arigatofijal: uh?  re issue84607:50
arigatoI think your last comment makes no sense07:50
Action: fijal looks07:50
fijalarigato: so you think that a subclass of basestring that returns an instance of something else in __new__ should work?07:50
arigatoshould be closed as "resolved".  there was no upstream patch07:50
fijalthere was a patch in nltk07:51
Alex_Gaynorfijal: pff, we have no numpypy.sqrt07:51
arigatoah07:51
fijalAlex_Gaynor: pffff07:51
arigatoso the patch attached was not applied to pypy?07:51
fijalAlex_Gaynor: ** 0.5 would do?07:51
fijalarigato: yes, it's linked07:51
arigatofijal: ok, sorry, then I misread.07:52
Alex_Gaynorfijal: using math.sqrt, iterative solution under pypy is ~3x faster than the numeric solution under CPython07:52
fijalthat's not very good no?07:52
Alex_Gaynorfijal: uhh, why not?07:53
Alex_Gaynorfijal: FYI flatiter is missing __iter__ so you can't iterate over it07:53
fijalhow does that compare to C++?07:53
arigatofijal: do you still need an admin password for pypy-issue?07:54
fijalarigato: or someone to push/kill the last mail07:55
fijalthat got stuck07:55
fijalAlex_Gaynor: how did you run it?07:55
arigatoI can do that.  note that I see the same password for admin for all pypy-* lists07:55
fijalnot that I know it07:55
Alex_Gaynorfijal: http://paste.pocoo.org/show/513928/ I removed tons of stuff from the file07:55
fijalbut I'm fine with someone else :)07:55
Alex_Gaynorfeel fre to make sure I didn't break it07:55
fijalok, numpy.flatiter is broken07:55
fijalit should behave like a normal array07:55
fijalat least with __getitem__07:56
fijalAlex_Gaynor: feel like adding missing pieces?07:56
fijallike numpy.dot, numpy.sqrt07:56
fijalwhatever is there07:56
fijal__iter__07:56
Action: arigato would love a mailman admin option "Accept the message but trim it"07:56
Alex_Gaynornot ATM, I'm procrastinating a paper ATM :/07:56
fijalAlex_Gaynor: ok, I'll add them later maybe then07:57
Alex_Gaynorfijal: C++ iterative is .22, pypy iterative is .2607:57
Alex_Gaynorvery good IMO07:57
Alex_Gaynorthat's g++ -O307:57
fijalcool ;)07:58
fijalwhat's "iterative"?07:58
Alex_Gaynorthe version called slow in there07:58
Alex_Gaynoraka not numpy exprs07:58
fijalpfff07:58
fijalyou mean not numericTimeStep?07:58
fijaland by numpy exprs you don't mean numexpr by chance (obviously) :)07:59
fijalanyway, that's pretty awesome07:59
Alex_Gaynorfijal: right07:59
fijalI'll add missing pieces so we can run it without too many modifications08:00
fijaland write a blog post08:00
fijalsounds like a plan?08:00
Alex_Gaynorfijal: our "slow" version is 200x faster then their slow (doing 5 iterations)08:00
Alex_Gaynorfijal: sounds great08:00
fijalI suspect if we had SSE we could outperform GCC08:01
Alex_Gaynor367x faster at 10 iterations08:01
fijalwith our "fancy array operations library"08:02
fijal(not to be confused with numpy)08:03
Alex_Gaynorfijal: coincidentally named numpypy08:03
JaRoel|4d (~jaroel|4d@office.fourdigits.nl) joined #pypy.08:05
JaRoel|4d (~jaroel|4d@office.fourdigits.nl) left irc: Remote host closed the connection08:05
JaRoel|4d (~jaroel|4d@office.fourdigits.nl) joined #pypy.08:05
asmeurer__ (~asmeurer@dhcp-baca-230.resnet.nmt.edu) left irc: Quit: asmeurer__08:06
Alex_Gaynorfijal: sloccount says numpypy is worth $125k, not bad I guess :D08:06
Action: fijal wants his 125k$08:07
Alex_Gaynorfijal: I guess we owe people money for the JIT first, it's worth 2.5 million08:07
fijal:]08:07
Alex_Gaynorrlib.rsre is worth a measley 88k08:08
Alex_Gaynorsloccount is the ultimate in nonsense numbers08:08
fijalrandom module is good as well08:08
stakkars (~tismer@p5DDB7E68.dip.t-dialin.net) joined #pypy.08:09
fijalarigato: I have a suspicion that a lack of unicode-only list strategy makes django slow08:10
Alex_Gaynorfijal: should be easy to add one08:11
arigatoat some point we will have to support 8-, 16- and 32-bit unicodes, at least in py3k08:14
arigatothinking about it from this point of view makes it a bit harder (but not impossible --- just saying that's why unicode-lists were not done so far)08:15
Alex_Gaynorarigato: carl has ideas about this I believe, it should startr with refactoring unicode and strobjects to share more code though IMO08:15
arigatoAlex_Gaynor: I agree that it would be nice to end up with the equivalent of CPython's stringlib/08:18
Alex_Gaynorarigato: that's basically what RPython has right?  just a matter of cleaning up the objspace impls I think08:19
arigatonot completely08:19
mwhudson (~mwh@linaro/mwhudson) left irc: Ping timeout: 258 seconds08:19
arigatobecause we cannot so far express things like bytearrays or unicode-of-16-bits08:19
arigatoalso, stringlib/ has support for more operations than RPython's strings, as far as I know08:20
arigatoe.g. string_format.h08:21
arigatono, reading more stringlib/, it contains tons of things that are not RPython08:24
arigatoto copy it seems to require a large .py file that gets specialized08:25
Alex_Gaynorarigato: well, just some "Python string mixin" I think08:25
arigatoyes08:25
arigatobut a large one :-)08:25
arigatothen we'll end up with a good implementation of bytearray, too08:26
Alex_Gaynorarigato: well, perhaps but I estimate that it'll be .55x as much code as stringobject.py + unicoedobject.py08:26
arigatosure08:26
Alex_Gaynorless if you add in bytesarray08:26
arigatojust trying to frame the problem08:26
Alex_Gaynoryes, sure08:26
arigatoyou have to be a bit careful in order to support bytearray (as a list of chars) or 16-bit unicode (as a list of USHORT maybe?)08:27
Alex_Gaynorone thing at a time, I suppose08:28
arigatoa list of USHORT is obscure and looses the immutability property at the RPython level08:28
Alex_Gaynoryes, it requires care since bytearray is mutable08:28
canta (~canta@77-20-123-240-dynip.superkabel.de) left irc: Quit: Leaving.08:28
fijalso dot is not a ufunc even08:32
Alex_Gaynorno, it's not08:32
Alex_Gaynorsqrt should be trivial though08:32
fijalyes08:34
fijalhave you seen my overflow comment btw?08:34
Alex_Gaynorfijal: yes, but tests pass with -A so I don't know what you're talking about08:34
Alex_Gaynorfijal: FYI you can't even iterate over a flatiter ATM08:34
fijalyes, I added __iter__08:35
Alex_Gaynoryou didn't commit it08:35
fijalI totally did08:36
Alex_Gaynordid you push it?08:36
fijalI'm pushing :)08:36
fijalrelax08:36
Alex_Gaynorwell, I was confused :)08:36
kenaan12fijal default 113f56396d0ea2 15/pypy/module/micronumpy/test/test_numarray.py: newline08:36
kenaan12fijal default 11e847fa0fc602 15/pypy/module/micronumpy/: __iter__ for flatiter08:36
kenaan12fijal default 112337eca967fa 15/pypy/module/micronumpy/interp_numarray.py: those two functions are unroll_safe08:36
kenaan12fijal default 11ad86736e7b43 15/pypy/module/micronumpy/: sqrt ufunc08:36
fijaltada!08:37
fijalflatiter should convert to array without iterating as well08:37
Alex_Gaynorfijal: in CPython if you have a C type with tp_iternext does that automatically create tp_iter if you don't have one?08:38
fijalwe don't have iternext08:39
fijalwe have just a next method08:39
Alex_Gaynoryes, I'm asking about CPython C-API so I can understand the source08:40
Alex_Gaynorfijal: no tests for nan/inf/negative values :(08:40
arigato (~arigo@89.204.138.33) left irc: Ping timeout: 244 seconds08:40
fijalfor?08:41
fijalsqrt?08:41
bivab (~david@fwstups.cs.uni-duesseldorf.de) joined #pypy.08:41
Alex_Gaynoryes08:41
fijalwell, we have tests for math.sqrt no?08:41
fijalor they behave differently?08:41
Alex_Gaynorthey do, numpy.sqrt doesn't raise errors08:42
fijalmath.sqrt on RPython level does?08:43
Alex_Gaynoryes08:43
Alex_Gaynorthis is why tests ;)08:43
fijalpffff08:45
kenaan12fijal default 11931815553e4c 15/pypy/module/micronumpy/: a dot implementation08:46
fijalok ok08:47
fijal37min to translate08:51
fijalnot bad08:51
lucian (~lucian@93-97-174-114.zone5.bethere.co.uk) joined #pypy.08:54
derdon (~derdon@p5DE8B50D.dip.t-dialin.net) joined #pypy.08:54
arigato (~arigo@fwstups.cs.uni-duesseldorf.de) joined #pypy.09:05
hruske (~Gasper@188-230-156-183.dynamic.t-2.net) joined #pypy.09:12
G2P (~G2P@fw-asn1.ornis.com) joined #pypy.09:13
EnCuKou (~encukou@a91-156-243-246.elisa-laajakaista.fi) left irc: Remote host closed the connection09:17
bbot24Failure: 15http://buildbot.pypy.org/builders/pypy-c-jit-win-x86-32/builds/27609:18
antocuni (~antocuni@host200-125-dynamic.25-79-r.retail.telecomitalia.it) joined #pypy.09:28
ojii (~ojii@40-34.60-188.cust.bluewin.ch) joined #pypy.09:29
Alex_Gaynorfijal: ping09:29
fijalAlex_Gaynor: pong09:30
Alex_Gaynorfijal: you said someone had written papers about agile stuff with pypy?09:30
fijalyes09:30
Alex_Gaynordo yo by chance have a link or reference?09:30
fijalno :)09:31
fijalarmin might know?09:32
fijalit was university of limerick09:32
fijaland there were EU reports also09:32
Alex_Gaynorgoogline pypy agile foudn some paper, good enough09:32
antocuniyes, I remember that at some point someone went to a sprint to observe us09:36
antocuni(hi)09:36
arigatohi09:37
Action: fijal fights with flatiter09:39
fijalAlex_Gaynor: so it's not a subclass of array, but it is accepted everywhere in numpy09:40
Alex_Gaynorfijal: this is what happens when you design code in C and expose it to Python, you get class heirarchies that make no sense and don't match how you use it09:40
ltratt (~ltratt@87.115.168.27) joined #pypy.09:41
unbit (~unbit@93-58-162-185.ip159.fastwebnet.it) joined #pypy.09:43
kenaan12alex_gaynor default 1125f6f5031705 15/pypy/module/micronumpy/: fix for invalid inputs in numpy.sqrt09:50
bbot2Started: 15http://buildbot.pypy.org/builders/pypy-c-jit-linux-x86-64/builds/594 [12alex]09:51
Alex_Gaynorfijal: so now in theory we can run it all unmodified?09:51
fijalAlex_Gaynor: no09:51
fijalfighting09:51
stakkars (~tismer@p5DDB7E68.dip.t-dialin.net) left irc: Quit: schnarch09:52
apanda (~apanda@c-24-7-101-62.hsd1.ca.comcast.net) left irc: Remote host closed the connection09:54
fijalAlex_Gaynor: a controversial checkin coming09:56
Alex_Gaynorfijal: want me to review first?09:56
fijalcan you review in a mail client?09:56
fijalyou can always backout it09:56
Alex_Gaynorthat's fine09:56
Alex_Gaynor:)09:56
Action: fijal is busy pushing09:58
kenaan12fijal default 119c707f4a6aa4 15/pypy/module/micronumpy/: hopefully make flatiter interoperate nicely with the rest by making it a part of BaseArray hierarchy10:00
kenaan12fijal default 11f8cc091116a7 15/pypy/module/micronumpy/: merge10:00
fijalnow :)10:00
fijalAlex_Gaynor: 10:00
JaRoel|4d (~jaroel|4d@office.fourdigits.nl) left irc: Remote host closed the connection10:00
JaRoel|4d (~jaroel|4d@office.fourdigits.nl) joined #pypy.10:01
Alex_Gaynorfijal: should have more tests IMO, including one that the MRO is correct, but it looks ok IMO.10:01
fijalAlex_Gaynor: probablty10:02
fijalI doubt mro is that interesting though10:02
Alex_Gaynorfijal: I want to make sure it stays boring :)10:02
Alex_Gaynorassert flatiter.mro() == [flatiter, object]10:02
fijalwe should revisit an experiment with lazy string slices10:02
fijaland maybe even lazy list slices10:02
Alex_Gaynoris this in any way related to numpy discussion?10:03
fijalno10:04
Action: fijal translating10:07
fijalrelated to what mraleph said10:07
Alex_Gaynorfijal: well, maybe, I want to write the str concat-loop optimization for the JIT first10:09
tilgovi (~randall@couchdb/developer/tilgovi) left irc: Remote host closed the connection10:13
Alex_Gaynorfijal: optimizers that work at the levle of the source language scare me :/10:14
fijalwell10:16
fijalit depends how complex your source languae is10:16
fijalour jit is essentially an advanced RPython JIT10:16
fijaland RPython is simpler than python10:16
Alex_Gaynorno, it's not RPython, it's SpaceOps really10:17
fijalwell10:17
Alex_Gaynoroptimizer doesn't know about getitem, it knows about arraygetitem10:17
fijalit's compiled rpython10:17
Alex_Gaynorright10:17
fijalok right10:17
fijalit's lltype rpython10:17
fijalreally10:17
fijalright?10:17
Alex_Gaynorwell, not if we ever merge antocuni's work, but yes :)10:17
fijalthen we would have two10:17
fijal(in some sense)10:17
fijalootype & lltype rpython10:17
fijalin one source10:17
fijalbut yes, in principle this is the level on which it operates10:18
fijalalmost, because you have oopspecs10:18
fijalbut yes, I see your point10:19
fijalalready encoding rpython semantics in the JIT would be more work10:20
fijalAlex_Gaynor: do you have the link to original slides handy?10:20
Alex_Gaynoryes, e.g. you don't have initialization in the JIT, you simply have allocate and set fields10:20
Alex_Gaynorhttps://mkw.st/p/gdd11-berlin-v8-performance-tuning-tricks/#1810:20
Alex_Gaynorfijal: I'm not sure he gets that JIT doesn't really have special support for anything.10:21
Alex_Gaynor(except maaaybe strings)10:21
Alex_Gaynorbut even then it's not Python string, it's RPython string10:21
antocuniAlex_Gaynor: which of my branches are you talking about?10:21
Alex_Gaynorantocuni: oojit or whatever it's called10:21
antocuniah, that one10:22
Alex_Gaynorfijal: do I need to kick buildbot again now that you comitted flat stuff?10:23
fijalAlex_Gaynor: I'm almost done with the translation10:24
fijalI'll tell you where it is in /tmp on tannit10:24
derdon (~derdon@p5DE8B50D.dip.t-dialin.net) left irc: Remote host closed the connection10:28
ltratt (~ltratt@87.115.168.27) left irc: Ping timeout: 244 seconds10:29
ltratt (~ltratt@31.185.235.29) joined #pypy.10:33
Action: arigato carefully untangles the mess of jit/backend/llsupport/gc.py10:33
Alex_Gaynorarigato: I improved the situation there, very slightly a while ago10:34
kenaan12arigo op_malloc_gc 11ebc08ae0d2df 15/pypy/jit/backend/llsupport/rewrite.py: No-op.10:37
kenaan12arigo op_malloc_gc 11622108ea8dfe 15/pypy/jit/backend/llsupport/: Progress.10:37
kenaan12arigo op_malloc_gc 1105eb02916cbe 15/pypy/jit/backend/llsupport/: Clean-ups, comments, reorderings.10:37
kenaan12arigo op_malloc_gc 11160fba43cdbd 15/pypy/jit/backend/llsupport/test/test_rewrite.py: Progress.10:37
kenaan12arigo op_malloc_gc 11b5153823527e 15/: hg merge default10:37
luciani've been thinking of something. PyPy has a function JIT, right?10:38
fijallucian: not really10:38
fijalit's a normal tracing that can start at the beginning of a function10:38
fijalit's not a function jit though10:38
Alex_Gaynorarigato: is this branch designed to assist the STM stuff?10:39
arigatonot specially10:39
arigatoit's just a clean-up of the multiple backends, together with potentially a bit of performance improvement10:39
lucianfijal: ah, ok10:39
Alex_Gaynorcool10:39
lucianfijal: i was thinking perhaps AOT-ing with something like that might reduce the warm-up penalty10:40
arigatoAlex_Gaynor: the cool thing is that it should allow several NEW* operations to be concatenated into a single MALLOC_NURSERY10:40
Alex_Gaynorarigato: interesting, a peephope optimization?10:40
arigatowhich also means less write barriers10:40
Alex_Gaynordefinitely a good thing, they're huge10:40
arigatoAlex_Gaynor: yes, a "peephoPe" optimization :-)10:40
fijallucian: how would you AOT that?10:43
fijalyou don't know types10:43
fijalyou can't do tracing10:43
lucianfijal: oh, with some profiling10:43
Alex_Gaynorthe entire JIT is based on tracing and runtime feedback, I don't see how you can AOT that10:43
fijallucian: that's hard/impossible/a major mess to use10:43
lucianok10:43
fijalAlex_Gaynor: /tmp/usession-default-1802310:44
Alex_Gaynorfijal: can you make a release tarball?10:44
fijalgo there  copy this to your pypy-c and run the package.py?10:45
Alex_Gaynorfijal: his is why I like buildbot, it does stuff fo rme :)10:45
fijalyeah, it also run tests10:45
witulski (~stupsi@fwstups.cs.uni-duesseldorf.de) joined #pypy.10:47
fijal (~fijal@197.105.185.169) left irc: Read error: Connection reset by peer10:47
witulski (stupsi@fwstups.cs.uni-duesseldorf.de) left #pypy.10:47
fijal (~fijal@197.174.207.128) joined #pypy.10:50
fijalAlex_Gaynor: it's super slow10:51
fijalno idea why10:51
Alex_Gaynorfijal: yes, we need to review traces and profiles10:51
Alex_Gaynorfijal: we should wait to fix that before blogging, I want to be able to say, "PyPy is always fast" :)10:51
fijalwell10:51
fijalit might be that you can't write this function really well in vectorized version10:52
fijalwithout sacrificing perf10:52
Alex_Gaynor?10:52
fijalOMG10:52
Alex_Gaynor?10:52
fijalhave you seen how setTimeStepper is written?10:52
Alex_Gaynoryes, and?10:52
fijalgetattr?10:53
bbot24Failure: 15http://buildbot.pypy.org/builders/pypy-c-jit-linux-x86-64/builds/594 [12alex]10:53
Alex_Gaynorfijal: yeah, who cares, let's focus on real problem s;)10:53
Alex_Gaynorfirst loop looks ok, except for having too many counters10:54
fijalyuo know what's slow?10:55
fijalg.old_u = u.copy()10:55
Alex_Gaynorfijal: pff, I can't find a loop with float_mul, am I just dumb?10:55
Alex_Gaynorfijal: how much time does it take?10:55
fijal1.3s per iterations10:56
Alex_Gaynorso, what percent of total time?10:56
fijalalmost 100?10:56
fijal92%10:56
Alex_Gaynorpff, I'll fix10:56
fijalpfff10:57
fijalit can't be fast the way it is :)10:57
fijalloops look good otherwise ;-)10:57
fijalanyway, it's still possible that in-place will be faster10:58
fijal(obviously)10:58
fijalso the numpy version will not surpass the slow version10:58
fijalwhat you can do instead of copying however, would be to have 2 arrays and simply move one to another10:59
fijalI suppose GC pressure would trash all the caches otherwise11:00
fijalbut I don't know how to express it using numpy though11:02
Alex_Gaynorfijal: well, for starters I'm going to make it use C memcpy, ratehr than call __new__11:02
Alex_Gaynorfijal: http://paste.pocoo.org/show/513962/ look ok?11:02
fijalgood11:03
fijalI don't know, loading11:04
kenaan12alex_gaynor default 1156290d9adb6d 15/pypy/module/micronumpy/interp_numarray.py: use memcpy for numpy.array.copy(), rather than going through the constructor11:04
fijalyeah sounds good11:04
Alex_Gaynorfijal: I'll punch the buildbot11:04
fijalalthough if you want to be crazy, you don't have to force it11:04
Alex_Gaynorfijal: is there a reason to be crazy?11:05
fijalno idea11:05
Alex_Gaynorlet's assume no, for now11:05
Alex_Gaynorfijal: also, expression gets compiled into like 4 loops, any idea why?11:05
Topic changed on #pypy by arigato!arigo@fwstups.cs.uni-duesseldorf.de: PyPy, a fast and compliant python interpreter | IRC logs: http://www.tismer.com/pypy/irc-logs/pypy/latest.log.html | if you want to be crazy, you don't have to force it11:05
bbot2Started: 15http://buildbot.pypy.org/builders/pypy-c-jit-linux-x86-64/builds/595 [12alex]11:06
Action: arigato likes this sentence taken out of any context11:06
fijalAlex_Gaynor: investigating11:06
Alex_Gaynorfijal: ok, they all have good assembler, so maybe it's ok enough11:06
fijalI did old_u[:, :] = u[:, :] instead of copy()11:07
Alex_Gaynorthat's probably slower than memcpy though11:07
Trundle (~andy@p578bfdcf.dip0.t-ipconnect.de) joined #pypy.11:07
Trundle (~andy@p578bfdcf.dip0.t-ipconnect.de) left irc: Changing host11:07
Trundle (~andy@python/site-packages/trundle) joined #pypy.11:07
fijalprobably11:08
fijalbut it does not create new arrays11:08
fijalthat trash your caches11:08
fijalif you do that, you're fucked11:08
fijalremember allocating zeros in a loop?11:08
Alex_Gaynor?11:08
fijalif you allocate arrays in a loop it's slow11:08
Alex_Gaynorif you allocate anything in a loop it's slow compared to not allocating11:08
Action: Alex_Gaynor utilizes the advanced philosophy of: computers do nothing very quickly.11:09
fijalfor i in range(100): z = zeros((100, 100))11:09
fijalthat's slow11:09
fijalincrease range if you feel like it11:09
Alex_Gaynorerr, yes11:09
fijalso copying contents should be better in general11:10
fijaland it's only our fault if we're slower than memcpy11:10
Alex_Gaynorwell, going through __new__ is what's slow11:10
Alex_Gaynorbecause it allocates and then unboxes everything11:10
fijalit's zeros11:10
fijalso no11:10
fijalwell yes, but no11:11
kennethreitz (~kennethre@c-24-127-96-129.hsd1.va.comcast.net) left irc: Quit: Textual IRC Client: http://www.textualapp.com/11:11
bbot24Failure: 15http://buildbot.pypy.org/builders/pypy-c-jit-linux-x86-64/builds/595 [12alex]11:11
fijalit's faster than now11:11
fijalbesides, you broke it11:11
Alex_Gaynorfijal: there's no copying there thoguh, zeroes((100, 100)) should literally *just* be a malloc11:11
fijalright11:12
fijalbut if you do raw_malloc in a loop it's slow11:12
fijalbecause you trash your caches11:12
Alex_Gaynorfijal: do you understand translation error?11:12
fijalsize is not in _attrs_?11:13
Alex_GaynorNDimArray has no _attrs_11:13
fijalBaseArray, not NDimArray11:13
fijalyou're trying to read size out of BaseArray11:13
Alex_Gaynoryes, but the error is inside of NDimArray.__init__11:13
Alex_Gaynorhow can self be a BaseArray in NDimArray11:14
lizardo (~lizardo@189.2.128.130) joined #pypy.11:14
fijalare you sure it's self?11:14
Alex_Gaynorlook at the error message11:14
fijalhm11:14
fijalAlex_Gaynor: you fight with your error, I'm looking at traces!11:15
fijal:-)11:15
Alex_Gaynork11:15
fijalobscure11:15
fijallook why it's raising something11:15
chronitis (~chronitis@79-68-112-81.dynamic.dsl.as9105.com) joined #pypy.11:15
Alex_Gaynorknk11:15
Alex_GaynorA Scalar11:16
Action: fijal wants jitdriver.get_printable_location11:17
Alex_Gaynorfijal: implement it?11:18
kenaan12alex_gaynor default 111baa8d5c0e0b 15/pypy/module/micronumpy/: fix for translation and copying scalars11:18
bbot2Started: 15http://buildbot.pypy.org/builders/pypy-c-jit-linux-x86-64/builds/596 [12alex]11:18
fijaland why do we get so many forcings?11:18
Alex_Gaynorforcing in what sense?11:18
fijalso many different loops11:19
Alex_GaynorI dunno, I asked this question originally I belive ;)11:19
fijalAlex_Gaynor: hm, slow for me takes 2x time as C++ roughly11:20
fijalnot 30%11:20
fijalpyrex :/11:21
Alex_Gaynorno idea, maybe your gcc is better than mine11:21
Alex_Gaynorwhat options did you compile wiht?11:21
fijalah no11:22
fijalgood11:22
Alex_Gaynor?11:23
fijalok, I have to go to the doctor11:23
fijalwill fight with it later11:23
Alex_Gaynorheh, ok11:23
fijalI injured myself while sneezing :/11:23
arigato:-/11:23
Alex_Gaynorimpressive11:23
fijalfor what is worth, if ** is broken, this is broken as well11:24
Alex_Gaynor?11:25
fijalAlex_Gaynor: re - your mailing list post?11:25
Alex_Gaynorfijal: yes, I'm saying what's "this"11:25
Alex_Gaynorit's only broken with ints11:25
fijalah ok11:25
fijalhappy nonsense11:25
Alex_Gaynoryes11:26
fijalcan 0.5/(dx2 + dy2) force it?11:26
fijalby chance?11:26
Alex_GaynorI should hope not, but we can write a test11:27
fijalno11:27
fijaldoes not seems like it11:27
Alex_Gaynorsomething about setslice perhaps?11:27
fijalsetslice forces it11:28
fijalobviously11:28
Alex_Gaynoryes, but does it force it in parts somehow?11:28
fijalnot that I can think about11:28
fijal            # for now we just copy if setting part of an array from11:29
fijal            # part of itself. can be improved.11:29
fijalhaha!11:29
Alex_Gaynorfijal: but... it calls get_concrete() either way11:30
Alex_Gaynoralso, it shoudl copy a better way11:31
Alex_Gaynoralso, it can have more than one roto storage if it's a Call2, can't it?11:31
Alex_Gaynorbasically, I claim this code doesn't make much sense11:31
fijalnot that I wrote it...11:31
fijalbut yes, I agree11:31
kenaan12alex_gaynor default 114ef4547fa441 15/pypy/module/micronumpy/interp_numarray.py: use the internal copy, rather than reinstantiate, faster.11:33
Alex_Gaynorfijal: well, I made a small improvement11:33
fijalhttp://technicaldiscovery.blogspot.com/2011/06/speeding-up-python-numpy-cython-and.html11:34
fijalthis one is interesting to run as well11:34
Alex_Gaynorhow do we do?11:34
bbot24Failure: 15http://buildbot.pypy.org/builders/pypy-c-jit-linux-x86-64/builds/596 [12alex]11:34
fijalI didn't run it yet11:34
fijalfix the translation first11:35
Alex_Gaynorfijal: eh, what the hell?  look at the error message11:35
fijalnice11:35
fijalit means you forgot to pass some stuff to the jitdriver11:35
fijalusually11:35
Alex_Gaynornone of my commits touched somethign with a jitdriver11:36
Alex_GaynorAFAIK11:36
fijalrun test_zjit11:37
Alex_Gaynorit passes11:38
fijalwtf11:39
Alex_Gaynorfijal: pretty much11:39
fijal"Given the fact that it's so easy to develop with Python, this speed reduction is not very significant."11:40
Alex_Gaynor?11:40
fijalfrom the PerformancePython wiki11:40
fijalsince when is inlining fortran "developing in Python"...11:40
Alex_GaynorPerformancePython could charitably called CPython11:40
fijalthe article itself predates pypy11:42
Alex_Gaynorthe article predates this millenia11:42
luciancould still be renamed to PerformanceCPython11:42
lucianmaybe not even that, i followed some of the advice a long time ago and it was wrong11:43
bbot2Started: 15http://buildbot.pypy.org/builders/pypy-c-jit-linux-x86-64/builds/597 [12alex]11:44
kennethreitz (~kennethre@c-24-127-96-129.hsd1.va.comcast.net) joined #pypy.11:46
fijal (~fijal@197.174.207.128) left irc: Read error: Connection reset by peer11:52
nedbat (~nedbat@python/psf/nedbat) joined #pypy.11:57
bbot24Failure: 15http://buildbot.pypy.org/builders/pypy-c-jit-linux-x86-64/builds/597 [12alex]11:59
Alex_Gaynorpfff12:00
kenaan12alex_gaynor default 1167ac82a4bea4 15/pypy/module/micronumpy/interp_numarray.py: fix translation hopefully12:04
bbot2Started: 15http://buildbot.pypy.org/builders/pypy-c-jit-linux-x86-64/builds/598 [12alex]12:04
mattip (c1a9466d@gateway/web/freenode/ip.193.169.70.109) joined #pypy.12:06
mattiphi12:06
Alex_Gaynormattip: hey12:06
mattipI found some tim eto sneak off and read the irc log.12:06
mattips/tim eto/time to/12:06
mattipI have to say it was worth the "learning curve" and the long nights, you and fijal are great.12:07
Alex_Gaynorwe love new contributors12:07
mattipThe privilege is all mine.12:07
mattipWhat test are you trying?12:08
Alex_Gaynorhttp://www.scipy.org/PerformancePython12:08
mattipOK, I have to go but you got me motivated to finish the (re)shape mess.12:09
Alex_Gaynorcool12:09
mattip (c1a9466d@gateway/web/freenode/ip.193.169.70.109) left irc: Client Quit12:09
cheater (~cheater@178-26-58-7-dynip.superkabel.de) joined #pypy.12:18
cheaterhi12:18
cheateris there something like cprofile for pypy?12:18
Alex_Gaynorcprofile :)12:18
cheaterdoes it give useful results?12:18
Alex_Gaynormore-or-less, yes12:18
cheatercan you explain?12:18
Alex_Gaynor(although I think there are bugs on windows)12:18
cheateri'm on a free operating system.12:19
Alex_Gaynorthe results are basically accurate, total execution time is significantly increased though12:19
Alex_Gaynorthat is, the ratio of time spent is accurate12:19
cheaterwhat about inline time?12:19
Alex_Gaynorratios are accurate12:19
cheaterok, so inline time / rcalls is accurate, but inline time is not?12:20
cheaterand similarly total time / calls is, but total time isn't?12:20
cheaterwhere does this come from?12:20
Alex_Gaynorno, the % of total time is accurate, the total time is much higher than it is in a program without cprofile12:20
cheateroh12:20
cheaterthat is what you have meant12:20
timotimotimoisn't that usually the case with profiling?12:20
cheatertimotimotimo: no.12:21
Alex_Gaynoryes, but it's more extreme with pypy than CPython12:21
Alex_Gaynorcheater: well, profiling always has an overhead to some extent12:21
cheaterAlex_Gaynor: yeah, but it's not really that noticable with cProfile on CPython12:21
Alex_Gaynorexactly12:21
cheateri mean you can run it in production12:21
timotimotimoOK12:21
cheateris there no other way to do this?12:22
cheatersomething better suited for pypy?12:22
Alex_Gaynorwe have other performance tools, but nothing that gives you a time profile12:22
Alex_Gaynorhttp://lostinjit.blogspot.com/2011/11/analysing-pythons-performance-under.html12:22
mitsuhiko (~mitsuhiko@ubuntu/member/mitsuhiko) left irc: Ping timeout: 245 seconds12:24
hruske (~Gasper@188-230-156-183.dynamic.t-2.net) left irc: Ping timeout: 248 seconds12:32
cheateri have read that post, thanks Alex_Gaynor 12:32
hruske (~Gasper@188-230-156-183.dynamic.t-2.net) joined #pypy.12:45
ousado (~ousado@31-18-122-38-dynip.superkabel.de) left irc: Ping timeout: 244 seconds12:48
ousado (~ousado@31-18-122-38-dynip.superkabel.de) joined #pypy.12:50
mitsuhiko (~mitsuhiko@ubuntu/member/mitsuhiko) joined #pypy.12:54
Alex_Gaynorfijal (logs): with faster copy we appear to be about the same or slightly slower than numpy on the numeric version12:54
kenaan12alex_gaynor default 113869e30bbf49 15/pypy/module/micronumpy/test/test_numarray.py: a failing test for integer pow12:58
bbot24Failure: 15http://buildbot.pypy.org/builders/pypy-c-jit-linux-x86-64/builds/598 [12alex]13:00
nedbat (~nedbat@python/psf/nedbat) left irc: Ping timeout: 255 seconds13:03
aboudreault (~alanb@osgeo/member/aboudreault) left irc: Quit: Leaving13:07
chronitis (~chronitis@79-68-112-81.dynamic.dsl.as9105.com) left irc: Read error: Connection reset by peer13:11
DanielHolth (~dholth@stuccoproject.com) joined #pypy.13:13
lmoura (~lmoura@186.215.206.130) joined #pypy.13:17
stakkars (~tismer@f052069126.adsl.alicedsl.de) joined #pypy.13:18
nedbat (~nedbat@python/psf/nedbat) joined #pypy.13:21
fijal (~fijal@197.104.244.170) joined #pypy.13:23
fijalcheater: cprofile might not give you reasonable time for very short functions13:25
fijalfor example empty ones13:25
fijalbecause the cost of profile is constant per-call13:25
fijalbut the cast of called function changes13:25
fijalanyway, it give you some sort of approximation usually13:25
Alex_Gaynorfijal: did you see my message?13:26
fijalyeah13:26
Alex_Gaynorfijal: still not sure why the traces are weird13:26
fijalthat's not great, it's slower than slow right?13:26
Alex_Gaynoryes13:27
Alex_Gaynoralso, according to cProfile 1/4 of time is spent in dot13:27
fijalcProfile is a lie13:27
Action: fijal attacks the werid traces13:27
Alex_Gaynorthat's probably most useful13:27
cheaterfijal: do you mean in pypy or in cpython as well?13:28
Alex_Gaynorfijal: btw, why does a getarrayitem_rw require a MOV and thena a MOVSD?13:28
fijalcheater: in pypy13:28
fijalAlex_Gaynor: good question13:29
fijalAlex_Gaynor: probably loading the address?13:29
cheaterfijal: what about functions that happen over and over, but are fairly short?13:29
Alex_Gaynorfijal: http://paste.pocoo.org/show/513983/ are the instructions13:29
Alex_Gaynorfijal: I guess this is probably not the most important thing, but I guess I'm a bit confused (I'm also not good at x86 so...)13:30
cheaterfijal: i've got an analysis script based on cprofile (and maxprofile, a modified version of cprofile that measures something else than average time)13:31
cheaterfijal: maybe i could add some sort of pypy compatibility that takes that constant cost of the profiler into account13:31
fijalAlex_Gaynor: ah this is easy13:31
fijalyou need to load it to the register13:31
Alex_Gaynor"it"?13:31
fijalcheater: a bit hard13:32
fijalAlex_Gaynor: memory location13:32
Alex_Gaynorfijal: from where?13:32
cheaterfijal: why would it be hard? isn't the profiler cost *constant*?13:32
Alex_Gaynorfijal: I gotta get ready for class, I'll be around-ish13:32
fijalAlex_Gaynor: from memory13:33
fijalcheater: I guess so13:33
fijalcheater: indeed, you can try13:33
exarkuncheater: How will you know when you have succeeded?13:34
chronitis (~chronitis@79-68-112-81.dynamic.dsl.as9105.com) joined #pypy.13:35
fijalthat's a good question13:36
cheaterexarkun: light rays from clouds drifting apart, and bells playing beautiful music13:37
cheaterfijal: is the profiling cost constant across functions? would two different functions incur the same penalty?13:38
fijalat the first approximation13:38
cheaterfijal: which would mean that you could just take the cost of a nop function13:38
mitchellh (~mitchellh@c-98-245-152-34.hsd1.co.comcast.net) joined #pypy.13:39
fijalit's slightly more complex than that13:41
fijalbut well13:41
fijalI can think about ways to measure it13:41
fijalexarkun: you can see how well your metric works13:41
mitchellh (~mitchellh@c-98-245-152-34.hsd1.co.comcast.net) left irc: Client Quit13:41
fijalby measuring time it takes with/without cProfile with different call characteristics13:41
fijaland try to match your curve against measurment data13:41
cheateri have a different problem also13:42
santagada (~leonardo@201.86.242.84.dynamic.adsl.gvt.net.br) joined #pypy.13:42
cheaterin the first call to a function the jit is run on it, right?13:42
cheateris / might be13:42
cheaterwhich could skew the results for functions that only get called a few times... right?13:43
fijalwhat?13:43
cheaterwell what happens if a function gets called in pypy?13:44
unbit_ (~unbit@93-58-162-185.ip159.fastwebnet.it) joined #pypy.13:44
cheaterit gets jit compiled, right?13:44
unbit (~unbit@93-58-162-185.ip159.fastwebnet.it) left irc: Read error: Connection reset by peer13:44
Nick change: unbit_ -> unbit13:44
cheateror its cached version gets used13:44
fijalno, not necesarilly13:47
fijaljit starts somewhere, it might be the middle of the function13:47
cheateryeah but what i mean is.. while a function is being called, the jit might become engaged13:49
cheaterwhich could mean skewed results13:49
JaRoel (~jaroel|4d@office.fourdigits.nl) joined #pypy.13:51
fijalyes13:51
fijalbut you know exact conditions under which the JIT gets executed13:51
fijalmaybe13:51
Alex_Gaynorfijal: I'm sorry, can you explain why getarrayitem_raw needs 2 movs but getarrayitem_gc only needs one?13:51
aboudreault (~alanb@osgeo/member/aboudreault) joined #pypy.13:51
fijalAlex_Gaynor: it depends if the argument comes in memory location or is already in the register13:52
Alex_Gaynorfijal: well, it's the first thing in a loop, why doesn't it get put in a register?13:52
fijalbecause our register allocator is dumb13:52
fijalprobably13:52
fijalit's 32 or 64 bit?13:52
Alex_Gaynor6413:53
fijalthen no idea :)13:53
cheaterfijal: if i'm debugging a complicated app with 15 man-years of commits i do not. :(13:53
fijalregister allocator is kinda dumb13:53
fijalcheater: so constant time per function is probably a good first approx13:53
Alex_Gaynorfijal: http://dpaste.com/662728/13:53
Alex_Gaynorour register allocator is kind of stupid.13:54
JaRoel|4d (~jaroel|4d@office.fourdigits.nl) left irc: Ping timeout: 248 seconds13:54
fijalthere is a task to improve that13:54
Alex_Gaynorfijal: it's literally reading it off the stack, then putting it back there :/13:54
fijalyes13:54
Alex_GaynorI understand teh ASM, I guess I've worked on pypy too much13:54
fijalit does that13:54
fijalAlex_Gaynor: does array.__internal_repr__() sounds like a good way to display the structure?13:55
cheaterfijal: hmm i think i might want to find a better way to measure that..13:55
Alex_Gaynorfijal: is that a numpy thing, or your invention?13:55
fijalmy invention13:55
Alex_Gaynorfijal: how about numpy.__pypy__.internal_repr() ?13:55
Alex_Gaynorerr numpypy13:55
Alex_Gaynorwhat do you need it for?13:56
fijalto see about forcing13:56
Alex_Gaynorok, I prefer my name :)13:56
ramusara (~ramusara@220.156.210.236.user.e-catv.ne.jp) joined #pypy.13:57
Alex_Gaynorfijal: but yes, sounds like an ok idea, for our own stuff in the module, I'd prefer they- be unittests thou13:57
stakkarsarigato: hi!13:59
stakkarsthere is this pervert case in llinterp with14:00
stakkarsop_int_mod_ovf_zer14:01
kkris (~kris@80-123-37-238.adsl.highway.telekom.at) left irc: Quit: Leaving.14:02
stakkarsthe ovfcheck acts upon (x % y), that is after the op,14:02
Nick change: Gulaway -> Gulopine14:03
stakkarsbut the only indicator of overflow is the change from int to long14:04
stakkarsbut exactly that distinction is gone when ints are emulated by long.14:04
stakkarsquestion: should I hack the llinterp implementation,14:05
stakkarsor make a global change to r_uint ???14:06
Alex_Gaynorfijal: going to class for 2.5 hours, if you figure out what's up with the weird forcing let me know14:07
fijalAlex_Gaynor: okey14:07
bivab_ (~david@134.99.39.146) joined #pypy.14:08
arigatostakkars: hack at the llinterp implementation14:08
fijalhm14:08
stakkarsarigato: ok, thank you14:08
fijalmaybe I should put it on the signature instead?14:08
bivab (~david@fwstups.cs.uni-duesseldorf.de) left irc: Read error: No route to host14:09
kenaan12fijal default 11a21d75a378ad 15/pypy/module/micronumpy/: a debug repr for numpy objects14:09
bivab_ (~david@134.99.39.146) left irc: Ping timeout: 248 seconds14:12
Rhy0lite (dje@nat/ibm/x-qzhhueitehyidvct) joined #pypy.14:15
whitelynx (~whitelynx@63.241.75.144) joined #pypy.14:16
fijaluh?14:17
kenaan12fijal default 1186a762812948 15/pypy/module/micronumpy/interp_numarray.py: fix translation hopefully14:22
stakkars (~tismer@f052069126.adsl.alicedsl.de) left irc: Quit: schnarch14:24
ltratt (~ltratt@31.185.235.29) left irc: Quit: using sirc version 2.211+KSIRC/1.3.1214:25
arigato>>>> numpypy.divide(numpypy.array([-10]),numpypy.array([2]))14:25
arigatoarray([9223372036854775803], dtype=int64)14:25
arigatobivab and me understood why, but it takes a lot of efforts to14:26
fijalarigato: would you give a spoiler? ;-)14:26
fijalit might be the same way as a ** 2 is broken for int arrays?14:26
arigatothat's using a version from the 23 of Nov14:27
arigatothe issue is that the ArithmeticTypeMixin contains a "def div()"14:27
arigatowhich is however transformed with a @binop14:27
arigatoas a result, it contains one function "div" that is a wrapper that calls the "real div"14:28
arigatobut now it's a mixin14:28
arigatoso every concrete subclass ends up containing its own copy of "div"14:28
arigatobut only of the "div" wrapper14:28
arigatothey all end up calling the same "real div"14:28
gtaylor (~gtaylor@108-196-160-120.lightspeed.tukrga.sbcglobal.net) joined #pypy.14:28
arigatoand *now* in the branch we're working on, this no longer translates (which is nice)14:28
fijal"good"14:29
fijalfor some definition of good14:29
arigatobecause the "real div" is called with either Signed or Unsigned14:29
fijal AttributeError': 'FunctionDesc' object has no attribute 'mergeattrfamilies'14:29
fijalarigato: any clue???14:29
arigatobut in 'default' of course, the real div is just annotated with Unsigned, and the division occurs with Unsigned14:29
arigatowho fixes this issue?  I don't have any experience digging in micronumpy14:31
fijalI believe alex14:31
CIA-13031989lzhh 07roundup * 10#943/pypy 1.7 the pow operation of numpypy's array is not correct: 14:31
CIA-13[new] import numpypy as np14:31
CIA-13a = np.array([1, 2, 3])14:31
CIA-13print a**2 #the result is [0, 0, 0] under Windows * 14https://bugs.pypy.org/issue94314:31
fijalhe's the author of the metaprogramming14:31
arigato:-)14:32
fijalthis is a dupe from pypy-dev I think14:32
Action: fijal broke translation14:34
kenaan12bivab disable_merge_different_int_types 11e129a347ccdd 15/pypy/rlib/rmmap.py: (arigo, bivab) this checks does not make sense14:35
kenaan12bivab disable_merge_different_int_types 11f344e15fadc1 15/pypy/rlib/rbigint.py: fix14:35
kenaan12bivab disable_merge_different_int_types 11841843401a55 15/pypy/rlib/test/test_rarithmetic.py: remove a testcase that is now valid14:35
kenaan12bivab disable_merge_different_int_types 11f05f7014462b 15/pypy/objspace/std/setobject.py: Fix for unsigned-ness in hash computation14:35
kenaan12bivab disable_merge_different_int_types 1102debe3a5ebc 15/pypy/rlib/rbigint.py: (cfbolz, bivab): unsigned fixes for rbigint14:35
kenaan12bivab disable_merge_different_int_types 1196f224c56d33 15/pypy/rlib/: translation fix for rmd5 and rsha14:35
kenaan12bivab disable_merge_different_int_types 11663969efb61d 15/pypy/module/binascii/interp_crc32.py: translation fix for binascii14:35
kenaan12bivab disable_merge_different_int_types 1166ee3fbaa96a 15/pypy/rlib/rsocket.py: make sure calls to ntohs and htons get unsigned values14:35
kenaan12bivab disable_merge_different_int_types 11dbfc77bcb912 15/pypy/: more fixes14:35
kenaan12bivab disable_merge_different_int_types 118e235e060794 15/pypy/annotation/: (arigo, bivab) allow merging unsigned and non-negative signed values14:35
kenaan12bivab disable_merge_different_int_types 11b6177243b63d 15/pypy/interpreter/: (arigo, bivab) make sure next_instr is unsigned and handlerposition is a non-neg signed O.o14:35
kenaan12bivab disable_merge_different_int_types 1144192f720e21 15/pypy/: (arigo, bivab): fix some annotation issues and revert some changes14:36
kenaan12bivab disable_merge_different_int_types 119d67bd8ccb4a 15/pypy/rlib/rsocket.py: (arigo, bivab): make sure makeipv4addr is called with an unsigned14:36
kenaan12bivab disable_merge_different_int_types 11033150c474a1 15/: merge default14:36
stakkars (~tismer@i59F75C7D.versanet.de) joined #pypy.14:37
arigatoha ha14:38
arigatoin my attempt to look at llsupport/gc.py14:39
arigatoI found out that "has_finalizer" is always False14:39
lmoura (~lmoura@186.215.206.130) left irc: Ping timeout: 248 seconds14:39
Nick change: Gulopine -> Gulaway14:40
fijalnice14:40
lmoura (~lmoura@186.215.206.130) joined #pypy.14:41
fijalarigato: so the "slow" version in case you haven't seen it, is 30% slower than C++14:42
fijalthan non-fancy c++14:42
fijalarigato: I think alex also broke tests14:44
fijalso numpy is in a bit bad state14:45
fijalbut I can try to fix the issue if he does not show up14:45
DanielHolth (~dholth@stuccoproject.com) left irc: Remote host closed the connection14:45
DanielHolth (~dholth@stuccoproject.com) joined #pypy.14:46
kenaan12fijal default 117e26680ebb76 15/pypy/module/micronumpy/: another attempt to fix translation14:48
durin42 (~durin@173-15-14-174-Illinois.hfc.comcastbusiness.net) joined #pypy.14:49
durin42 (~durin@173-15-14-174-Illinois.hfc.comcastbusiness.net) left irc: Changing host14:49
durin42 (~durin@adium/durin42) joined #pypy.14:49
arigatofijal: ok, feel free to14:50
arigatoin the meantime, we merged default but are going to translate --withoutmod-numpy14:50
arigatoit's cool enough that our branch showed an existing bug in some random code14:51
arigatowithout it, I've a bit no clue how you (=numpy developers) would have found it, apart from gdb'ing the C sources14:51
fijalarigato: actually alex commited a test that shows that behavior14:52
fijalbecause someone complained14:52
fijalbut maybe it's not connected14:52
sven_hager (~sven@fwstups.cs.uni-duesseldorf.de) joined #pypy.14:55
fijalarigato: I wonder how much our productivity would be boosted if we had say 5min translation14:56
fijalpushing takes forever14:58
fijalI wonder why14:58
kenaan12fijal default 1117ecbcddab11 15/pypy/module/micronumpy/test/test_numarray.py: skip the failing test for now15:00
Action: arigato fixes rmmap15:01
arigatoah bah, it's in our branch15:01
fijal?15:04
jimbaker (~jbaker@c-75-71-80-146.hsd1.co.comcast.net) joined #pypy.15:06
jimbaker (~jbaker@c-75-71-80-146.hsd1.co.comcast.net) left irc: Changing host15:06
jimbaker (~jbaker@canonical/jimbaker) joined #pypy.15:06
kenaan12ctismer win64-stage1 11dc399690828c 15/pypy/rpython/llinterp.py: simple but ugly hack to make 'test_mod_ovf_zer' pass. ovfcheck(x % y) does not help when int and long are un...15:07
CIA-1303jmr 07roundup * 10#935/Possible typing error in micronumpy: 15:08
CIA-13Hard to tell; translation still fails as of a21d75a378ad but with a different15:08
CIA-13error. * 14https://bugs.pypy.org/issue93515:08
CIA-1303fijal 07roundup * 10#935/Possible typing error in micronumpy: I just broke it. 17ecbcddab11 should translate just fine * 14https://bugs.pypy.org/issue93515:11
cheaterhey guys, where is the best place to read up on compatibility of standard modules with pypy?15:12
cheatersuch as libxml2, libxslt1, mysqldb..15:13
exarkunon the compatibility wiki page15:16
cheateroh /me googles15:16
cheaterthanks, that was very helpful15:20
stakkars (~tismer@i59F75C7D.versanet.de) left irc: Quit: schnarch15:28
kenaan12bivab disable_merge_different_int_types 1161b011382aef 15/pypy/: (arigo, bivab) get rid of the last traces of ROverflowError15:38
kenaan12bivab disable_merge_different_int_types 11be6b9593a9db 15/pypy/module/rctime/interp_time.py: (arigo, bivab): cast15:38
kenaan12bivab disable_merge_different_int_types 11538d6f408379 15/pypy/jit/backend/llsupport/asmmemmgr.py: (arigo, bivab): add some r_uints here and there15:38
arigatochronitis: hi15:48
chronitishello15:49
chronitisarigato: re pull request?15:50
arigatoyes15:51
arigatoI'm sorry not to be particularly helpful, but just a general question:15:51
chronitisI assume I need to get hold of amaury to approve it (since it's all py3k)15:51
chronitisok15:51
arigatodo you want access directly to our repo, instead of doing pull requests?15:51
chronitisthe answer at some point would certainly be yes15:52
chronitisbut it would probably be worth me spending a bit more time with pypy and mercurial first lest I spectacularly break things ;)15:52
arigatook :-)  as far as I can tell, you can probably also make a branch of py3k and work there, if you prefer to maintain some separation15:53
arigatoanyway, feel free to ask at any point15:53
chronitisok, thanks15:53
stakkars (~stakkars@82.113.119.74) joined #pypy.15:54
chronitisI'll think about it15:54
cheater (~cheater@178-26-58-7-dynip.superkabel.de) left irc: Quit: leaving15:54
asmeurer_ (~asmeurer@dhcp-baca-230.resnet.nmt.edu) joined #pypy.16:00
kenaan12arigo default 113f3caf8b978b 15/pypy/jit/backend/llsupport/gc.py: Mallocs of objects with a __del__ are handled differently anyway. This was untested code, and pointless.16:00
CIA-1303arigo 07roundup * 10#943/pypy 1.7 the pow operation of numpypy's array is not correct: 16:05
CIA-13[chatting] Another bug, maybe related:16:05
CIA-13>>>> numpypy.divide(numpypy.array([-10]),numpypy.array([2]))16:05
CIA-13array([9223372036854775803], dtype=int64)16:05
CIA-13... * 14https://bugs.pypy.org/issue94316:05
mattip (~chatzilla@bzq-79-177-116-180.red.bezeqint.net) joined #pypy.16:09
mattiphi16:09
arigatohi16:09
mattipexciting times. I looked at the code I commited this morning and couldn't recognize it :)16:10
arigato:-)16:13
mattip (~chatzilla@bzq-79-177-116-180.red.bezeqint.net) left irc: Client Quit16:13
fijalmattip: we did a bit of cleanup :)16:13
stakkars (~stakkars@82.113.119.74) left irc: Quit: Colloquy for iPhone - http://colloquy.mobi16:15
dmalcolm (~david@c-24-61-12-82.hsd1.ma.comcast.net) joined #pypy.16:17
andrewfr (~andrew@modemcable211.101-70-69.static.videotron.ca) joined #pypy.16:18
JaRoel (~jaroel|4d@office.fourdigits.nl) left irc: Remote host closed the connection16:22
sven_hager (~sven@fwstups.cs.uni-duesseldorf.de) left irc: Quit: Leaving16:22
andrewfrhi, I noticed on the blog work on STM16:23
mattip (~chatzilla@bzq-79-177-116-180.red.bezeqint.net) joined #pypy.16:23
mattipfijal: sorry, internet. Yes it's all good. I just wanted to ask if I should keep plugging away at reshape16:24
mattipor if priorities have changed16:25
andrewfrI am looking at the STM branch - is there a nightly build for this?16:25
andrewfrshould I ask about STM on the mailing list (I guess it is supper time in Western Europe right now?)16:26
kenaan12bivab disable_merge_different_int_types 1120e94d09f38f 15/pypy/jit/metainterp/optimizeopt/fficall.py: (arigo, bivab): Ah, read field as a signed value16:32
arigatoandrewfr: hi16:33
andrewfrhi there16:33
andrewfrI am looking at the bit bucket right now16:33
arigatostm is not ready so far, but there is the branch "stm"16:33
arigatoit can only translate simple RPython programs, not the whole pypy16:33
andrewfryes my main problem is I don't have computers large enough to compile pypy16:33
andrewfrwell I don't mind learning and using RPython16:34
andrewfri would like to dive into STM sooner than latter16:34
andrewfrright now I am starting to learn Haskell16:34
arigatoah16:34
arigatoI see16:34
andrewfrand for my Stackless Python stuff, I have a partial implementation of join patterns16:34
andrewfrand I have a fairly short implementation of the Santa Claus problem16:35
andrewfrit is actually trivially simple 16:35
andrewfrthere is a Haskell version that uses STM16:35
andrewfri am wondering what it would look like in STM with PyPy16:35
`fox` (~fox@95.232.111.141) joined #pypy.16:35
arigato:-)16:36
andrewfrby next week I am hoping to understand enough Haskell to follow the solution and translate it16:36
Alex_Gaynorarigato: pow and div are for different reasons, it sounds like div only affects translated versions, but pow is broken untranslated16:36
arigatoAlex_Gaynor: ah, ok16:36
arigatosorry to put it in the same bug report, then :-)16:37
andrewfralso Armin you talk about composability16:37
arigatoandrewfr: have a look in the stm branch at pypy/translator/stm/test/test_ztranslated for how a simple program can be written in RPython16:38
andrewfrhave you read anything on Concurrent ML or Transactional Events16:38
Trundle (~andy@python/site-packages/trundle) left irc: Remote host closed the connection16:38
arigatoor .../targetdemo.py16:38
andrewfrI was looking at targetdemo16:38
andrewfrand I will look at test_ztranslated16:38
andrewfri'll also look for the original STM papers you are referencing16:39
arigatook.  the only strange thing is the call to rstm.transaction_boundary()16:39
arigatoit marks the end of a transaction and the start of the next one16:39
andrewfrdoes this mark the beginning/end of the transaction?16:39
arigatoyes16:39
stakkars (~tismer@p5DDB7E68.dip.t-dialin.net) joined #pypy.16:39
andrewfrI will assume as per STM, no I/O operations16:39
arigatoit is what would be written in the interpreter main loop between two opcodes, if targetdemo was a complete interpreter16:39
andrewfreverything has to be rollbacked if possible16:40
arigatono, I/O is fine: it changes the mode of the transaction to be "inevitable", i.e. "cannot be rollbacked"16:40
Alex_Gaynorfijal: any luck with why things are forced?16:40
arigatoactually so far a lot of stuff will change the transaction to "inevitable", probably more than we want16:41
andrewfrI was under the impression, at least in Haskell's STM, I/O could not be a part of a transaction - let me re-read this16:41
arigatobut the result is correct, just not parallelized enough (of course only one transaction can be inevitable at a time)16:41
andrewfrLet me think about that scenario16:42
arigatoyes, you need "inevitable transactions", as they are called, which is considered an optional part in STM libraries16:42
arigatobut that's really essential in the context of pypy16:42
andrewfryes one transaction has to be inevitable or viable but the problems occur if an aborted transaction cannot be rollback back then that violates various ACID properties16:43
andrewfragain I just starting to dive deeper into this stuff myself16:43
andrewfrit will help when I actually get my hands dirty and write stuff16:44
andrewfrin my own stuff, I have transactions of a sort16:44
andrewfrI can't rollback a send on a channel :-)16:45
andrewfrthe event occurred....16:45
andrewfranother question Armin, you have STM but you haven't removed the GIL yet?16:45
arigatoI have STM for simple RPython programs, I haven't tried to apply it to the whole of PyPy yet16:47
arigatothese simple RPython programs don't have a GIL16:47
andrewfrwell for now RPython is good enough16:47
andrewfrbaby steps are good16:47
andrewfragain, for my join pattern stuff, the authors of "Scalable Join Patterns" referred me to Concurrent ML and transactional events16:48
andrewfrthey have a very interesting take on composability16:49
andrewfressentially they cleave the synchronisation operation from the description 16:49
andrewfrat a more practical level, synchronisation mechanisms are like decorators16:50
andrewfragain, to implement some of that stuff, they get into transactions16:50
andrewfrI want to write to model this with stackless.py16:51
andrewfrlike the STM stuff, I need to bone up on ML16:51
andrewfranyhow Armin thanks for your explanation, if I can't get the STM branch built, would it be possible for me to get a pre-built?16:53
arigatoyou have misunderstood me16:53
arigatothere is no build of the STM branch yet16:54
andrewfroh sorry for the misunderstanding16:54
andrewfrlet me re-read what you wrote. Wait a moment16:54
andrewfrokay16:55
JaRoel|4d (~jaroel|4d@sink.jaroel.nl) joined #pypy.16:59
kenaan12alex_gaynor default 1174b0db67fd56 15/pypy/module/micronumpy/: correct the dtype of scalar arrays17:00
kenaan12alex_gaynor default 116f6b258f307e 15/pypy/module/micronumpy/: fix for pow on ints, and translated binops17:00
kenaan12arigo stm 11bdf32e433c10 15/pypy/translator/stm/test/test_rstm.py: Fix the test.17:00
kenaan12arigo stm 111df7fe908221 15/pypy/translator/stm/src_stm/et.c: Fix.17:00
CIA-1303agaynor 07roundup * 10#943/pypy 1.7 the pow operation of numpypy's array is not correct: This should be fixed by 6f6b258f307e * 14https://bugs.pypy.org/issue94317:01
arigatoAlex_Gaynor: ...and @unaryop, and @raw_binop?17:01
Alex_Gaynorarigato: only if someone has a test case17:02
arigatoAlex_Gaynor: well it's trivial to come up with some17:02
G2P (~G2P@fw-asn1.ornis.com) left irc: Quit: Leaving.17:02
Alex_Gaynorok :) so let's17:02
arigatoarray([-2]) < array([2])17:03
Alex_Gaynorreturns [True] for me17:03
arigatoah17:03
arigatosign(array([-2]))?17:03
Alex_Gaynor[-1] :)17:04
arigatoabs(array([-2]))?17:04
Alex_Gaynor[2]17:05
bbot2Started: 15http://buildbot.pypy.org/builders/pypy-c-jit-linux-x86-64/builds/599 [12alex]17:07
`fox` (~fox@95.232.111.141) left irc: Ping timeout: 258 seconds17:07
arigatoAlex_Gaynor: then I give up, and I'll rely on it not translating with the branch to say "it's not RPython, you have to fix it anyway" :-)17:09
Alex_Gaynorarigato: deal :)17:09
fijalAlex_Gaynor: no, went for a walk17:15
fijalAlex_Gaynor: but the simpler example does the same17:16
Alex_Gaynorfijal: what do you mean?17:16
mwhudson (~mwh@linaro/mwhudson) joined #pypy.17:17
kenaan12arigo stm 111b1707eedf87 15/: hg merge default17:18
fijalAlex_Gaynor: nonsense :/17:21
Alex_Gaynorwhat?17:21
fijalthe fact that it compiles a couple of loops instead of one17:21
Alex_Gaynorright, do we have an idea of why, or a failing test?17:22
fijalno17:22
andrewfrhi Armin, I am a little confused.17:22
andrewfrwhat exactly do I need to get in order to play with RPython and STM, I downloaded pypy but I do not see translator/stm17:23
fijal'Call2(multiply, Call2(add, Call2(multiply, Call2(add, Slice(Array), Slice(Array)), Scalar), Call2(multiply, Call2(add, Slice(Array), Slice(Array)), Scalar)), Scalar)'17:23
fijalAlex_Gaynor: this is what the debug_repr says, sounds like it should compile one loop no?17:24
Alex_Gaynorfijal: yes, so the bug is in the evaluation, somehow17:24
Action: fijal writes test_zjit17:24
timotimotimoandrewfr: you have to check out the "stm" branch with hg17:25
andrewfrwhat is the URL for the STM branch?17:26
Alex_Gaynorfijal: pff, why do the tests have full ops for preamble as well in them?  I can't tell what's important and what isn't now17:26
arigatoandrewfr: "hg up stm"17:26
andrewfrthanks17:26
chronitis (~chronitis@79-68-112-81.dynamic.dsl.as9105.com) left irc: Quit: Lost terminal17:26
fijalAlex_Gaynor: ?17:27
fijalAlex_Gaynor: you mean check_loops?17:27
arigatoAlex_Gaynor: I can explain that17:27
Alex_Gaynorfijal: yes17:27
Alex_Gaynorarigato: ok17:27
andrewfrgreat! Thanks!17:27
arigatoAlex_Gaynor: it's done as part of the incoming jit-targets branch, where the existing check_loops() doesn't work any more17:27
arigatobut I know it's annoying17:28
arigatoas a result we have more recently re-added some kind of check_loops17:28
arigatoit would fail if there is more than one loop, though17:28
Alex_Gaynorit makes the tests very fragile IMO, numpy in particular has preambles that do random stuff and change all the time, but loops are stillv ery good17:28
fijalarigato: can you explain a bit?17:28
arigatoyes17:28
fijalabout jit-targets17:28
arigatowell this was done in its own branch, and merged to default17:29
lucian (~lucian@93-97-174-114.zone5.bethere.co.uk) left irc: Read error: Connection reset by peer17:30
andrewfr (andrew@modemcable211.101-70-69.static.videotron.ca) left #pypy.17:31
arigatohum, check_simple_loop() was re-added only on branch jit-targets (7b57bed3d875)17:32
Action: arigato adds it manually to 'default' too17:34
timotimotimo (~timo@gallifrey.wybt.net) left irc: Ping timeout: 258 seconds17:35
arigatofijal: basically jit-targets adds "rop.LABEL", and changes rop.JUMP to target not a loop but such a label17:36
fijalhow is that related to removal of check_loops17:36
arigatoit is better because it allows us to generate longer "loops" (that are now just called "traces") containing both preamble and loop17:37
arigatoso check_loops has no obvious correspondance in the new world17:37
arigatothe re-added check_simple_loop() checks only the tail of the main loop, after the LABEL in the middle17:38
fijaluh17:39
fijalAlex_Gaynor: ok, got it17:39
fijalarigato: what's the gain?17:39
fijalAlex_Gaynor: it's sliceloop that calls setitem that calls get_concrete17:40
fijalit's amazing it works at all :)17:40
arigatofijal: avoids a horribly long sequence of MOVs in the JUMP17:41
arigatoand cleans up some concepts too17:41
fijalhm is it17:41
arigatoat least I think the x86 backend can be cleaned up17:41
fijalarigato: does this require a better regalloc btw?17:41
arigatono17:41
fijalso there are no joint points?17:46
Niedar (~nnscript@ip68-231-248-182.hr.hr.cox.net) left irc: Quit: ( www.nnscript.com :: NoNameScript 4.22 :: www.esnation.com )17:46
timotimotimo (~timo@gallifrey.wybt.net) joined #pypy.17:46
fijalarigato: I hope you understand you completely broke test_zjit in numpy?17:47
fijalI mean as of now it does not represent anything17:47
arigatoyes17:47
arigatowait a sec17:47
Action: fijal waits a sec17:48
fijalok17:48
fijalit would actually be better if you skipped all those tests and left the counters as they were17:48
fijalcause they were not there by accident17:48
apanda (~apanda@dhcp-44-187.EECS.Berkeley.EDU) joined #pypy.17:49
kenaan12hakanardo default 114083ffe1dd1e 15/pypy/jit/metainterp/: A check function that allows counting the ops in the peeled loop only. It only works in the simplest case when ...17:49
kenaan12arigo default 11d0d82346acf2 15/pypy/jit/metainterp/history.py: Fix.17:49
voidspace (~voidspace@python/psf/voidspace) left irc: Quit: Uhm... gotta go17:50
fijaleh17:50
fijalAlex_Gaynor: ok so those are test_zjit tests that I were too lazy to port17:50
fijalI guess we're bitten now :/17:51
fijalmaybe I'll learn one day17:51
Action: fijal off for a beer17:51
fzzzy (~donovan@76-198-130-19.lightspeed.mtvwca.sbcglobal.net) joined #pypy.17:51
kenaan12ctismer win64-stage1 11d9abe6cd48d4 15/pypy/: blaming myself about bad ideas. is_valid_int should never raise an exception, basta!17:51
bbot2Started: 15http://buildbot.pypy.org/builders/own-win-x86-64/builds/71 [12ctismer, win64-stage1]17:54
kenaan12arigo default 1126e7aaccc675 15/pypy/module/micronumpy/test/test_zjit.py: Revert most of f47785c0df50, using check_simple_loop() instead.17:55
arigatoAlex_Gaynor: done.  there is only one test that still uses check_resops() now, and an XXX comment explaining why17:55
arigatofor future reference: look at "hg diff -c d9ef0a8f3fa2"18:00
arigatoand figure out if it makes more sense for a test to use check_resops() like now, or check_simple_loop() using the old parameters18:01
arigatooups, I missed hakanardo's checkin18:03
bbot24Failure: 15http://buildbot.pypy.org/builders/pypy-c-jit-linux-x86-64/builds/599 [12alex]18:04
arigatouh!  sorry nonsense18:04
arigatothat's a checkin I did, as a "hg transplant"18:04
Action: arigato -> dinner18:09
arigato (~arigo@fwstups.cs.uni-duesseldorf.de) left irc: Quit: See you18:09
stakkarsconfusing :-)18:09
apanda (~apanda@dhcp-44-187.EECS.Berkeley.EDU) left irc: Remote host closed the connection18:10
amaury__ (~amaury_@46-127-23-192.dynamic.hispeed.ch) joined #pypy.18:18
apanda (~apanda@dhcp-44-187.EECS.Berkeley.EDU) joined #pypy.18:24
CIA-1303jmr 07roundup * 10#935/Possible typing error in micronumpy: [resolved] Confirmed. * 14https://bugs.pypy.org/issue93518:27
Guest_ (~textual@63-228-127-70.dia.static.qwest.net) joined #pypy.18:29
DasIch (~dasich@p3E990818.dip.t-dialin.net) joined #pypy.18:35
Nick change: Guest_ -> rekamso18:38
mwhudson (~mwh@linaro/mwhudson) left irc: Ping timeout: 258 seconds18:40
fijal (~fijal@197.104.244.170) left irc: Read error: Connection reset by peer18:47
kenaan12ctismer win64-stage1 1102c5ddb49a48 15/pypy/objspace/std/test/test_intobject.py: fixed test_intobject.18:50
schmichael (~schmichae@gir.lofiart.com) joined #pypy.19:05
fzzzy (~donovan@76-198-130-19.lightspeed.mtvwca.sbcglobal.net) left irc: Quit: fzzzy19:06
mwhudson (~mwh@linaro/mwhudson) joined #pypy.19:07
apanda (~apanda@dhcp-44-187.EECS.Berkeley.EDU) left irc: Remote host closed the connection19:08
bogner (~bogner@2600:3c03::f03c:91ff:fedf:7ef4) joined #pypy.19:12
schmichael (schmichae@gir.lofiart.com) left #pypy.19:13
Sho_ (~EHS1@dslb-094-222-204-119.pools.arcor-ip.net) joined #pypy.19:15
Sho_ (~EHS1@dslb-094-222-204-119.pools.arcor-ip.net) left irc: Changing host19:15
Sho_ (~EHS1@kde/hein) joined #pypy.19:15
Sho_ (~EHS1@kde/hein) left irc: Quit: Stop leaking memory like it's going out of fashion.19:26
Sho_ (~EHS1@kde/hein) joined #pypy.19:26
fzzzy (~donovan@nat/mozilla/x-duavnjidvhnwbujf) joined #pypy.19:28
tilgovi (~randall@couchdb/developer/tilgovi) joined #pypy.19:30
jnoller (~jnoller@vpn.nasuni.com) joined #pypy.19:34
nedbat (~nedbat@python/psf/nedbat) left irc: Quit: Leaving19:36
nedbat (~nedbat@python/psf/nedbat) joined #pypy.19:40
kenaan12hakanardo jit-targets 1191088be9f99a 15/pypy/jit/metainterp/optimizeopt/test/test_util.py: fix tests19:41
kenaan12hakanardo jit-targets 11c5de3f7beba8 15/pypy/jit/metainterp/optimizeopt/: aliasing vrituals are pointless except that it generates a crash19:41
Rhy0lite (dje@nat/ibm/x-qzhhueitehyidvct) left irc: Quit: Leaving19:41
Tobu (~Tobu@unaffiliated/tobu) joined #pypy.19:42
tlynn (~tlynn@cpc6-cmbg14-2-0-cust121.5-4.cable.virginmedia.com) joined #pypy.19:47
etrepum (~bob@accessnat4.mochimedia.net) joined #pypy.19:47
antocuni (~antocuni@host200-125-dynamic.25-79-r.retail.telecomitalia.it) left irc: Ping timeout: 252 seconds19:47
hruske (~Gasper@188-230-156-183.dynamic.t-2.net) left irc: Ping timeout: 248 seconds19:50
mattip (~chatzilla@bzq-79-177-116-180.red.bezeqint.net) left irc: Quit: ChatZilla 0.9.87 [Firefox 8.0/20111115183158]19:52
DasIch (~dasich@p3E990818.dip.t-dialin.net) left irc: Ping timeout: 244 seconds19:55
santagada (~leonardo@201.86.242.84.dynamic.adsl.gvt.net.br) left irc: Quit: Ex-Chat19:56
apanda (~apanda@dhcp-44-187.EECS.Berkeley.EDU) joined #pypy.20:05
DasIch (~dasich@p3E991185.dip.t-dialin.net) joined #pypy.20:07
nedbat (~nedbat@python/psf/nedbat) left irc: Quit: Leaving20:09
k_bx (~k_bx@94.244.19.62) joined #pypy.20:17
Justin (~justin@irssi.datacomm.albany.edu) joined #pypy.20:23
cwillu_at_work (~cwillu@cwillu-1-pt.tunnel.tserv13.ash1.ipv6.he.net) joined #pypy.20:23
bbot24Failure: 15http://buildbot.pypy.org/builders/own-win-x86-64/builds/71 [12ctismer, win64-stage1]20:27
lucian (~lucian@cpc1-newc15-2-0-cust84.gate.cable.virginmedia.com) joined #pypy.20:33
fzzzy (~donovan@nat/mozilla/x-duavnjidvhnwbujf) left irc: Read error: Connection reset by peer20:39
fzzzy (~donovan@nat/mozilla/x-thcrmcbkvwelbvfn) joined #pypy.20:39
apanda (~apanda@dhcp-44-187.EECS.Berkeley.EDU) left irc: Remote host closed the connection20:40
ojii (~ojii@40-34.60-188.cust.bluewin.ch) left irc: Quit: Leaving20:44
fprimex (~fprimex@fl-67-233-144-228.dhcp.embarqhsd.net) joined #pypy.20:46
kenaan12mattip matrixmath-reshape 11b87bb33414be 15/pypy/module/micronumpy/: fix shape,reshape. Ready for review20:47
Tobu (~Tobu@unaffiliated/tobu) left irc: Remote host closed the connection20:48
lizardo (~lizardo@189.2.128.130) left irc: Quit: Leaving20:48
apanda (~apanda@airbears-136-152-21-255.AirBears.Berkeley.EDU) joined #pypy.20:48
chronitis (~chronitis@79-68-112-81.dynamic.dsl.as9105.com) joined #pypy.20:51
jnoller (~jnoller@vpn.nasuni.com) left irc: 20:51
apanda (~apanda@airbears-136-152-21-255.AirBears.Berkeley.EDU) left irc: Remote host closed the connection20:53
kkris (~kris@80-123-36-225.adsl.highway.telekom.at) joined #pypy.20:55
nedbat (~nedbat@python/psf/nedbat) joined #pypy.20:58
nedbat (~nedbat@python/psf/nedbat) left irc: Disconnected by services21:02
nedbat_ (~nedbat@python/psf/nedbat) joined #pypy.21:03
`fox` (~fox@95.232.111.141) joined #pypy.21:06
`fox` (~fox@95.232.111.141) left irc: Read error: Connection reset by peer21:17
mtigas (~Adium@users.spokesman.com) joined #pypy.21:19
apanda (~apanda@2607:f140:400:1020:c9de:eb22:b106:3eef) joined #pypy.21:20
mtigas (Adium@users.spokesman.com) left #pypy.21:21
mtigas (~Adium@users.spokesman.com) joined #pypy.21:21
nedbat (~nedbat@python/psf/nedbat) joined #pypy.21:27
nedbat_ (~nedbat@python/psf/nedbat) left irc: Ping timeout: 255 seconds21:30
apanda (~apanda@2607:f140:400:1020:c9de:eb22:b106:3eef) left irc: Remote host closed the connection21:31
Arach (~arach@2.95.163.200) left irc: Ping timeout: 240 seconds21:34
`fox` (~fox@95.232.111.141) joined #pypy.21:35
Arach (~arach@2.95.163.200) joined #pypy.21:35
kkris (~kris@80-123-36-225.adsl.highway.telekom.at) left irc: Quit: Leaving.21:40
`fox` (~fox@95.232.111.141) left irc: Read error: Connection reset by peer21:45
mtigas (~Adium@users.spokesman.com) left irc: Quit: Leaving.21:45
`fox` (~fox@95.232.111.141) joined #pypy.21:46
Dulak (~michael@unaffiliated/dulak) joined #pypy.21:50
hruske (~Gasper@188-230-156-183.dynamic.t-2.net) joined #pypy.21:53
Sho_ (~EHS1@kde/hein) left irc: Quit: Stop leaking memory like it's going out of fashion.22:08
Sho_ (~EHS1@kde/hein) joined #pypy.22:08
`fox` (~fox@95.232.111.141) left irc: Ping timeout: 260 seconds22:09
DanielHolth (~dholth@stuccoproject.com) left irc: Ping timeout: 244 seconds22:09
apanda (~apanda@dhcp-44-187.EECS.Berkeley.EDU) joined #pypy.22:21
EnCuKou (~encukou@a91-156-243-246.elisa-laajakaista.fi) joined #pypy.22:23
DanielHolth (~dholth@ip98-180-34-112.ga.at.cox.net) joined #pypy.22:23
verte (~verte@python/site-packages/verte) joined #pypy.22:27
lucian (~lucian@cpc1-newc15-2-0-cust84.gate.cable.virginmedia.com) left irc: Ping timeout: 248 seconds22:39
bbot2Started: 15http://buildbot.pypy.org/builders/jit-benchmark-linux-x86-64/builds/15322:39
bbot2Started: 15http://buildbot.pypy.org/builders/own-macosx-x86-32/builds/72722:39
bbot2Started: 15http://buildbot.pypy.org/builders/jit-benchmark-linux-x86-32/builds/95822:39
ramusara (~ramusara@220.156.210.236.user.e-catv.ne.jp) left irc: Quit: Leaving...22:41
bbot24Failure: 15http://buildbot.pypy.org/builders/jit-benchmark-linux-x86-32/builds/95822:46
JaRoel|4d (~jaroel|4d@sink.jaroel.nl) left irc: Remote host closed the connection22:50
hruske (~Gasper@188-230-156-183.dynamic.t-2.net) left irc: Ping timeout: 252 seconds22:57
mitchellh (~mitchellh@12.130.118.8) joined #pypy.22:58
stakkars_ (~tismer@p5DDB7E68.dip.t-dialin.net) joined #pypy.23:00
stakkars (~tismer@p5DDB7E68.dip.t-dialin.net) left irc: Read error: Connection reset by peer23:00
Nick change: stakkars_ -> stakkars23:00
fprimex (~fprimex@fl-67-233-144-228.dhcp.embarqhsd.net) left irc: Quit: later!23:01
rekamso (~textual@63-228-127-70.dia.static.qwest.net) left irc: Ping timeout: 244 seconds23:03
k_bx (~k_bx@94.244.19.62) left irc: Ping timeout: 248 seconds23:18
rekamso (~textual@63-228-127-70.dia.static.qwest.net) joined #pypy.23:19
whitelynx (~whitelynx@63.241.75.144) left irc: Quit: Ex-Chat23:25
voidspace (~voidspace@python/psf/voidspace) joined #pypy.23:32
apanda (~apanda@dhcp-44-187.EECS.Berkeley.EDU) left irc: Read error: Connection reset by peer23:33
aurojit (~apanda@dhcp-44-187.EECS.Berkeley.EDU) joined #pypy.23:33
EnCuKou (~encukou@a91-156-243-246.elisa-laajakaista.fi) left irc: Quit: MizĂ­m.23:46
nedbat (~nedbat@python/psf/nedbat) left irc: Ping timeout: 255 seconds23:51
voidspace_ (~voidspace@python/psf/voidspace) joined #pypy.23:54
voidspace (~voidspace@python/psf/voidspace) left irc: Read error: Connection reset by peer23:54
Nick change: voidspace_ -> voidspace23:54
chronitis (~chronitis@79-68-112-81.dynamic.dsl.as9105.com) left irc: Ping timeout: 244 seconds23:55
chronitis (~chronitis@88-104-118-201.dynamic.dsl.as9105.com) joined #pypy.23:57
--- Tue Nov 29 201100:00

Generated by irclog2html.py 2.9.2 by Marius Gedminas - find it at mg.pov.lt!