#pypy IRC log for Tuesday, 2011-12-27

voidspace (~voidspace@python/psf/voidspace) left irc: Quit: Uhm... gotta go00:01
espes (espes@2001:470:1f05:ca1::) left irc: Read error: Connection timed out00:01
espes (espes@2001:470:1f05:ca1::) joined #pypy.00:01
nedbat (~nedbat@python/psf/nedbat) joined #pypy.00:12
`fox` (~fox@host2-105-dynamic.54-82-r.retail.telecomitalia.it) left irc: Ping timeout: 248 seconds00:15
antocuni (~antocuni@host117-71-dynamic.58-82-r.retail.telecomitalia.it) left irc: Ping timeout: 252 seconds00:35
nickik (~nick@2001:67c:20a1:1018:226:9eff:feef:8369) left irc: Quit: Verlassend00:36
overminder (~overminde@wf118-245.ust.hk) joined #pypy.00:57
nedbat (~nedbat@python/psf/nedbat) left irc: Ping timeout: 276 seconds00:57
bbot23Success: 15http://buildbot.pypy.org/builders/jit-benchmark-linux-x86-64-2/builds/1300:58
etrepum (~bob@75-101-96-144.dsl.static.sonic.net) joined #pypy.00:58
bbot23Success: 15http://buildbot.pypy.org/builders/jit-benchmark-linux-x86-64/builds/18501:03
overminder (~overminde@wf118-245.ust.hk) left irc: Read error: Connection reset by peer01:06
nedbat (~nedbat@python/psf/nedbat) joined #pypy.01:14
espes (espes@2001:470:1f05:ca1::) left irc: Read error: Connection timed out01:14
espes (espes@2001:470:1f05:ca1::) joined #pypy.01:16
bbot24Failure: 15http://buildbot.pypy.org/builders/own-macosx-x86-32/builds/75701:18
bbot2Started: 15http://buildbot.pypy.org/builders/own-linux-x86-32/builds/190201:30
bbot2Started: 15http://buildbot.pypy.org/builders/pypy-c-jit-linux-x86-64/builds/65301:30
bbot2Started: 15http://buildbot.pypy.org/builders/pypy-c-jit-macosx-x86-64/builds/31801:30
bbot2Started: 15http://buildbot.pypy.org/builders/pypy-c-jit-win-x86-32/builds/30801:30
bbot2Started: 15http://buildbot.pypy.org/builders/pypy-c-app-level-linux-x86-64/builds/65701:30
bbot2Started: 15http://buildbot.pypy.org/builders/pypy-c-jit-linux-x86-32/builds/117301:30
bbot2Started: 15http://buildbot.pypy.org/builders/pypy-c-Ojit-no-jit-linux-x86-32/builds/84001:30
bbot2Started: 15http://buildbot.pypy.org/builders/pypy-c-app-level-linux-x86-32/builds/149201:30
bbot2Started: 15http://buildbot.pypy.org/builders/own-linux-x86-64/builds/76701:30
derdon (~derdon@pD9E1DB40.dip.t-dialin.net) left irc: Remote host closed the connection01:33
Garen (garen@69.76.18.3) joined #pypy.01:58
bbot23Success: 15http://buildbot.pypy.org/builders/jit-benchmark-linux-x86-32/builds/99402:45
Alex_Gaynor (u1246@gateway/web/irccloud.com/x-wcjjsnupscpcbpiy) left irc: Read error: Connection reset by peer02:55
hongminhee (u295@gateway/web/irccloud.com/x-rfobsaryywiizdur) left irc: Remote host closed the connection02:55
asmeurer (~asmeurer@c-174-56-21-245.hsd1.nm.comcast.net) left irc: Quit: asmeurer02:56
Alex_Gaynor (u1246@gateway/web/irccloud.com/x-wtpicwepjcumfjkp) joined #pypy.03:02
#pypy: mode change '+o Alex_Gaynor' by ChanServ!ChanServ@services.03:02
CIA-5303mikefc 07roundup * 10#959/numpypy array operation improperly jitted away: 03:03
CIA-53Bug seems to have disappeared on latest nightly. Would be useful if we knew how03:03
CIA-53this was fixed before closing issue ... * 14https://bugs.pypy.org/issue95903:03
mikefc_ (~mikefc@60-242-240-196.static.tpgi.com.au) joined #pypy.03:06
mikefc (~mikefc@60-242-240-196.static.tpgi.com.au) left irc: Read error: Connection reset by peer03:06
Nick change: mikefc_ -> mikefc03:06
nettok (~quassel@190.143.182.114) joined #pypy.03:13
hongminhee (u295@gateway/web/irccloud.com/x-muooeydowgqwbonx) joined #pypy.03:14
nettok_ (~quassel@190.143.178.250) left irc: Ping timeout: 268 seconds03:16
nettok_ (~quassel@190.143.182.114) joined #pypy.03:25
nettok (~quassel@190.143.182.114) left irc: Ping timeout: 252 seconds03:26
lahwranwhat is the recommended way to interface C code with pypy?03:31
mikefcctypes (alex told me yesterday when I asked the same question)03:36
jyp (~interrupt@175.197.245.105) joined #pypy.03:41
anish_ (anish@nat/redhat/x-apqmwgcutpqsgfwl) joined #pypy.03:42
bbot23Success: 15http://buildbot.pypy.org/builders/own-linux-x86-32/builds/190203:47
nedbat (~nedbat@python/psf/nedbat) left irc: Ping timeout: 248 seconds04:15
bbot23Success: 15http://buildbot.pypy.org/builders/pypy-c-app-level-linux-x86-64/builds/65704:53
bbot23Success: 15http://buildbot.pypy.org/builders/pypy-c-Ojit-no-jit-linux-x86-32/builds/84004:54
bbot23Success: 15http://buildbot.pypy.org/builders/pypy-c-app-level-linux-x86-32/builds/149205:05
mat^2 (~mathias@87.55.0.198) left irc: Ping timeout: 248 seconds05:14
bbot23Success: 15http://buildbot.pypy.org/builders/pypy-c-jit-linux-x86-32/builds/117305:17
bbot23Success: 15http://buildbot.pypy.org/builders/pypy-c-jit-linux-x86-64/builds/65305:27
LiliEA (~LiliEAm@97-88-28-179.dhcp.roch.mn.charter.com) left irc: Ping timeout: 268 seconds05:57
Vorpal (~AnMaster@unaffiliated/anmaster) left irc: Ping timeout: 276 seconds06:00
aleksi (~aleksi@85.235.191.82) joined #pypy.06:25
bbot23Success: 15http://buildbot.pypy.org/builders/own-linux-x86-64/builds/76706:27
bbot24Failure: 15http://buildbot.pypy.org/builders/pypy-c-jit-win-x86-32/builds/30807:08
Ademan (~dan@adsl-71-141-224-79.dsl.snfc21.pacbell.net) left irc: Quit: leaving07:08
mat^2 (~mathias@87.55.0.198) joined #pypy.07:11
bbot24Failure: 15http://buildbot.pypy.org/builders/pypy-c-jit-macosx-x86-64/builds/31807:12
ShadowE989 (~ShadowE98@74-47-218-61.dr02.rmny.wv.frontiernet.net) left irc: Quit: Leaving07:28
ShadowE989 (~ShadowE98@74-47-218-61.dr02.rmny.wv.frontiernet.net) joined #pypy.07:32
nettok_ (~quassel@190.143.182.114) left irc: Remote host closed the connection07:37
JaRoel|4d (~jaroel|4d@office.fourdigits.nl) joined #pypy.07:42
jyp (~interrupt@175.197.245.105) left irc: Read error: No route to host07:46
jyp (~interrupt@220.73.24.227) joined #pypy.07:47
jyp (~interrupt@220.73.24.227) left irc: Remote host closed the connection07:47
jyp (~interrupt@220.73.24.227) joined #pypy.08:02
Ademan (~dan@adsl-71-141-224-79.dsl.snfc21.pacbell.net) joined #pypy.08:04
mat^2 (~mathias@87.55.0.198) left irc: Ping timeout: 252 seconds08:13
mat^2 (~mathias@87.55.0.198) joined #pypy.08:15
antocuni (~antocuni@host117-71-dynamic.58-82-r.retail.telecomitalia.it) joined #pypy.08:15
oal (u4126@gateway/web/irccloud.com/x-npcymlcuqwjnizie) left irc: Read error: Operation timed out08:16
alexsuraci (u1962@gateway/web/irccloud.com/x-oksppyhxdlxsrudn) left irc: Read error: Operation timed out08:16
Alex_Gaynor (u1246@gateway/web/irccloud.com/x-wtpicwepjcumfjkp) left irc: Excess Flood08:19
bfirsh (u1308@gateway/web/irccloud.com/x-pkpvgeeuoeqbnthj) left irc: Excess Flood08:19
alexsuraci (u1962@gateway/web/irccloud.com/x-tjiivangckhudhak) joined #pypy.08:21
kkris (~kris@178.114.47.89.wireless.dyn.drei.com) joined #pypy.08:23
Alex_Gaynor (u1246@gateway/web/irccloud.com/x-hppuplnqihshavje) joined #pypy.08:25
#pypy: mode change '+o Alex_Gaynor' by ChanServ!ChanServ@services.08:25
oal (u4126@gateway/web/irccloud.com/x-rcoxytvybconcfsl) joined #pypy.08:25
bfirsh (u1308@gateway/web/irccloud.com/x-pdonzebgljlrwclh) joined #pypy.08:30
espes (espes@2001:470:1f05:ca1::) left irc: Read error: Connection timed out08:31
espes (espes@2001:470:1f05:ca1::) joined #pypy.08:32
Arach (~arach@128.75.133.76) left irc: Ping timeout: 240 seconds08:49
kkris1 (~kris@80-123-45-103.adsl.highway.telekom.at) joined #pypy.08:50
kkris (~kris@178.114.47.89.wireless.dyn.drei.com) left irc: Ping timeout: 248 seconds08:52
kennethreitz (~kennethre@c-24-127-96-129.hsd1.va.comcast.net) left irc: Ping timeout: 240 seconds08:56
jyp (~interrupt@220.73.24.227) left irc: Remote host closed the connection08:59
Arach (~arach@128.75.168.204) joined #pypy.09:01
kenaan12fijal default 11a91f8bee90b0 15/pypy/tool/jitlogparser/: some fixes09:07
kenaan12fijal jitviewer 110768b9ec165c 15/bin/jitviewer.py: fix PATH09:14
kenaan12fijal jitviewer 1118fe3ae6deff 15/: some progress. Disable css_class because it's outrageous09:14
`fox` (~fox@host139-108-dynamic.245-95-r.retail.telecomitalia.it) joined #pypy.09:17
G2P (~G2P@fw-asn1.ornis.com) joined #pypy.09:23
legogris (~legogris@c80-217-207-157.bredband.comhem.se) left irc: Ping timeout: 252 seconds09:27
legogris (~legogris@c80-217-207-157.bredband.comhem.se) joined #pypy.09:29
kenaan12hakanardo jit-improve-nested-loops 113eafa11552d5 15/pypy/jit/metainterp/optimizeopt/: prevent a BadVirtualState exception from escaping here, the loop will be discovered to be inva...09:41
fijalhakanardo: ping09:47
hakanardomorning09:47
fijalso what is ABORT_BAD_LOOP?09:47
hakanardoit is used when we try to retrace a loop09:49
hakanardobut ends up escaping the loop and tracing something else09:49
hakanardoie we find some other loop than the one we were trying to retrace09:50
hakanardothis happens after we have traced a bridge that ends in a virtual state that does not mach the loop09:52
hakanardothis non-matching bridge is optimized and placed in self.partial_trace09:52
hakanardoending with a label instead of a jump09:52
hakanardowe then continue to trace from that point expecting to retrace the loop to make a new specialized version of it09:53
hakanardobut the trace closes in on it self at some other point than the label at the end of the initial partial_trace09:54
kenaan12fijal default 117924477d6680 15/pypy/tool/jitlogparser/parser.py: some simplifications09:54
hakanardoso what we could do instead of raising ABORT_BAD_LOOP would be to extend the trace09:54
hakanardosorry extend the partial_trace with the extra operations between the label at it's end09:55
hakanardoand the point where the loop closes09:55
hakanardoand then produce a new loop insteda of a retrace09:55
hakanardoam I making sens to you?09:55
fijalyes09:56
fijalI wonder how harmful that is09:56
fijalhakanardo: right now I see it few times per a test run09:56
fijal(~20s on CPython)09:56
fijaland tons of ABORT_TOO_LONG09:56
kenaan12fijal jitviewer 117d2ec216e348 15/_jitviewer/: simplify and reintroduce cssclass, this time saneish09:56
hakanardofor somewhat longer loops ABORT_BAD_LOOP should be rare09:57
hakanardosince it will only hapen if you try to retrace the very last iteration09:58
fijalwell, it's by far not rare09:58
hakanardoie you have to escape the loop exactly while tracing09:58
mat^2 (~mathias@87.55.0.198) left irc: Ping timeout: 276 seconds09:59
hakanardook so we should fix that case then :)09:59
fijalok10:01
fijalwell, I'm observing it on a regular basis10:01
fijalwhen running pypy py.test module/micronumpy/test/test_numarray.py10:01
hakanardocases like this where you dont get simple loops but more advanced graphs of traces needs some attention I think...10:01
fijalI'm also observing ABORT_TOO_LONG way more often10:01
fijalbut I have no idea where10:01
fijalno idea how to fix ti10:01
hakanardoinreacse the limit?10:02
fijalno, that ends up with tons and tons of useless tracing10:04
fijalit's not that case10:04
fijalit even hits too long if I only trace from start10:04
hakanardoso, estimate a good limit for each loop dynamically at runtime 10:04
fijalhow do you know?10:04
fijalthat's probably not even doing any good10:04
fijalthe thing is you do ABORT_TOO_LONG10:04
fijaland then you do that again10:04
fijaland again and again10:04
fijalso you end up tracing all the time, for no good reason10:05
fijalwe need something like "it's ok, just don't inline anything in this piece of code"10:05
hakanardoso the issue is that you inline too much?10:06
hakanardoie you call the same function several times during a single iteration and you want a call instead of inlineing it?10:07
fijalfor example10:09
hakanardohow about this then, if you hit the limit, scan the trace for some sensible part to comple10:12
hakanardofor example the longest inlined function10:12
hakanardoand compile that part10:13
hakanardonext time you trace it should generate call_assempler for that part, right?10:13
fijalwhat happens now is you mark the longest function as "dont inline"10:13
fijaland you continue10:13
fijalthat however does not work too well10:13
fijalbecause you need tons and tons of tracing before the situation stabilizes10:14
hakanardoI see10:14
fijallike *tons*10:14
fijalbut also10:15
fijalyou might never finish10:15
fijalbecause you mark the longest function10:15
fijaland then you do from start from the longest function10:15
fijaland it might still be the longest function10:15
lahwranwho originally wrote the sandboxing code?10:17
fijallahwran: armin10:17
lahwrandarn, not here10:17
lahwranso I can't pester them about why marshal was used :|10:18
fijalas opposed to?10:18
fijalI can probably answer few questions10:18
lahwranas opposed to any other data structure - could be json, even10:18
fijaljson does not work because it does not preserve types well10:19
lahwranah10:19
fijalbesides, it's not really a good layer10:19
fijalyou need something binary10:19
lahwranwell, I suppose that's fair10:22
lahwranmy worry is that marshal is specifically marked as untrustworthy in the standard library documentation10:23
fijallahwran: you messed up marshal with pickle10:24
fijaldidn't you>?10:24
fijalhakanardo: so, what about loops that have no label?10:24
lahwranpickle is bad too, but marshal has a big warning that says something about untrusted marshals10:24
lahwranhttp://docs.python.org/library/marshal.html10:25
fijalok10:25
fijalso our marshal does not have this problem10:25
lahwranalright10:25
fijalnote that in any way it doesn't use builtin marshal10:26
fijalit uses marshal written in pure python10:26
lahwranokay then10:26
fijalhakanardo: they have no counters10:26
fijalhakanardo: yet, they're run10:26
hakanardothe target of the jump op is now a label10:28
hakanardoso you cant realy form loops without labels...10:28
hakanardobut I guess you have a trace starting at the interpreter and ending with a finnish?10:29
hakanardofor bridges (ie traces starting at a guard) we add a counter at the begining even if there is no label10:30
hakanardomaybe we should do so for all traces?10:30
hakanardothat way you'll get a counter for the part of the trace that is before the first label as well10:31
fijalthe part before the first label is usually very boring10:34
fijalunless you start there and end with a finish10:34
fijalan example is a short entry bridge that ends with a finish10:35
fijalthat has a bridge going out of there10:35
fijalnow bridge has a counter, loop does not10:35
fijalalso note that it does not have to be "from the interpreter"10:35
fijalit can also be call_assembler10:35
fijalhakanardo: am I making any sense?10:40
hakanardoyes I see10:40
hakanardoso how about we add a counter at the beginning of all traces containing no labels?10:41
fijalyes10:44
fijalthen we have to be a bit more careful10:44
fijalbecause showing those operations not always makes sense10:44
fijallike in case there is no bridge10:44
fijalalso, while we're at it10:44
fijalwhy there is greenkey displayed at "loop xxx has address yyy"10:45
fijalbut not at the beginning of loop ops?10:45
fijalthis way we can get the greenkey even if there is no debug_merge_point10:45
kenaan12fijal default 11e105505cc286 15/pypy/jit/backend/x86/assembler.py: append debugging code at the beginning even if for loops10:47
fijalAlex_Gaynor: ping10:47
fijalhakanardo: ^^^?10:48
hakanardosorry, I dont get it, are you missing debug_merge_points?10:49
fijalthere are loops that have no debug_merge_points10:49
fijalso how do I determine what sort of greenkey it is?10:49
hakanardodunno...10:49
hakanardothere is TargetToken.targeting_jitcell_token10:50
hakanardowhich refers to the jitcell, can you get the greenken from there?10:51
hakanardomaybe not10:51
fijalthere is loopname being passed to assemble_loop10:51
fijalsurely you can pass it to logging as well10:51
fijalno?10:51
hakanardoI have a bit no clue, but I suppose so...10:54
fijalhakanardo: can we remove if False: in compile.py?10:54
hakanardoassemble_loop logs the loopname, but assemble_bridge does not, is that the issue?10:54
fijalno10:55
fijalit's that assemble_loop logs the loopname10:55
fijalbut the part that logs IR does not10:55
fijalwhich means that you have to fish loopname from somewhere else10:55
hakanardoIR?10:55
fijalintermediate representation10:55
fijallog-opt10:55
fijalhakanardo: can I remove debugging leftovers from compile.py?10:56
fijalthey're inside if False10:56
fijaland contain assert False10:56
hakanardofijal: sure, please do10:56
hakanardoI can see no issue with passing the loopname to log_loop in send_loop_to_backend10:58
fijalyes11:01
Action: fijal does it11:01
kenaan12fijal default 118417bf8c3a59 15/pypy/tool/jitlogparser/: more small fixes11:02
kenaan12fijal default 11d69a1af57f0c 15/pypy/jit/metainterp/compile.py: remove some nonsense11:02
kenaan12fijal default 112971cd3e5323 15/pypy/jit/metainterp/: pass the loop name around11:02
hakanardore e105505cc286: we need to fix _register_counter as well11:02
hakanardothere is now 3 cases, bridge, loop, and labels11:03
hakanardoso I guess struct.bridge needs to be a 3 value enum instead of a boolean11:03
fijalright11:05
fijalhakanardo: do you feel like fixing?11:05
fijaland maybe writing a test?11:05
fijal;-)11:05
hakanardook11:05
hakanardobut I need lunch first11:06
fijalthat is fine :)11:06
kenaan12fijal default 1117fd3576a153 15/pypy/jit/metainterp/test/test_compile.py: fix the test11:06
lahwranfrom pypy.tool.lib_pypy import import_from_lib_pypy; marshal = import_from_lib_pypy('marshal')11:07
lahwranheavens, why11:07
lahwranthat import is taking approximately 30% of pypy_interact's time :|11:07
azanella (~azanella@189.6.80.207) joined #pypy.11:08
fijallahwran: precisely because marshal from stdlib python can't be trusted11:10
fijalwhy are you spawning pypy_interact all the tmie11:11
fijal?11:11
lahwranonce per request11:11
lahwranshould I not be?11:11
lahwrannedbat wanted to as well11:11
fijalok11:12
fijalwell, I don't know, maybe you can optimize it by keeping it in memory11:12
fijalso which part is taking how much time?11:12
lahwranhm, perhaps11:13
lahwranI honestly didn't even think of keeping pypy_interact in memory but starting new subprocesses11:13
lahwrannot sure how well twisted would like that11:13
fijaltwisted like such stuff I believe11:14
kenaan12fijal default 1159d04a0ce4a7 15/pypy/module/micronumpy/: provide get_printable_location for numpy11:14
fijallahwran: anyway, I can probably help with import times a bti11:14
Action: fijal is on the rage today11:14
lahwranwell, the issue seems to be focused mainly in version.py11:14
pedronis (~pedronis@73-53.195-178.cust.bluewin.ch) left irc: Ping timeout: 244 seconds11:14
lahwranwould you like a copy of the cProfile output?11:15
fijalpaste it somewhere11:15
lahwranpypy/mobule/sys/version.py specifically11:15
fijaloh11:15
kenaan12fijal default 1113448846544c 15/pypy/jit/metainterp/test/test_logger.py: fix another test11:15
fijalwhy does it import pypy/module?11:15
lahwranI think pypy/tool/lib_pypy is importing that11:16
lahwranalthough11:16
lahwranno, yeah, that's the one11:16
lahwranfrom pypy.module.sys.version import CPYTHON_VERSION11:16
pedronis (~pedronis@73-53.195-178.cust.bluewin.ch) joined #pypy.11:19
fijallahwran: yeah, fixing11:20
lahwranit looks to me like it only needs LIB_PYPY, not any of the other LIB_ variables11:21
kenaan12fijal default 11ecf82a6d3959 15/pypy/translator/sandbox/: remove some unnecessary dependencies11:24
lahwranoh?11:25
lahwranthought we were looking at lib_pypy.py11:25
lahwranerr, crap, I'm no good with hg. how do I merge with the default branch? I'm currently on nedbat-sandbox11:27
fijalhg merge default11:28
fijalwasn't nedbat sandbox merged anyway?11:28
fijalyes, it was11:28
fijalyou can just do hg up default11:28
lahwranah11:31
lahwrandrat, pypy is huge11:34
lahwranso many things I don't know how to use :|11:34
fijallike what?11:35
lahwranpy.log11:36
lahwran"Warning: cannot find your CPU L2 cache size in /proc/cpuinfo"11:37
fijalI don't think that's interesting any more11:42
fijalwe decided not to care11:42
lahwraneh?11:43
fijallahwran: you can rebuild your sandbox11:43
lahwrando I have to? :(11:43
lahwranthat takes hours11:43
fijalyou certainly don't11:44
fijaland it certainly does not take hours11:44
fijal20min11:44
fijal?11:44
fijalmaybe 1511:44
kenaan12fijal default 11b67ec3f6c00f 15/pypy/jit/backend/x86/: Fix assembler to differentiate between 3 kinds of counters11:44
lahwranuhhh, what kind of relativistic timeframe do you live in?11:45
dgl (~dgl@109.86.165.231) joined #pypy.11:45
lahwrantakes four hours for me11:45
fijallahwran: me?11:45
dgl (~dgl@109.86.165.231) left irc: Client Quit11:45
fijallahwran: wow, what OS/RAM/CPU you have?11:45
ronnyfijal: building pypy takes 3-4 hours here11:45
lahwranp4ht/3gb11:45
lahwranlinux2, ubuntu 10.1011:45
fijallahwran: 32bit version using pypy should build in ~30min11:45
fijalpython2.7 is ~30% slower11:46
fijalpython2.6 is slow as hell11:46
lahwranoh11:46
lahwranthat would be it. I'm on python2.6 for default cpython installation11:46
fijalthen stop :)11:46
fijaluse pypy :)11:46
lahwrank11:46
fijalanyway11:46
fijalyou only need to rebuild if you want the warning to go away ;-)11:46
lahwranare there any other "freak warnings" like that?11:47
lahwranie, can I count on stdout/stderr being clean11:48
fijalthis is not a freak warning11:48
lahwranoh?11:48
fijalthis is actually important, or rather was11:48
fijalit's for picking up the correct size of GC structures depending on your cache11:49
fijalbut it seems that modern architectures don't care that much11:49
lahwranah11:50
kenaan12fijal default 116d97be67953c 15/pypy/jit/backend/x86/test/test_runner.py: fix the test11:53
fijallahwran: we seriously don't give users freak warnings11:53
lahwranalright11:53
fijalit's less of an issue for sandbox11:53
lahwranso that's the one "freak" warning I can expect?11:54
lahwranby "freak" I mean "out of the ordinary"11:54
hakanardofijal: so you fixed it yourself?11:55
tellone (~tellone@81-232-82-223-no32.tbcn.telia.com) joined #pypy.11:57
hakanardohere is my version: http://paste.pocoo.org/show/526376/11:57
lahwranerm, did the required arguments change to translate.py? https://gist.github.com/60f64f5e11f27565afc312:00
lahwranI don't remember specifying -Oanything last time I translated12:00
nickik (~nick@2001:67c:20a1:1018:226:9eff:feef:8369) joined #pypy.12:08
nedbat (~nedbat@python/psf/nedbat) joined #pypy.12:08
lahwrannedbat: I ran cProfile on pypy_interact, seems that it was importing pypy.module.sys.version indirectly, which took ... quite a long time12:14
fijalhakanardo: yeah, I need a test probably12:14
nedbatlahwran: I don't know what pypy.module.sys.version is.12:15
lahwranneither do I, I just know it took forever to import :p12:15
fijallahwran: I removed the import, didn't i?12:15
lahwranyes12:15
mat^2 (mathias@87.55.0.198) joined #pypy.12:15
lahwranemphasis on "was" :P12:15
nedbatlahwran: where is the import?12:15
lahwrannedbat: fijal already fixed it12:15
lahwranI was getting to that :P12:16
fijallahwran: -O2 or -Ojit12:16
fijaldepending if you want a jit or not12:16
nedbatchangeset?12:16
nedbati'd like to understand what happened.12:16
anish_ (anish@nat/redhat/x-apqmwgcutpqsgfwl) left irc: Quit: Leaving12:16
lahwranecf82a6d395912:16
fijalnedbat: nothing interesting really12:16
fijaljust making sure that imports don't pull the entire pypy12:17
nedbatfijal: right.  I have a few to merge also.12:20
fijalcool12:20
squiddy (~squiddy@46.128.142.118.dynamic.cablesurf.de) joined #pypy.12:22
fijalhakanardo: either way, thanks for helping12:23
lahwranheavens, even run in pypy translate.py takes ages12:23
fijallahwran: as I said ~20min for the sandbox12:23
fijaljitless probably less12:23
lahwrandoes jit cause any kind of performance degradation?12:25
lahwranI know it's a net gain, but are there any parts of it that are a loss?12:25
fijaltranslation is longer12:27
fijalbut also, in theory, the security is slightly lower12:27
ivanand uses more memory, no?12:27
lahwranfijal: ho?12:28
lahwranoh*12:28
lahwranhow's that work?12:28
fijalivan: yes12:29
fijallahwran: how does what work?12:29
lahwranthe security is slightly lower in what way12:29
kenaan12ned nedbat-sandbox-2 117aa1376fb3bc 15/: Merged d9b372cf25b0 to nedbat-sandbox12:30
derdon (~derdon@pD9E1D2A3.dip.t-dialin.net) joined #pypy.12:30
fijalsay if we have a bug in handling out-of-memory conditions in the JIT12:30
fijalthat doesn't respect return value of malloc (we had this before)12:31
fijalyou might lead to a potential segfault12:31
lahwranah12:31
lahwranso it'd be the kinda thing where someone would /really/ have to be looking12:31
fijalwell, yes12:31
lahwranto the point that pypy-sandbox isn't even the optimal point of entry12:31
lahwran:P12:31
fijalit's just that the default sandbox is *really* secure12:31
lahwrank, so -O2 is jitless?12:32
fijalthe jitted one is slightly less so12:32
fijalyes, -O2 is jitless12:32
lahwranalso, this seems to have happened while compiling: https://gist.github.com/c283493cb1c711c3d42c12:32
fijalsimply because bugs in jit more often can cause segfault12:32
fijallahwran: install libffi-dev12:32
lahwranis that a dependency for jit?12:33
fijalno idea why it did not check at the very beginning :(12:33
fijalapparently :/12:33
lahwrannah, I'll just switch to -opt=212:33
fijalmakes sense if you don't care about performance12:34
fijalnote that for the sandbox to make sense being jitted, the programs would have to consume >0.5s CPU time12:34
lahwranah12:34
lahwrandoubtful that will happen except when someone's trying to break it12:34
fijalso the JIT is not buying you anything12:35
kenaan12ned nedbat-sandbox-2 11808b12f6b195 15/: Merge tip to nedbat-sandbox-212:39
tellone (~tellone@81-232-82-223-no32.tbcn.telia.com) left irc: Ping timeout: 240 seconds12:40
fijalcrap, compiled the wrong branch12:41
nedbatfijal: can I merge my logging changes in the sandbox to default?12:42
fijalnedbat: you mean nedbat-sandbox-2?12:42
nedbatyes12:42
fijalcan I have a quick look?12:42
nedbatsure, no prob.12:42
nedbatI should re-build everything after my (uninteresting) merge anyway12:42
fijalnedbat: LGTM12:43
fijalmaybe debug should be false by default12:44
jnoller (~jnoller@vpn.nasuni.com) joined #pypy.12:44
nedbatfijal: yeah, I left it true since that was the "default" before. But you're probably right.12:44
Da_Blitz (~Da_Blitz@203.56.250.63) left irc: Ping timeout: 244 seconds12:44
fijalnedbat: you're the one who can make an informed decision12:45
fijaljnoller: hi jesse12:45
jnollermorning12:45
nedbatfijal: "informed decision": it's about what sandbox users expect.  It seems like most are annoyed by the logging instead of pleased.12:47
fijalyes12:47
kenaan12fijal default 11a8481f7a34ed 15/pypy/module/micronumpy/: a neat debugging feature12:48
Da_Blitz (~Da_Blitz@203.56.250.63) joined #pypy.13:02
squiddy (~squiddy@46.128.142.118.dynamic.cablesurf.de) left irc: Remote host closed the connection13:08
Rhy0lite (dje@nat/ibm/x-bivwrfbkvkqxighf) joined #pypy.13:21
fijal (fijal@196-210-199-8.dynamic.isadsl.co.za) left #pypy ("Leaving").13:21
fijal (~fijal@196-210-199-8.dynamic.isadsl.co.za) joined #pypy.13:21
fijalRhy0lite: hi13:21
Rhy0litehi13:21
Rhy0litehow goes it?13:21
Rhy0litejitviewer working again?13:21
fijalalmost :)13:23
fijaldoing final fixes13:23
lahwranffffff13:27
lahwranit still said 'Warning: cannot find your CPU L2 cache size in /proc/cpuinfo'13:27
lahwranguess I'll make my handling code ignore that then :|13:27
fijallahwran: ?13:28
fijallahwran: what did you recompile?13:28
fijalthe pypy-c-sandbox?13:28
lahwranfijal: the newly translated pypy-c13:28
lahwranyeah13:28
lahwran(sidenote, it doesn't seem to produce pypy-c-sandbox, only pypy-c13:28
fijalUGH13:29
fijalyes, different GC :/13:29
fijalsorry forgot to update stuff13:30
lahwranthe length of the string is probably stored in the binary, isn't it13:30
lahwranuh13:30
lahwrannevermind, that was a really horrible idea, shouldn't have even brought it up13:30
lahwranrelatedly, I'm gonna go sleep. o/13:30
fijallahwran: ok, I'll try to fix it13:30
fijalbut I have 17 things to fix on my mind right now13:30
fijaland 2 that I would actually like to work on13:31
lahwranwell, fix the important ones first13:31
lahwranthis is easily workaroundable13:31
CIA-5303fijal 07roundup * 10#673/Linux L2 cache size: We should remove it from other GCs as well. * 14https://bugs.pypy.org/issue67313:31
tellone (~tellone@81-232-82-223-no32.tbcn.telia.com) joined #pypy.13:42
nedbatfijal: My changes got rid of the need to "import py", and your change uses "py.path.local"...13:48
fijalchange it to use os.path then13:49
fijalit's just annoying13:49
nedbatit's a little more complicated than that:   py.path.local(pypy.__file__).join('..', '..', 'lib_pypy', 'marshal.py').pyimport()13:49
nedbat.pyimport is not in os.path13:50
nedbatfijal: also, I worry about the comment explaining why stdlib marshal isn't used.13:51
fijalit adds stuff to sys.path, imports it and removes I believe13:51
fijalor execfiles it13:51
fijalI dunno13:51
fijalwell13:51
fijalstdlib marshal has a bug fat warning stating "don't use it to unmarshal untrusted data"13:52
nedbatright.13:52
nedbatfijal: but here we're unmarshalling data we marshalled.13:53
tellone (~tellone@81-232-82-223-no32.tbcn.telia.com) left irc: Ping timeout: 244 seconds13:55
fijalso?13:56
fijalwell ok13:56
fijalI see the point13:56
fijalah no13:58
fijalwrong13:58
fijalthe question is how easy it is to print arbitrary data to the output?13:58
nedbatfijal: the output of the sandbox?14:00
fijalyes14:00
nedbatfijal: the python run in the sandbox doesn't print marshal data.  It will print strings, which are then marshalled.  I guess the question is: if I give you a string, and then marshal it, can I somehow jump outside the string to make bad marshal data?  I don't think so.14:01
fijalI think you have to argue with armin, but I tend to agree14:01
nedbatok, I'll look into it, and talk to armin14:03
davidpk (~davidpk@5e09151b.bb.sky.com) joined #pypy.14:04
derdon (~derdon@pD9E1D2A3.dip.t-dialin.net) left irc: Remote host closed the connection14:04
lahwranbetter to just include the pypy marshal.py anyway14:08
voidspace (~anonymous@87-194-212-65.bethere.co.uk) joined #pypy.14:11
voidspace (~anonymous@87-194-212-65.bethere.co.uk) left irc: Changing host14:11
voidspace (~anonymous@python/psf/voidspace) joined #pypy.14:11
kenaan12fijal jitviewer 115f18887dce76 15/log.pypylog: regen log14:11
kenaan12fijal default 11974244d55fe2 15/pypy/tool/jitlogparser/: adjust to new situation14:11
fijalnedbat: how's sandbox working for you btw?14:17
nedbatfijal: it's good, but i haven't stressed it much. Been working on other aspects of the system.14:17
JaRoel (~jaroel|4d@office.fourdigits.nl) joined #pypy.14:26
espes (espes@2001:470:1f05:ca1::) left irc: Read error: Connection timed out14:27
fijalAlex_Gaynor: geeez14:28
espes (espes@2001:470:1f05:ca1::) joined #pypy.14:28
kenaan12fijal default 116406dbf9a60e 15/pypy/tool/jitlogparser/: improvements14:28
JaRoel|4d (~jaroel|4d@office.fourdigits.nl) left irc: Ping timeout: 276 seconds14:28
JaRoel|4d (~jaroel|4d@office.fourdigits.nl) joined #pypy.14:30
JaRoel (~jaroel|4d@office.fourdigits.nl) left irc: Ping timeout: 248 seconds14:31
nedbatlahwran: I'm trying to minimize the dependencies, and it's good to reason about these things.14:33
kenaan12hakanardo jit-improve-nested-loops 114481b4f255f7 15/pypy/: hg merge c8ddbb44298614:38
kenaan12fijal default 11ff4589dda2b1 15/pypy/: add some info "we're in re" or "we're in numpy"14:39
_aleksi (~aleksi@85.235.191.82) joined #pypy.14:40
PiotrSikora (~none@nginx/adept/piotrsikora) left irc: Excess Flood14:40
PiotrSikora (~none@nginx/adept/piotrsikora) joined #pypy.14:41
aleksi (~aleksi@85.235.191.82) left irc: Ping timeout: 244 seconds14:43
asmeurer (~asmeurer@c-174-56-21-245.hsd1.nm.comcast.net) joined #pypy.14:47
bbot2Started: 15http://buildbot.pypy.org/builders/pypy-c-jit-linux-x86-64/builds/654 [12hakanardo, jit-improve-nested-loops]14:49
reiner (~reiner@46.128.136.223.dynamic.cablesurf.de) joined #pypy.14:51
reiner (~reiner@46.128.136.223.dynamic.cablesurf.de) left irc: Remote host closed the connection14:51
reiner (~reiner@46.128.136.223.dynamic.cablesurf.de) joined #pypy.14:51
Nick change: reiner -> squiddy14:51
Vorpal (~AnMaster@unaffiliated/anmaster) joined #pypy.14:52
fijalAlex_Gaynor: ping14:57
bfirsh (u1308@gateway/web/irccloud.com/x-pdonzebgljlrwclh) left irc: Quit: Connection closed for inactivity15:00
tellone (~tellone@81-232-82-223-no32.tbcn.telia.com) joined #pypy.15:02
_aleksi (~aleksi@85.235.191.82) left irc: Remote host closed the connection15:13
aleksi (~aleksi@85.235.191.82) joined #pypy.15:14
etrepum (~bob@75-101-96-144.dsl.static.sonic.net) left irc: Read error: Connection reset by peer15:20
gehho2 (~gehho2@ZJ040192.ppp.dion.ne.jp) left irc: Quit: Leaving...15:31
Alex_Gaynorfijal: pong15:34
Kaskuka (~John@osbk-4db177c4.pool.mediaWays.net) left irc: Ping timeout: 240 seconds15:40
Kaskuka (~John@osbk-4db1432e.pool.mediaWays.net) joined #pypy.15:40
kenaan12alex_gaynor default 11653d45fce4df 15/pypy/module/micronumpy/strides.py: this can be unrolled sometimes15:45
derdon (~derdon@pD9E1D53F.dip.t-dialin.net) joined #pypy.15:45
bbot23Success: 15http://buildbot.pypy.org/builders/pypy-c-jit-linux-x86-64/builds/654 [12hakanardo, jit-improve-nested-loops]15:50
antocuni (~antocuni@host117-71-dynamic.58-82-r.retail.telecomitalia.it) left irc: Ping timeout: 252 seconds15:54
aleksi (~aleksi@85.235.191.82) left irc: Remote host closed the connection15:56
kenaan12alex_gaynor default 11c2d42bf471da 15/pypy/jit/metainterp/: A guard can't invalidate the heap cache in the metainterp.16:09
jyp (~interrupt@175.197.245.105) joined #pypy.16:11
JaRoel|4d (~jaroel|4d@office.fourdigits.nl) left irc: Remote host closed the connection16:11
arigato (~arigo@xdsl-188-155-166-12.adslplus.ch) joined #pypy.16:12
arigatohi16:13
arigatofijal: ok to publish the sprint announcement?  (sorry, you're the only one more-or-less on-line nowadays)16:14
jterrace (~jterrace@aegis.CS.Princeton.EDU) joined #pypy.16:16
etrepum (~bob@75-101-96-144.dsl.static.sonic.net) joined #pypy.16:19
fijalarigato: hehe :)16:24
fijalarigato: I can have a look16:24
arigatohttp://morepypy.blogspot.com/p/global-interpreter-lock-or-how-to-kill.html16:24
arigatothat's the link for now (ignore the bogus name in the link)16:24
fijalyes?16:24
arigatothat's what I plan to publish16:25
arigatoif it looks ok for you16:25
fijalarigato: do you want me to add extra stuff, or not really?16:25
arigatosure, if you have ideas16:25
fijalotherwise looks good16:25
fijalARM buildbot16:25
fijalfor example16:25
arigatoyou don't *have* to add extra stuff16:25
Arfrever (~Arfrever@apache/committer/Arfrever) joined #pypy.16:26
fijalok :)16:26
fijalthen looks good16:26
fijalAlex_Gaynor: I'm doing experiments16:26
arigatoyes, that's part of "integrate tests for ARM"16:26
fijalI can't quite get why gcc is faster, but well, whatever16:26
Alex_Gaynorfijal: I'm about to start working on a mockup.16:26
fijalcool16:26
Action: arigato publishes16:27
Action: Alex_Gaynor should probably start with pen and paper.16:27
fijalAlex_Gaynor: probably :)16:27
fijalAlex_Gaynor: anyway, I can now write a blog post with various algorithms and version, explaining where we are and where we are going16:27
fijalregarding laplace.py16:27
Alex_Gaynorfijal: sounds good16:28
fijalthere are various questions which I don't quite understand16:28
fijalbut well16:28
fijalarigato: I did some work on jitviewer, but there are few questions regarding "what do you want to see"16:28
Alex_Gaynorfijal: I've been playing a bit with making the JIT do better on the Python side of x[:,:] = y16:28
fijaland?16:29
Alex_Gaynorfijal: I haven't been benchmarking evne, just reading traces16:29
fijalok16:29
Alex_Gaynorfijal: re-translating with the heapcache opt I just pushed16:29
fijalAlex_Gaynor: I'm a bit skeptical about all this unrolling16:30
fijalmaybe we should just be happy with interpreter-level speed16:30
fijalbut whatever16:30
Alex_GaynorI'm going to find breakfast and sketch16:31
Rhy0litefijal: what's still confusing about GCC?16:31
Rhy0liteI haven't seen the latest code from pypy16:31
fijalRhy0lite: why our memcopy is so slow16:32
fijalAlex_Gaynor: this does not quite cut, especially that we don't always have it jitted16:32
Rhy0litefijal: not laplace?16:32
fijalRhy0lite: I'm shredding it a bit to pieces16:32
Rhy0liteokay16:32
fijalbut anyway, I guess I'm ready to just post it to the blog16:33
Rhy0litewhat is the current pypy gap?16:33
fijalit depends :)16:33
fijalthe same sort of style is almost 2x faster16:33
fijaler, slower16:33
Rhy0liteokay, then it likely still is memory accesses16:33
fijalbut there is different cost associated with the equivalent operations16:34
fijaland I don't quite understand why16:34
fijalarigato: can we discuss this briefly?16:34
arigatoa sec16:35
kenaan12arigo extradoc 11e88672794b8f 15/sprintinfo/leysin-winter-2012/announcement.txt: Fix the description16:35
fijalAlex_Gaynor: http://www.reddit.com/r/Python/comments/nr8g8/python_3_wall_of_shame_updates/c3bj5fb16:36
arigatofijal: ok16:39
fijalarigato: so right now we (usually) get 3 pieces of code16:39
fijalbefore the first label, from first label and from second label16:40
fijalhow would you like this presented16:40
fijal?16:40
arigatogood question16:40
fijalnote that before-the-first-label part is usually boring16:41
fijalbut sometimes not at all16:41
fijallike if we have a from-start trace that ends with a finish16:41
arigatodo from-start traces have labels?16:41
fijalno16:42
arigatoI see16:42
arigatointeresting question16:42
fijalright now at least the list and numbers of hits are correct16:42
fijalloading of source code does not quite work16:42
arigatonumber of hits is counted where?16:42
arigatoat the start of the trace, after every label, and at the start of every bridge?16:43
fijalyes16:43
arigatook16:43
fijalall of the above16:43
arigatogood16:43
fijaltook few iterations :)16:43
arigato:-)16:43
fijalthe trace world is completely different now - no wonder jitviewer stopped working16:44
arigatoyes16:44
fijalso indeed, what would you like to see16:44
fijalsome of the code for example have no longer any obvious place in the python source code to correspond do16:45
arigatomaybe it should display one line per loop (like now), but if necessary with two links: one link that goes to the start of the loop, and another link that goes to the loopy part only16:45
fijalcan I assume from start loops have no labels?16:45
fijalso a loopy thing is with a trace16:45
fijaler16:45
arigatono16:46
fijala loopy thing is with a label16:46
fijalok16:46
fijalso what assumptions are there?16:46
arigato"a loopy thing" would mean "something where the final jump goes back to a label in the same loop16:46
fijalok16:46
arigato(usually the last one, that's good enough)16:46
fijaland from start traces?16:47
fijalbecause now we can nicely distinguish them16:47
fijal(maybe)16:47
arigatoah16:47
arigato"from start" traces can end with a loop, and *bridges* can too16:47
fijalwith a loop?16:48
arigatowith a "loopy thing" as defined above16:48
fijalwhat do you mean?16:48
arigatoa bridge can contain a label and end in a jump back to this label16:48
fijalah, so a bridge can have a label and a jump to that label?16:48
fijalfor example?16:48
arigatoyes16:48
fijaloh wow16:48
arigatoyes :-)16:48
fijalthat's new :)16:48
fijalthe current model does not fit at all16:48
arigatoindeed, but that's also simpler16:48
arigatofrom the point of view of the front-end16:49
G2P (~G2P@fw-asn1.ornis.com) left irc: Quit: Leaving.16:49
arigato:-/16:49
fijalit's ok16:49
fijalcomplexity moves to better places16:49
arigatomaybe you should have two lists of stuff to display16:49
arigatoone that shows the "loopy things"16:50
arigatoand the other that shows the "loops" in the sense of the front-end16:50
arigatoI think it was renamed "procedures"16:50
arigatoyes16:50
arigatonow a "procedure" is the new name for what used to be a "loop"16:51
arigatoyou have procedures and bridges16:51
kkris1 (~kris@80-123-45-103.adsl.highway.telekom.at) left irc: Ping timeout: 240 seconds16:51
Alex_Gaynorfijal: pypy is a visualization project that builds VMs to have cool data16:57
Alex_Gaynorfijal: (I have a sketch now)16:57
dracman (~draco@31-17-70-40-dynip.superkabel.de) joined #pypy.17:01
squiddy (~reiner@46.128.136.223.dynamic.cablesurf.de) left irc: Remote host closed the connection17:01
kkris (~kris@188-22-115-215.adsl.highway.telekom.at) joined #pypy.17:07
mat^2 (mathias@87.55.0.198) left irc: Ping timeout: 240 seconds17:08
fijalarigato: hm, that sounds very confusing17:12
fijalif you ask me17:12
arigatowhat does?17:16
fijalto have two views17:16
arigatoah17:16
fijalbecause for any non-trivial program you would have a mix of from-start and loops17:17
arigatook17:17
fijalnon-trivial = py.py, twisted.web etc.17:17
arigatosure17:17
fijalI think a consolidated view makes some sense17:17
fijalbut to order it somehow in most-commonly-run order17:18
fijaland not just randomly17:18
arigatogit tells me:17:20
arigatoYour branch is behind 'origin/master' by 197 commits, and can be fast-forwarded.17:20
arigatohow do I fast-forward it???17:20
arigato(I tried to find the answer to this question for a while yesterday, and failed)17:20
rguillebertgit rebase origin/master17:21
rguillebertprobably17:21
arigatoeh???? ok thanks17:21
arigatoworks17:21
arigato(I have no local changes at all)17:22
Alex_Gaynorarigato: git pull17:22
fijalarigato: that's a really useful message17:23
fijallast time I used git it was17:23
fijalerror: 717:23
fijalor something as informative17:23
arigato:-(17:23
nettok (~quassel@190.143.183.5) joined #pypy.17:24
cco3 (conleyo@nat/google/x-evoqkaxsbacvtqkr) left irc: Read error: No route to host17:27
Alex_Gaynorfijal: that's not an error message, it comes from `git status` or so17:27
fijalmaybe17:27
fijalI think it came from git clone17:28
nedbat (~nedbat@python/psf/nedbat) left irc: Ping timeout: 245 seconds17:30
fijalarigato: another idea17:33
fijalarigato: how about we perform the major collection in parallel17:33
fijalso we still pause, but we do 1/n17:34
fijalwhere n is number of cores17:34
arigatoyes, that's what I'm trying on the concurrentgen branch17:34
arigatoeven with the minor collection17:34
fijalah ok17:34
arigato(more important, because it has a larger percentage of time)17:34
arigatobut it parallelize only 20% of the work or so, I don't see how more than 2 cores helps17:35
fijalmost of the time I've seen copy-and-drag-out-of-nursery17:35
fijalbeing the main blocker17:35
fijalisn't it more parallelizable?17:35
arigatoit takes 20% of a typical run17:36
fijaltypical run of ...17:36
fijal?17:36
arigatoI've no clue how you can "parallelize" it more than by taking 20% away17:36
fijalof course17:36
Action: arigato -> dinner17:36
fijalok17:36
Alex_Gaynoramdahl's law!17:37
fijalAlex_Gaynor: if you find some cool trace visualization let me know ;-)17:37
Alex_Gaynorfijal: working on turning my sketch into HTML atm17:37
fijalokey17:38
mat^2 (mathias@87.55.0.198) joined #pypy.17:40
arigato (~arigo@xdsl-188-155-166-12.adslplus.ch) left irc: Ping timeout: 252 seconds17:42
Rhy0litefijal: Are PyPy developers aware of the weird terms of service for Bitbucket?17:49
ivan3.12. Service Rules.?17:52
ronnyRhy0lite: after a first skimming the current tos doesnt seem to be too weird for me, whats the issue?17:54
Rhy0literonny: Confidentiality17:54
Rhy0litespecifically: "Confidential Information will also include the Applications, Software, Documentation and any new product or service information."17:55
fijalRhy0lite: what does that mean?17:55
Rhy0liteexactly17:55
Rhy0liteIt's a bi-directional confidentiality agreement17:55
Rhy0liteit essentially makes every subscriber part of a non-disclosure agreement17:56
fijalisn't it a standard clause?17:56
Rhy0litefijal: Github does not have similar terms17:56
fijalok17:56
fijalI personally don't care I didn't read it17:56
ronnyRhy0lite: i wonder if that one applies to working on shared private repos17:57
Rhy0liteIt's not clear what they mean by "Applications, Software, Documentation and any new product or service"17:57
Rhy0liteit's not clear *whose* applications, software, etc.17:57
Rhy0liteand what is a new product17:57
Alex_GaynorRhy0lite: I wouldn'tbe surprised if it was something atlasian stuck there from some other product without realizing it didn't apply very well.  I know the guy who created bitbucket I can ask him what it means if you like?17:57
kenaan12fijal default 1184b731e827a6 15/pypy/tool/jitlogparser/: a minor fix17:58
kenaan12fijal default 1156272f637518 15/pypy/: merge17:58
Rhy0liteAlex_Gaynor: yes, that could be helpful17:58
Rhy0litea naive interpretation is that any new feature in PyPy is a new product or service17:58
Rhy0liteand implicitly confidential based on the ToS17:59
Rhy0liteI think they meant it for a subscriber hosting confidential / proprietary information on BitBucket18:00
Rhy0liteand BitBucket employees might need permission to access for a service request18:00
Rhy0liteas well as any analytics information about BitBucket to which subscribers are allowed access18:01
Rhy0litefor instance, is any of the JavaScript that BitBucket downloads a confidential application?18:03
Action: ronny wonders what the correct word for programmers speculating on law is18:05
Alex_Gaynorfijal: ping18:05
Rhy0literonny: it's not my speculation, it's questions from my employer18:05
Alex_Gaynorronny: The Law :)18:05
ronnyRhy0lite: so why do you wonder here, instead of consulting people that should know?18:07
Rhy0literonny: I want to know what pypy developers think of this relative to other hosting websites without the clause.18:08
Alex_GaynorRhy0lite: By all accounts we don't :)18:11
Rhy0lite:-)18:11
fijalAlex_Gaynor: pong18:11
Alex_Gaynorfijal: what do you think the "key stats" for a log are?  So far I've gone with traces compiled and traces aborted, 2 or 3 more might be good18:11
fijalwhat do you mean?18:12
Alex_Gaynorjust some stuff I'm putting in the sidebar which will show up on every page18:12
fijalso what would you put in aborted traces?18:12
fijalI don't think they're that interesting18:13
fijalbecause you usually want to inspect them one-by-one18:13
Alex_Gaynorfijal: these are just counts of things18:13
fijalah ok18:13
fijalthen yes18:13
Alex_Gaynorso compiled traces: 4, aborted traces: 018:13
fijalyou can break down traces18:13
fijalinto compiled from start18:13
Alex_Gaynorjust any other important numbers?18:13
Alex_Gaynorfijal: too detailed, that's for a different page :)18:14
fijalI think we should measure number of aborted assemblers due to quasi immutable failing18:14
fijalbut I guess we'll figure this out as we go18:14
Rhy0liteAlex_Gaynor: can you provide any stats about coverage?18:15
Rhy0litehow much of the application has been traced?18:15
Rhy0litehow much of the traces have been compiled?18:15
Alex_GaynorRhy0lite: hmm, perhaps someday, ATM coverage isn't possible18:15
Alex_Gaynorthough that's definitely an interesting idea for the future18:15
fijalAlex_Gaynor: I want some warmup info18:17
Rhy0liteAlex_Gaynor: what about stats for failed guards in traces?18:17
fijalRhy0lite: this is a bit reflected in bridges18:17
fijalbut also not as interesting for us as for other jits18:17
fijalbecause we can *always* compile assembler18:18
fijalAlex_Gaynor: honestly, I don't find any single summary any useful so far18:18
fijalthink about it, what would you use that for?18:20
fijalunless it's highlighted in red *IT WAS TRACING ALL THE TIME*18:20
Rhy0liteAlex_Gaynor: average length of traces in # of native instructions?18:20
fijalsome sort of average density of native instructions per bytecode18:20
fijalor IR/bytecode18:20
fijalis sometimes useful18:21
fijal*sometimes*18:21
Alex_Gaynorfijal: I'm working on something that will make it very clear if you are tracing all the time, give me 10 minutes and I can show you the first secreenshot18:21
fijalokey18:21
Action: fijal is not in a rush18:21
fijalAlex_Gaynor: I'm trying to make senses out of traces *on python level*18:21
fijalfailing so far ;]18:21
ronnyrandom idea, would it be possible to make traces reasonably symbolic wrt things like global objects/function parameter/object layouts18:25
Rhy0literonny: do you mean additional comments in the traces relating back to user variables?18:26
fijalronny:yes, but hard18:27
fijalronny: besides, I don't think anyone wants that18:27
Rhy0litefijal: maybe I am thinking about something different than what ronny intends, but it seems that it would have been useful to relate the address to user array elements while investigating laplace.py performance18:28
ronnyRhy0lite: my basic idea is, that if a trace is completely symbolic, one can dump the traces and generated code and reload them later18:30
Rhy0liteah, hmm18:30
Rhy0liteis this related to pre-warmed cache?18:30
ronnydefine pre-warmed cache18:30
Rhy0liteloading traces and machine code at startup when re-running an application instead of waiting for pypy to rediscover traces on each invocation18:32
fijalRhy0lite: what I'm, saying is I know where those things come from18:33
voidspace (~anonymous@python/psf/voidspace) left irc: Quit: voidspace18:37
antocuni (~antocuni@host117-71-dynamic.58-82-r.retail.telecomitalia.it) joined #pypy.18:38
fijalRhy0lite: I can sell knowledge how to use it, but there is very little incentive for me to make them more user friendly18:38
fijalsince I can usually find where stuff comes from18:38
fijalRhy0lite: btw, I'm always getting cynical when people want from me features that don't really sort any of my problems18:39
Alex_Gaynorfijal, Rhy0lite: Ok first pass at the mockup, so far just the page+sidebar: http://i.imgur.com/CWQVv.png18:39
fijalAlex_Gaynor: sounds good18:39
fijalAlex_Gaynor: note that we can't right now have both of those info's there18:40
fijalfrom one log18:40
fijalbut that is fixable18:40
Alex_Gaynorfijal: sure you can PYPYLOG=file.pypylog18:40
fijalyes, but then there are no traces18:40
fijalso you have to rerun to get actual loops18:40
Alex_Gaynoryou get traces with that18:40
fijalno you don't18:40
fijalAlex_Gaynor: I would put some diagram at the front with numbers18:41
fijalor maybe hmmm18:41
fijallet me do some mockup as well18:41
fijalAlex_Gaynor: I would do relative bars with number of runs per iteration18:42
fijalas the main view18:43
Alex_Gaynorfijal: wtf you're right PYPYLOG=x.pypylog doesn't have logs18:43
fijaland then if you click on one you get only that trace18:43
fijalwith links18:43
Rhy0liteare there any stats about whether a trace returns to the interpreter versus jumping to another trace?18:43
fijalRhy0lite: no18:44
fijalyou can get that (probably) but it's relatively boring18:45
fijalunless you want to measure the warmup18:45
jnoller (~jnoller@vpn.nasuni.com) left irc: Quit: Leaving...18:57
jyp (~interrupt@175.197.245.105) left irc: Remote host closed the connection19:00
nedbat (~nedbat@python/psf/nedbat) joined #pypy.19:05
Trundle (~andy@p5B2DB738.dip.t-dialin.net) joined #pypy.19:22
Trundle (~andy@p5B2DB738.dip.t-dialin.net) left irc: Changing host19:22
Trundle (~andy@python/site-packages/trundle) joined #pypy.19:22
DasIch_ (~dasich@p3E99135B.dip.t-dialin.net) joined #pypy.19:26
fijalAlex_Gaynor: want me to do some layouts as well?19:26
Alex_Gaynorfijal: do you have some desire to?  I'm perfecltly happy to work on it19:27
fijalok19:28
fijalnot particularly, I can get stuff done on the pypy side19:29
DasIch (~dasich@p4FFDF3A8.dip.t-dialin.net) left irc: Ping timeout: 240 seconds19:30
davidpk (~davidpk@5e09151b.bb.sky.com) left irc: Quit: davidpk19:34
Rhy0liteI wish that I could come to the sprint, but it's a little less convenient from the U.S. at the last minute19:49
Alex_Gaynorfijal, Rhy0lite: And now with most of the overview page filled out: http://i.imgur.com/i2Z9V.png19:50
fijalAlex_Gaynor: I would put stderr/stdout and total time19:51
fijalsomewhee there19:51
Rhy0litevery nice19:51
Alex_Gaynorfijal: ah yes, stdin/stdout good idea19:51
Alex_Gaynorerr err19:52
fijal:]19:52
Alex_Gaynorfijal: execution time and stdout/stderr: http://i.imgur.com/eXaur.png19:55
Rhy0liteshoul Log Stats be on the right?19:56
Alex_Gaynorfijal: err, is there a way to get JIT options t runtime?  e.g. if I do pypy --jit threshold=4 where is that?19:56
Alex_GaynorRhy0lite: I don't know, I need to figure out how to better organize those sections19:57
Rhy0liteI understand that its associated with the "overview" page19:57
fijalAlex_Gaynor: you can get it from somewhere19:57
Rhy0liteand you want to distinguish it from key stats19:57
fijalin pypyjit19:57
Rhy0litebut it seems like a lot of wasted white space19:57
Alex_Gaynorfijal: pypyjit has defaults but that's the defaults...19:57
fijalhm19:57
Alex_GaynorRhy0lite: Key Stats will be visible on all pages, not just overview though19:57
Rhy0litealex: yes19:58
fijalAlex_Gaynor: want me to put it there?19:58
Alex_GaynorI'll figure out that detail later19:58
Alex_Gaynorfijal: would be nice yes19:58
durin42 (~durin@adium/durin42) left irc: Ping timeout: 252 seconds19:58
setmeaway (~setmeaway@119.201.52.190) left irc: Quit: Leaving19:58
fijalok19:58
setmeaway (~setmeaway@119.201.52.190) joined #pypy.19:58
Rhy0liteI understand that you want the right side bar to appear in all views19:58
fijalAlex_Gaynor: looks good btw19:59
jterrace (~jterrace@aegis.CS.Princeton.EDU) left irc: Quit: Leaving.20:00
Alex_Gaynorfijal: now I'll do the trace detail view I suppose, needs more though first20:00
durin42 (~durin@adium/durin42) joined #pypy.20:00
Alex_Gaynorfijal: a place where I can get GC flags would be cool too :)20:00
fijaluh20:01
fijalin __pypy__ maybe then?20:01
Alex_Gaynor__pypy__ would be fine20:01
Alex_Gaynor__pypy__.gc.flags or such20:01
Alex_Gaynoror gc_flags20:01
Alex_Gaynoror whatever20:01
davisagli (~davisagli@davisagli.com) left irc: Excess Flood20:01
azanella (~azanella@189.6.80.207) left irc: Read error: Operation timed out20:01
davisagli (~davisagli@davisagli.com) joined #pypy.20:02
antocuni (~antocuni@host117-71-dynamic.58-82-r.retail.telecomitalia.it) left irc: Ping timeout: 252 seconds20:02
fijalI love how interp_jit.py starts with "This is not the JIT :)"20:02
fijalAlex_Gaynor: for what is worth pypy --jit typo=3 does nothing20:04
Alex_Gaynorfijal: feel like fixing?20:05
fijalyeah20:06
Alex_Gaynorfijal: jitviewer is working ATM right?20:06
fijalAlex_Gaynor: kinda20:06
fijalit does not display source code20:06
fijaland loops are kinda confusing20:06
fijaland jumps don't work20:06
fijalother than that - yes20:06
Alex_Gaynorheh, k20:06
Alex_Gaynorfijal: my loops don't have names20:07
fijalAlex_Gaynor: do you have a very new pypy20:08
fijal?20:08
fijallike 3h new20:08
Alex_Gaynorfijal: c2d42bf471da20:08
Alex_Gaynorfijal: the only change is in logparser, so yes20:08
fijalindeed20:09
fijalhg pull jitviewer?20:09
fijaland make sure you're on default?20:09
Alex_Gaynoryes andyes20:09
fijalthen it should work20:10
fijalimg?20:10
Alex_Gaynorfijal: http://imgur.com/Vkxtc20:11
fijaloh nice20:12
fijalI haven't seen that20:12
Alex_Gaynoryou want the logfile + script?20:13
kenaan12fijal jitviewer 11e9485a7845a0 15/bin/jitviewer.py: slightly better - display loops at least20:13
fijalAlex_Gaynor: ^^^?20:13
fijalmissing password entry :/20:13
Alex_Gaynorok yes20:14
Alex_Gaynornow I can see the code20:14
Alex_Gaynorshow asm doesn't work btw :)20:14
fijalyes, I know20:15
fijalamong other things not working20:15
kenaan12fijal default 1194e9969b5f00 15/pypy/rlib/jit.py: Meh, at least raise error in case there is a typo20:17
lincolnn (~lincolnn0@cpe-98-154-215-147.socal.res.rr.com) joined #pypy.20:18
kenaan12mattip numpypy-axisops 11e5b246bae93a 15/pypy/module/micronumpy/: add tests for numpypy.reduce20:18
kenaan12mattip numpypy-axisops 11da091f2c5c7d 15/pypy/module/micronumpy/: checkpoint20:18
kenaan12mattip numpypy-axisops 11b12a872f0961 15/pypy/module/micronumpy/: rework AxisIterator, make single tests pass20:18
kenaan12mattip numpypy-axisops 11cd8f7d57fe3e 15/pypy/module/micronumpy/signature.py: why do default args cause problems?20:18
kenaan12mattip numpypy-axisops 11a1810346784a 15/pypy/module/micronumpy/: iterators, frames in place, but still not correct20:18
fijalpffff20:19
fijalAlex_Gaynor: of course different drivers can have different params20:20
fijalexcept noone uses this feature20:20
Alex_Gaynorfijal: isn't there now global defaults or such?20:20
fijalwell, yes20:20
fijalif you pass None as a driver, it works that way20:20
Alex_Gaynorfijal: and doesn't --jit change those?20:20
fijalbut you still *can* call separate drivers (from RPython)20:20
fijalat least in theory20:20
fijalshould I just read params from the first one then?20:21
Alex_Gaynorfijal: yes, sounds good enough for me20:21
fijalAlex_Gaynor: I wonder if I can't just store the global param20:23
fijaland be happy20:23
fijalinstead of *really* reading it from JIT20:23
Alex_Gaynorfijal: well, i have a table for it on the page now: http://i.imgur.com/rWvZL.png :)20:29
fijalk20:32
Alex_Gaynorfijal: random note BaseArray.order is annotated as a str, not a char20:35
Alex_Gaynorfijal: does http://paste.pocoo.org/show/526586/ look like the fix?20:35
Rhy0lite (dje@nat/ibm/x-bivwrfbkvkqxighf) left irc: Quit: Leaving20:44
rguillebert (~rguillebe@2a01:e34:eea7:c690:21f:c6ff:fe12:4dee) left irc: Remote host closed the connection20:53
rguillebert (~rguillebe@2a01:e34:eea7:c690:21f:c6ff:fe12:4dee) joined #pypy.20:56
fijalAlex_Gaynor: probably :)21:01
fijalyou never know though21:01
fijalAlex_Gaynor: so, I don't think the JIT params are that interesting21:01
fijalwhy would they be?21:01
fijalGC are though21:01
Alex_Gaynorfijal: if you run with a different trace_limit or something it can be useful21:02
kennethr_ (~kennethre@c-24-127-96-129.hsd1.va.comcast.net) joined #pypy.21:02
fijalyes, but then you know21:02
Alex_Gaynorfijal: ok but I say I run the same script with 2 different trace_limits to share with people21:02
Alex_Gaynorthat info should be on the page21:02
fijalyeah, maybe21:02
fijalI'm not 100% sure, but let's see21:02
Alex_Gaynorremember the idea is this is a hosted tool, not just something you run on your own machine21:03
fijalright21:08
fijalwhat does it differ?21:08
fijalgeez, I can't speak english today21:08
fijalhow does it make a difference?21:08
Alex_Gaynorfijal: because if I use it to link to someone it should have as much info as possible21:09
fijalpffff21:16
fijalAlex_Gaynor: annoying :/21:16
Alex_Gaynorfijal: what is?21:16
fijalgetting the parameters21:16
Alex_Gaynorreally? :(21:17
Alex_Gaynorfijal: well, it's not critical ATM, still in the mockup stage21:17
Alex_Gaynornow I'm trying to figure out what the actual traceviewer should look like21:17
fijalyeah, I know21:18
fijalbut there is a lot of work to be done on the pypy side and this is one of those21:18
fijalparsing of traces has to be revamped21:18
fijalhow to get a list of traces as well21:18
fijaletc. etc.21:18
nettok (~quassel@190.143.183.5) left irc: Remote host closed the connection21:19
fijalwould be a useful thing to have21:21
Trundle (~andy@python/site-packages/trundle) left irc: Ping timeout: 252 seconds21:22
fijalAlex_Gaynor: for what is worth, set_param can't be used in a jitted loop21:27
fijal(apparently)21:27
nedbat (~nedbat@python/psf/nedbat) left irc: Ping timeout: 255 seconds21:27
fijaloh noes21:29
fijalAlex_Gaynor: I give up, I'll just keep a copy of whatever was set from interp_jit21:29
fijalreading this stuff directly from jit is a mess :/21:30
fijalbut than someone at some point will fail to add some functionality :/21:30
fijalok, I give up21:30
Trundle (~andy@python/site-packages/trundle) joined #pypy.21:30
fijalAlex_Gaynor: I'll go to sleep maybe instead21:30
fijaland will try to organize traces in a nicer way21:31
fijaland write some decent tests that run pypy-c21:31
fijalNIGHTLY21:31
fijaland make sure jitviewer agrees21:31
[mat^2] (mathias@87.55.0.198) joined #pypy.21:50
mat^2 (mathias@87.55.0.198) left irc: Read error: Connection reset by peer21:51
mimi_ (~mimi@97-88-28-179.dhcp.roch.mn.charter.com) left irc: Ping timeout: 248 seconds21:57
nedbat (~nedbat@python/psf/nedbat) joined #pypy.21:58
ShadowE989 (~ShadowE98@74-47-218-61.dr02.rmny.wv.frontiernet.net) left irc: Ping timeout: 240 seconds22:12
mimi_ (~mimi@97-88-28-179.dhcp.roch.mn.charter.com) joined #pypy.22:12
Alex_Gaynorfijal: ok, good night then22:15
`fox` (~fox@host139-108-dynamic.245-95-r.retail.telecomitalia.it) left irc: Ping timeout: 240 seconds22:22
ShadowE989 (~ShadowE98@74-47-218-61.dr02.rmny.wv.frontiernet.net) joined #pypy.22:26
bbot2Started: 15http://buildbot.pypy.org/builders/own-macosx-x86-32/builds/75822:30
bbot2Started: 15http://buildbot.pypy.org/builders/jit-benchmark-linux-x86-64/builds/18622:30
bbot2Started: 15http://buildbot.pypy.org/builders/jit-benchmark-linux-x86-32/builds/99522:30
bbot2Started: 15http://buildbot.pypy.org/builders/jit-benchmark-linux-x86-64-2/builds/1422:30
kkris (~kris@188-22-115-215.adsl.highway.telekom.at) left irc: Quit: Leaving.22:35
EnCuKou (~encukou@ip-94-113-220-25.net.upcbroadband.cz) left irc: Remote host closed the connection22:42
nedbat (~nedbat@python/psf/nedbat) left irc: Ping timeout: 276 seconds22:52
Turicas (~alvaro@mail.justen.eng.br) joined #pypy.23:21
antocuni (~antocuni@host117-71-dynamic.58-82-r.retail.telecomitalia.it) joined #pypy.23:37
nickik (~nick@2001:67c:20a1:1018:226:9eff:feef:8369) left irc: Quit: Verlassend23:37
bfirsh (u1308@gateway/web/irccloud.com/x-ngyvtrttxpaxucvt) joined #pypy.23:41
nedbat (~nedbat@python/psf/nedbat) joined #pypy.23:55
Turicas (~alvaro@mail.justen.eng.br) left irc: Ping timeout: 240 seconds23:55
--- Wed Dec 28 201100:00

Generated by irclog2html.py 2.9.2 by Marius Gedminas - find it at mg.pov.lt!