Saturday, 2012-11-10

*** jtran has quit IRC00:01
*** edygarcia has joined #openstack-dev00:11
*** dani4571 has quit IRC00:14
*** mrodden has quit IRC00:15
*** Dr_Who has quit IRC00:17
*** kobier has quit IRC00:19
*** annashen has quit IRC00:20
*** jcmartin has joined #openstack-dev00:26
*** datsun180b has quit IRC00:29
*** imsplitbit has quit IRC00:29
*** kgriffs has quit IRC00:30
*** dims has quit IRC00:30
*** edygarcia has quit IRC00:32
*** andrewbogott_afk is now known as andrewbogott00:34
*** roge has joined #openstack-dev00:36
*** markwash has quit IRC00:37
*** metral_ has joined #openstack-dev00:39
*** gyee has quit IRC00:41
*** littleidea has quit IRC00:42
*** metral has quit IRC00:43
*** metral_ is now known as metral00:43
*** littleidea has joined #openstack-dev00:43
*** dims has joined #openstack-dev00:43
*** metral_ has joined #openstack-dev00:45
*** annashen has joined #openstack-dev00:49
*** metral has quit IRC00:49
*** metral_ is now known as metral00:49
*** arata has quit IRC00:51
*** kbringard has quit IRC00:52
*** annashen has quit IRC00:54
*** jcooley has quit IRC00:56
devanandarussellb: if you're around, could I get you to +2 again? :)00:57
*** jcmartin has quit IRC00:59
*** thingee has quit IRC01:00
*** jcooley has joined #openstack-dev01:01
*** epim has quit IRC01:03
*** pixelbeat has quit IRC01:04
*** anniec72 has joined #openstack-dev01:04
*** networkstatic_ has joined #openstack-dev01:04
*** jakedahn is now known as jakedahn_zz01:07
russellbdevananda: thought i did?01:07
russellbor did you have to updated again?01:07
*** networkstatic has quit IRC01:07
*** networkstatic_ is now known as networkstatic01:07
*** anniec72 has left #openstack-dev01:07
russellbdevananda: nm, i see what happened, and looks like vish and markmc got it already01:11
*** Mandell_ has quit IRC01:12
*** steveleon has quit IRC01:13
*** hemna has quit IRC01:14
*** danwent has quit IRC01:15
*** danwent has joined #openstack-dev01:15
*** arata has joined #openstack-dev01:16
*** jcooley has quit IRC01:19
*** binarysemaphore has joined #openstack-dev01:20
*** roge has quit IRC01:20
*** kbringard has joined #openstack-dev01:22
*** thingee has joined #openstack-dev01:23
*** winston-d has quit IRC01:24
devanandaah, so they did :)01:25
russellbyay01:26
*** thingee_zz has joined #openstack-dev01:29
*** thingee has quit IRC01:29
*** kbringard has quit IRC01:29
*** esp2 has left #openstack-dev01:36
*** littleidea has quit IRC01:40
*** jcooley has joined #openstack-dev01:40
*** jcooley has quit IRC01:41
*** jcooley has joined #openstack-dev01:41
aratadevananda: I've upload the patch again for unittest01:41
*** jcooley has quit IRC01:44
*** s0mik has quit IRC01:44
*** juice has quit IRC01:49
*** ewindisch_ has joined #openstack-dev02:01
*** littleidea has joined #openstack-dev02:02
*** ewindisch has quit IRC02:04
*** ewindisch_ is now known as ewindisch02:04
*** markmc has quit IRC02:06
*** andrewbogott is now known as andrewbogott_afk02:07
*** m4xmr has joined #openstack-dev02:07
*** andrewbogott_afk is now known as andrewbogott02:08
*** reed has joined #openstack-dev02:08
*** e1mer has joined #openstack-dev02:08
*** adjohn has quit IRC02:08
*** m4xmr has quit IRC02:11
*** nati_ueno has left #openstack-dev02:12
*** spiffxp has quit IRC02:12
*** yidclare has quit IRC02:13
*** ewindisch has quit IRC02:14
*** riskable has joined #openstack-dev02:15
riskableCan anyone tell me how to connect to the serial console on my KVM instances using the OpenStack API/tools?  I'm at a loss...  Googling and googling but coming up empty02:16
riskableI wrote a plugin of sorts for OpenStack that adds an 'SSH' tab to the instance detail dashboard.  That works fine (autoconnects for you but you still have to tell it the port and give a username/password).  Now I'd like to make it to work with the serial console--if the hypervisor in question suppots it02:17
russellbyou can get the console output02:17
russellbto interact with it you can use the VNC console02:18
russellbthat's it IIRC02:18
riskableThere's got to be a way to attach to the serial console02:20
riskableI don't want VNC.  With VNC I can't copy & paste and it is all blurry02:20
riskableLimited to 640x480 too.  IT sucks02:21
*** nunosantos has quit IRC02:21
*** jcooley has joined #openstack-dev02:22
*** roadTripper has quit IRC02:23
*** ewindisch has joined #openstack-dev02:24
*** kbringard has joined #openstack-dev02:27
*** littleidea has quit IRC02:28
*** kbringard has quit IRC02:30
devanandajog0: re your last comment, i linked to a bp in the Topic02:31
devanandajog0: am happy to not do that if links in commit is better, though02:31
jog0devananda: ahh, that is good but its also nice to have it in the commit msg as well02:31
devanandak02:32
jog0because once its merged you can't look at the topic02:32
devanandaahhh, right02:32
*** littleidea has joined #openstack-dev02:34
*** jcooley has quit IRC02:38
riskableHow do I send a keystroke to a running instance?  I could've sworn I saw somewhere there was a command or API to do that02:38
openstackgerritA change was merged to openstack/keystone: Use the right subprocess based on os monkeypatch  https://review.openstack.org/1579102:38
*** nati_ueno_2 has quit IRC02:39
jog0devananda: just sent the logs I have been taking about  to your gmail account02:43
*** zbitter has joined #openstack-dev02:48
riskableThis is annoying...  I can send keystrokes to the instance but they go to the VNC console and not the serial console02:51
riskableIf I could send keystrokes to the serial console I could just multiplex the output of the console.log while simultaneously sending it keystrokes02:52
*** zaneb has quit IRC02:52
*** adjohn has joined #openstack-dev02:53
devanandajog0: thanks. taking a look....02:53
*** melwitt has quit IRC02:53
*** dims has quit IRC02:54
jog0devananda:  let me know if you have any  questions, although I am going AFK in a few minutes02:54
devanandajog0: so longest query was 113ms. total query execution time 11s.03:00
devanandajog0: i'd want to reproduce this over a network and tcpdump the traffic to get a clearer picture, honestly.03:01
jog011 seconds? over the time to do the scheduling I counted <2 seconds03:01
*** danwent has quit IRC03:02
jog0devananda: lets talk next week, that may be possible03:02
devanandajog0: cool. i should be around most of the week03:02
devanandaas an aside, SELECT 1 and ROLLBACK account for 12% of total query time.03:03
devanandawhich is why i keep pushing to remove those ;)03:04
openstackgerritA change was merged to openstack/keystone: Rewrite initial migration  https://review.openstack.org/1572403:04
openstackgerritA change was merged to openstack/python-keystoneclient: Make initial structural changes to keystoneclient in preparation to moving auth_token here from keystone.  No functional change should occur from this commit (even though it did refresh a newer copy of openstack.common.setup.py, none of the newer updates   https://review.openstack.org/1569803:04
jog0wow, when I looked at the logs in 20 second period  (cast to compute) 3563 DB Calls were made  in 1.77971 seconds according to mysql-slow.log03:05
jog0and thats the part I don't know how to interpret03:06
devanandai'm not sure where you're quoting that from03:06
*** dims has joined #openstack-dev03:06
jog0I will have to review how I derived my notes.  But thats what I  wrote down03:08
*** salv-orlando has quit IRC03:08
jog0I correlated  the timestamps in the nova-api and nova-scheduler logs to the mysql-slow log and broke down how many c DB calls happened when03:09
devanandaah. maybe i need to trim the log file before digesting it?03:09
jog0devananda: yeah thats what I did.  I gave you the raw data03:09
devanandaor not: Time range: 2012-10-11 21:48:09 to 21:53:2403:09
devanandahm, k03:10
devanandawill look at the other logs03:10
jog0the scheduling was from 21:48:08 to 21:48:3203:10
devanandak03:10
jog0anyway I think its time to call it a night.  have a good weekend03:10
*** jog0 has quit IRC03:11
devanandag'night03:12
*** littleidea has quit IRC03:17
*** arata has quit IRC03:18
*** adjohn has quit IRC03:19
*** littleidea has joined #openstack-dev03:21
*** brucer has joined #openstack-dev03:23
*** novas0x2a|laptop has quit IRC03:24
*** riskable has quit IRC03:30
*** sunilthaha has quit IRC03:33
*** e1mer has quit IRC03:33
*** PaulM has quit IRC03:33
*** darjeeling has quit IRC03:36
*** maurosr has quit IRC03:40
*** ewindisch has quit IRC03:44
*** sunilthaha has joined #openstack-dev03:46
*** sunilthaha has quit IRC03:52
openstackgerritA change was merged to openstack/nova: Get rid of pylint E0203 in filter_scheduler.py  https://review.openstack.org/1580503:59
*** e1mer has joined #openstack-dev04:00
*** sunilthaha has joined #openstack-dev04:05
*** sunilthaha has quit IRC04:12
*** roge has joined #openstack-dev04:18
*** jcooley has joined #openstack-dev04:23
*** jakedahn_zz is now known as jakedahn04:25
*** sunilthaha has joined #openstack-dev04:25
*** jcooley has quit IRC04:29
*** arosen1 has quit IRC04:30
*** roge has quit IRC04:34
*** Mandell has joined #openstack-dev04:38
*** andrewbogott is now known as andrewbogott_afk04:39
*** andrewbogott_afk is now known as andrewbogott04:40
*** jakedahn is now known as jakedahn_zz04:42
*** sunilthaha has quit IRC04:47
*** andrewbogott is now known as andrewbogott_afk04:47
*** aeperezt has quit IRC04:47
*** darjeeling has joined #openstack-dev04:50
*** eglynn_ has joined #openstack-dev04:51
*** sunilthaha has joined #openstack-dev04:51
*** wenjianhn has joined #openstack-dev04:53
*** troytoman-away has quit IRC04:53
*** troytoman-away has joined #openstack-dev04:54
*** eglynn has quit IRC04:54
*** troytoman-away is now known as troytoman04:55
*** ewindisch has joined #openstack-dev04:56
*** harlowja has quit IRC04:57
*** Mandell has quit IRC04:57
*** sunilthaha has quit IRC04:58
*** digantya has quit IRC04:58
*** anniec72 has joined #openstack-dev05:06
*** anniec72 has quit IRC05:12
*** adjohn has joined #openstack-dev05:15
*** sunilthaha has joined #openstack-dev05:15
*** kbringard has joined #openstack-dev05:28
*** kbringard has quit IRC05:32
*** wenjianhn has quit IRC05:32
*** anniec72 has joined #openstack-dev05:35
*** renier_ has quit IRC05:35
*** renier has joined #openstack-dev05:36
*** troytoman is now known as troytoman-away05:37
*** koolhead17 has joined #openstack-dev05:37
*** renier has quit IRC05:41
*** anniec72 has quit IRC05:45
*** anniec72 has joined #openstack-dev05:47
*** ewindisch has quit IRC05:47
*** winston-d has joined #openstack-dev05:52
*** stevebake has joined #openstack-dev05:56
*** anniec72 has quit IRC05:57
*** aravindp has quit IRC06:19
koolhead17hi all06:22
*** zaitcev has quit IRC06:27
*** esp1 has joined #openstack-dev06:34
*** esp1 has left #openstack-dev06:34
*** s0mik has joined #openstack-dev06:35
*** s0mik has quit IRC06:38
*** darjeeling has quit IRC06:44
*** littleidea has quit IRC06:45
*** jcru has joined #openstack-dev06:46
*** sstent has quit IRC06:46
*** sstent has joined #openstack-dev06:49
*** thingee_zz has quit IRC06:57
*** winston-d has quit IRC07:15
*** darjeeling has joined #openstack-dev07:18
*** markmc has joined #openstack-dev07:22
*** kbringard has joined #openstack-dev07:28
*** kbringard has quit IRC07:33
*** binarysemaphore has quit IRC07:33
*** darjeeling has quit IRC07:38
*** armaan has joined #openstack-dev07:41
*** armaan has left #openstack-dev07:41
*** adjohn has joined #openstack-dev07:44
*** darjeeling has joined #openstack-dev07:45
*** darjeeli_ has joined #openstack-dev07:46
openstackgerritA change was merged to openstack/nova: Updated scheduler and compute for multiple capabilities.  https://review.openstack.org/1392007:49
*** darjeeling has quit IRC07:49
*** saurabh has quit IRC07:50
*** SlickNik has joined #openstack-dev07:54
*** markmc has quit IRC07:55
*** adjohn has quit IRC07:59
*** mnewby has quit IRC07:59
*** adjohn has joined #openstack-dev08:00
*** darjeeli_ has quit IRC08:01
*** sstent has quit IRC08:09
*** Shrews has quit IRC08:09
*** Vek has quit IRC08:09
*** cmagina has quit IRC08:09
*** iccha has quit IRC08:09
*** dhubbard has quit IRC08:09
*** sc68cal has quit IRC08:09
*** pvo has quit IRC08:09
*** jesusaurus has quit IRC08:09
*** Shrews has joined #openstack-dev08:11
*** Vek has joined #openstack-dev08:11
*** cmagina has joined #openstack-dev08:11
*** iccha has joined #openstack-dev08:11
*** dhubbard has joined #openstack-dev08:11
*** sc68cal has joined #openstack-dev08:11
*** pvo has joined #openstack-dev08:11
*** jesusaurus has joined #openstack-dev08:11
*** morgan.freenode.net sets mode: +v pvo08:11
*** e1mer has quit IRC08:13
*** adjohn has quit IRC08:13
*** renier has joined #openstack-dev08:15
*** FARiN has joined #openstack-dev08:16
*** e1mer has joined #openstack-dev08:19
*** renier has quit IRC08:19
*** sstent has joined #openstack-dev08:21
*** e1mer has quit IRC08:21
*** alobbs has joined #openstack-dev08:28
*** thingee has joined #openstack-dev08:34
*** thingee has quit IRC08:45
*** darjeeling has joined #openstack-dev08:46
*** rbasak has quit IRC08:54
*** SlickNik has quit IRC08:54
*** apevec has joined #openstack-dev09:13
*** apevec has joined #openstack-dev09:13
*** davidhadas has quit IRC09:14
*** davidha has joined #openstack-dev09:14
*** darjeeling has quit IRC09:22
*** markwash has joined #openstack-dev09:23
*** halfss has joined #openstack-dev09:24
*** s0mik has joined #openstack-dev09:39
*** e1mer has joined #openstack-dev09:47
*** Hien has quit IRC09:49
*** brucer has quit IRC09:51
*** brucer has joined #openstack-dev09:52
*** s0mik has quit IRC09:52
*** s0mik has joined #openstack-dev10:03
*** s0mik has quit IRC10:04
*** Hien has joined #openstack-dev10:09
*** markwash has quit IRC10:19
*** armaan has joined #openstack-dev10:22
*** armaan has left #openstack-dev10:25
*** kbringard has joined #openstack-dev10:29
*** alexpilotti has quit IRC10:31
*** kbringard has quit IRC10:34
*** Dorogs has joined #openstack-dev10:51
*** ogelbukh has quit IRC10:53
*** yjiang5 has quit IRC10:53
*** salv-orlando has joined #openstack-dev10:55
*** ogelbukh has joined #openstack-dev10:59
*** almaisan-away is now known as al-maisan11:00
*** yjiang5 has joined #openstack-dev11:01
*** al-maisan is now known as almaisan-away11:01
*** apevec has quit IRC11:06
*** darjeeli_ has joined #openstack-dev11:09
*** sunilthaha has quit IRC11:14
*** yolanda has joined #openstack-dev11:17
yolandahi, i'm a volunteer from Canonical, and i need my CLA to be approved in order to send a review for nova, can i get some help?11:17
*** rbasak has joined #openstack-dev11:18
*** Dorogs has quit IRC11:23
*** markwash has joined #openstack-dev11:25
*** yjiang5_ has joined #openstack-dev11:26
*** markwash has quit IRC11:29
*** kbringard has joined #openstack-dev11:30
*** kbringard has quit IRC11:34
*** armaan has joined #openstack-dev11:35
*** armaan has left #openstack-dev11:37
*** yjiang5 is now known as yjiang5_away11:51
*** renier has joined #openstack-dev11:51
*** Dorogs has joined #openstack-dev11:53
*** renier has quit IRC11:55
*** yjiang5_away has quit IRC12:01
*** Dorogs has quit IRC12:04
*** danwent has joined #openstack-dev12:05
*** roge has joined #openstack-dev12:07
*** salv-orlando has quit IRC12:08
*** roge has quit IRC12:11
*** roge has joined #openstack-dev12:12
*** pixelbeat has joined #openstack-dev12:15
*** dhellmann_ has joined #openstack-dev12:18
*** spn has quit IRC12:18
*** spn has joined #openstack-dev12:19
*** dhellmann has quit IRC12:20
*** dhellmann_ is now known as dhellmann12:20
*** kbringard has joined #openstack-dev12:30
*** kbringard has quit IRC12:34
openstackgerritA change was merged to openstack/quantum: Add QUANTUM_ prefix for env used by quantum-debug  https://review.openstack.org/1578012:37
openstackgerritA change was merged to openstack/quantum: Removing unsed code for Cisco Quantum Plugin V1  https://review.openstack.org/1571212:37
*** darjeeli_ has quit IRC12:43
*** renier has joined #openstack-dev12:45
*** yolanda has quit IRC12:49
*** renier has quit IRC12:50
*** egallen has joined #openstack-dev13:09
*** egallen has quit IRC13:10
*** yjiang5 has joined #openstack-dev13:14
*** yjiang5 has quit IRC13:16
*** egallen has joined #openstack-dev13:17
*** roge has quit IRC13:21
*** yjiang5_ has quit IRC13:29
*** yjiang5 has joined #openstack-dev13:29
*** yjiang5 is now known as yjiang5_away13:35
*** test_123 has joined #openstack-dev13:39
*** renier has joined #openstack-dev13:40
*** test_123 has quit IRC13:41
*** adjohn has joined #openstack-dev13:42
*** renier has quit IRC13:44
*** alexpilotti has joined #openstack-dev13:45
*** alexpilotti has quit IRC13:57
*** dhellmann has quit IRC13:58
*** Dr_Who has joined #openstack-dev14:05
*** Dr_Who has joined #openstack-dev14:05
*** renier has joined #openstack-dev14:12
*** Dr_Who has quit IRC14:20
*** roge has joined #openstack-dev14:26
*** roge is now known as [4n4rk]roge14:46
*** [4n4rk]roge has left #openstack-dev14:47
*** pixelbeat has quit IRC14:48
*** salv-orlando has joined #openstack-dev14:56
*** benner has quit IRC14:57
*** benner has joined #openstack-dev14:57
*** salv-orlando has quit IRC15:01
*** littleidea has joined #openstack-dev15:16
*** armaan has joined #openstack-dev15:18
*** darjeeling has joined #openstack-dev15:21
*** anniec72 has joined #openstack-dev15:22
*** egallen has quit IRC15:22
*** egallen has joined #openstack-dev15:22
*** littleidea has left #openstack-dev15:23
*** gongysh has quit IRC15:34
*** armaan has left #openstack-dev15:37
*** halfss has quit IRC15:51
*** davidkranz has quit IRC16:02
*** e1mer has quit IRC16:07
*** alexpilotti has joined #openstack-dev16:13
alexpilottismoser mikal: hi!16:17
*** oneiroi has quit IRC16:18
*** oneiroi has joined #openstack-dev16:21
*** timello has quit IRC16:26
*** littleidea has joined #openstack-dev16:31
*** littleidea has quit IRC16:32
*** littleidea has joined #openstack-dev16:33
*** littleidea has quit IRC16:36
*** salv-orlando has joined #openstack-dev16:37
*** salv-orlando has quit IRC16:38
*** salv-orlando has joined #openstack-dev16:39
*** littleidea has joined #openstack-dev16:47
*** sacharya has joined #openstack-dev16:51
*** dansmith has quit IRC16:51
*** littleidea has quit IRC16:52
*** littleidea has joined #openstack-dev16:54
*** blamar has joined #openstack-dev16:58
*** danwent has quit IRC17:01
*** ewindisch has joined #openstack-dev17:01
*** dansmith has joined #openstack-dev17:05
*** ewindisch has quit IRC17:06
*** aeperezt has joined #openstack-dev17:06
*** ewindisch has joined #openstack-dev17:08
*** aeperezt has quit IRC17:10
*** markwash has joined #openstack-dev17:22
*** samkottler|bbl is now known as samkottler17:22
*** sacharya1 has joined #openstack-dev17:23
*** sacharya has quit IRC17:26
*** ewindisch has quit IRC17:27
*** colinmcnamara has joined #openstack-dev17:28
*** kbringard has joined #openstack-dev17:32
*** danwent has joined #openstack-dev17:33
*** blamar has quit IRC17:34
*** kbringard has quit IRC17:36
*** vds has quit IRC17:40
*** rods has quit IRC17:40
*** colinmcnamara has quit IRC17:40
*** thingee has joined #openstack-dev17:46
*** danwent has quit IRC17:49
*** jcooley has joined #openstack-dev17:52
*** adalbas has quit IRC17:55
*** danwent has joined #openstack-dev17:56
*** davidkranz has joined #openstack-dev18:00
*** danishman has joined #openstack-dev18:01
*** renier has quit IRC18:03
*** renier has joined #openstack-dev18:04
*** davidkranz has quit IRC18:04
*** davidkranz has joined #openstack-dev18:04
*** renier has quit IRC18:08
*** renier has joined #openstack-dev18:10
*** adalbas has joined #openstack-dev18:12
*** danwent has quit IRC18:15
*** littleidea has quit IRC18:17
*** anniec72 has quit IRC18:18
*** jcooley has quit IRC18:18
*** anniec has joined #openstack-dev18:19
*** riskable has joined #openstack-dev18:22
*** anniec has quit IRC18:24
*** littleidea has joined #openstack-dev18:24
*** anniec has joined #openstack-dev18:25
*** timello has joined #openstack-dev18:27
*** heckj has joined #openstack-dev18:29
*** kbringard has joined #openstack-dev18:32
*** markwash has quit IRC18:33
*** kbringard has quit IRC18:37
*** markwash has joined #openstack-dev18:41
*** egallen has quit IRC18:45
*** egallen has joined #openstack-dev18:46
*** Dr_Who has joined #openstack-dev18:48
*** Dr_Who has joined #openstack-dev18:48
*** ijw1 has joined #openstack-dev18:48
*** heckj has quit IRC18:51
*** markwash has quit IRC18:51
*** ijw has quit IRC18:51
*** j_king has quit IRC18:52
*** Dorogs has joined #openstack-dev18:52
*** Dorogs has quit IRC18:54
*** littleidea has left #openstack-dev18:55
*** salv-orlando has quit IRC18:59
*** j_king has joined #openstack-dev19:04
*** Dr_Who has quit IRC19:04
*** clarkb has quit IRC19:06
*** clarkb has joined #openstack-dev19:06
*** renier has quit IRC19:16
*** renier has joined #openstack-dev19:17
*** egallen has quit IRC19:20
*** renier has quit IRC19:21
*** pixelbeat has joined #openstack-dev19:29
*** koolhead17 has quit IRC19:31
*** openstack_newbe has quit IRC19:32
*** openstack_newbe has joined #openstack-dev19:35
*** pixelbeat has quit IRC19:36
*** egallen has joined #openstack-dev19:37
*** markwash has joined #openstack-dev19:39
*** koolhead17 has joined #openstack-dev19:44
*** timjr has joined #openstack-dev19:47
*** egallen has quit IRC19:49
*** egallen has joined #openstack-dev19:50
*** aeperezt has joined #openstack-dev19:50
*** egallen has quit IRC19:51
*** Dr_Who has joined #openstack-dev19:52
*** Dr_Who has joined #openstack-dev19:52
*** egallen has joined #openstack-dev19:54
*** egallen has quit IRC19:55
*** markwash has quit IRC19:58
*** utlemming has joined #openstack-dev19:58
timjrso, one of the unit tests failed for a glance patch that I submitted for review...  but it passes when I run it20:05
timjrwould it be really rude if I submitted updated versions of the patch with prints in that unit test to figure out what went wrong?20:06
*** egallen has joined #openstack-dev20:08
clarkbtimjr: running `tox -epy27` locally does not reproduce?20:12
timjrwell, I used run_tests.sh... haven't tried that command20:13
clarkbtox -epy27 is the command Jenkins is running when that test fails20:13
*** Dr_Who has quit IRC20:14
timjrI basically changed comments.  And the same test passed in python 2.6... I think it's an intermittent failure20:15
clarkbdefinitely could be, was just curious if doing what Jenkins did resulted in failure20:16
clarkbif it is reproducable that way adding print statement to pushed code wouldn't be necessary. If it is intermittent, adding prints may not help either20:17
timjrindeed20:17
*** novas0x2a|laptop has joined #openstack-dev20:18
timjrunfortunately, I don't think RHEL has been so kind as to bestow python 2.7 on me just yet20:20
*** anniec has quit IRC20:31
*** ijw11 has joined #openstack-dev20:37
*** ijw1 has quit IRC20:40
timjrlooks like you can just add a comment that says "recheck" to get it to rebuild20:40
*** littleidea has joined #openstack-dev20:41
*** Tross has quit IRC20:43
*** alexpilotti has quit IRC20:44
*** aeperezt has quit IRC20:45
*** PaulM has joined #openstack-dev20:52
*** Tross has joined #openstack-dev20:55
*** renier has joined #openstack-dev20:59
*** blamar has joined #openstack-dev21:00
*** danishman has quit IRC21:01
*** renier has quit IRC21:04
*** cmagina has quit IRC21:13
fungitimjr: to retry the tests, yes21:38
fungitimjr: if it is intermittent/nondeterministic or if it was failing due to some other already merged bug which has since been fixed, then that's what you want21:39
timjrwell, I really only changed comments and removed an unused class... so there aren't a lot of really interesting failure modes you'd expect from that21:40
*** sandywalsh has joined #openstack-dev21:40
fungitimjr: agreed. on occasion we *do* see dependent tests (a bug in their own right honestly) which start failing due to seemingly trivial changes/removals of other bits21:41
timjrfailed in a different place this time... so flaky :(21:41
fungiugh. that really, really shouldn't happen21:42
fungiwhat's the url to the change (or the change number)?21:42
timjrhttps://review.openstack.org/#/c/15801/21:42
fungiyeah, something's definitelty not right. we've got failures there exposed in three different suites at different times (py26, py27 and tempest)21:43
fungiclarkb: ^ thoughts?21:44
timjrthe tempest failures (two different times) both look like some kind of timeout21:44
clarkbthe tempest failures are cinder failing which it seems to do semi frequently21:44
timjrmaybe the load is really high someplace...21:44
timjroh21:44
clarkbdespite our hitting it with a hammer21:44
clarkbI think the sleep length increase is probably a sane thing to do for the first py27 failure21:45
clarkbhowever, waiting in that manner is always a potential non deterministic failure point21:45
timjrindeed21:46
timjrthe thing it's waiting for is:21:46
timjr            write_pid_file(pid_file, pid)21:46
timjr            await_child(pid, await_time)21:46
timjrI don't think write_pid_file should take two seconds unless the system is like really buried21:46
clarkbwell these are cloud VMs21:47
fungiand one of the failures seemed to be a connection refusal21:47
timjrI'll do another recheck while we're thinkin' about it... maybe the nth time's the charm21:48
fungiwe normally don't see clusters of random failures with this kind of frequency, do we? it does make me wonder if $provider is having some kind of issue21:48
clarkbanother race condition? is that test sleeping to wait for the thing listening on the socket to come up?21:48
*** gongysh has joined #openstack-dev21:48
timjrlemmie look21:49
*** sandywalsh has quit IRC21:49
clarkbfungi: jenkins doesn't load balance tests well and consecutive runs of the same test tend to end up on the same slave21:50
timjryeah, that test might want to wait for the port to become available, I'll add that21:50
clarkbthe test should be the only thing running on the slave at the time, but the VM is sharing resources with the other hosts on the hypervisor21:50
timjrlooks like it's using a non-empty logfile as a proxy for that21:50
fungiclarkb: got it. that does then increase the visibility for a flaky slave at least, if to the unfortunate detriment of a particular change21:52
*** utlemming has quit IRC21:53
*** utlemming has joined #openstack-dev21:53
timjra good minimum performance SLA is probably an interesting problem21:53
timjrnah, the whole glance API test uses that same idiom of start_servers() followed immediately by a GET21:55
timjrstart_servers does wait for the port to become available21:56
*** sacharya1 has quit IRC21:58
*** Tross has quit IRC21:58
clarkbjgriffith: if you are around http://logs.openstack.org/15801/2/check/gate-tempest-devstack-vm/18059/ and http://logs.openstack.org/15801/1/check/gate-tempest-devstack-vm/18017/ are worth a look at22:02
mordredfungi, clarkb: we've discussed cycling unittest slaves more frequently22:04
mordredclarkb: it probably wouldn't be terribly much of a change to make the devstack-gate pool management also manage some unittest slaves in a second pool22:05
*** e1mer has joined #openstack-dev22:05
mordredand then perhaps expire/re-cycle them all after a few hours?22:05
mordredhrm. wrong channel...22:05
clarkbjgriffith: for the first one there are a couple tracebacks in n-api.txt22:06
clarkbjgriffith: looks like the same sort of traceback in the second22:07
*** annashen has joined #openstack-dev22:08
timjryay, great success22:19
*** sunil has joined #openstack-dev22:19
timjrit did seem to go much faster than the failures.  the tempest gate thing took 20 minutes instead of 5022:19
*** sunil has quit IRC22:20
*** sunil has joined #openstack-dev22:21
fungican't say that's making me feel any better. i just have to wonder if it lets through as many bugs via false negatives and it blocks on false positives22:22
fungis/and/as/22:23
*** sunil has quit IRC22:23
*** sthaha has joined #openstack-dev22:23
timjrif it's just caused by extreme slowness of the underlying system, I would be really surprised if that led to spurious test passes22:24
timjrif it continues to be flaky though, things could degenerate fast as people keep re-running the jobs22:24
timjrit also makes it a little tough to detect performance regressions, if the tests get a random amount of system resources22:25
fungiyeah, agreed. makes me uneasy regardless--it's not necessarily rational of me22:25
timjrordinarily, a test run taking twice as long would be a cause for concern...22:25
clarkbtimjr: I don't think the change in tempest run time is an issue22:26
clarkbifyou look at the timestamps all of that extra time is waiting for cinder to time out22:26
timjroh, ok22:27
*** sandywalsh has joined #openstack-dev22:32
*** blamar has quit IRC22:42
*** sandywalsh has quit IRC22:44
*** koolhead17 has quit IRC22:56
*** adjohn has joined #openstack-dev22:58
*** Tross has joined #openstack-dev22:59
*** reed has quit IRC23:00
*** Tross has quit IRC23:02
*** davidhadas has joined #openstack-dev23:02
*** Dr_Who has joined #openstack-dev23:06
*** Dr_Who has joined #openstack-dev23:06
*** novas0x2a|laptop has quit IRC23:11
*** novas0x2a|laptop has joined #openstack-dev23:11
*** davidha has quit IRC23:13
*** egallen has quit IRC23:32
*** renier has joined #openstack-dev23:41
*** littleidea has quit IRC23:45
*** renier has quit IRC23:45
*** Tross has joined #openstack-dev23:54
*** littleidea has joined #openstack-dev23:55

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!