| fungi | we could offer patches to (or fork) the third party things | 00:00 |
|---|---|---|
| clarkb | fungi: if I was a puppet upstream I wouldn't want these patches | 00:00 |
| clarkb | because they don't quite fit what puppet exists for | 00:00 |
| clarkb | (rightly or wrongly) | 00:00 |
| *** spiffxp has joined #openstack-infra | 00:00 | |
| *** yamahata has quit IRC | 00:00 | |
| *** gokrokve has quit IRC | 00:01 | |
| *** matsuhashi has joined #openstack-infra | 00:03 | |
| fungi | i almost want to say that for the nodepool+dib situation, maybe getting rid of puppet is the best path forward long-term | 00:03 |
| nibalizer | so no more puppet apply on those nodes? | 00:03 |
| lifeless | I dunno, it seems like a generally useful distinction for puppet to be able to draw | 00:04 |
| lifeless | as things like docker become more common | 00:04 |
| fungi | we're still operating there on a vestige of our original setup with long-lived jenkins slaves, but these systems don't really have nearly as much need for puppetry, nor overlap with what we're using puppet for in other places on our permanent servers | 00:04 |
| *** stevebaker has quit IRC | 00:04 | |
| *** stevebaker has joined #openstack-infra | 00:05 | |
| * adrian_otto ears perk up | 00:05 | |
| fungi | for nodepool, with all those systems being short-lived and single-use, we only ever run puppet once on the base image to configure it... | 00:05 |
| *** starmer has joined #openstack-infra | 00:06 | |
| adrian_otto | fungi, sounds to me like a Dockerfile is the right tool for that sort of a job. | 00:06 |
| fungi | and then never again, because there's little reason to update those systems further through their short (minutes to hours) lifespan | 00:06 |
| adrian_otto | rather than a puppet config | 00:07 |
| clarkb | adrian_otto: we are using dib | 00:07 |
| clarkb | fungi: the place I disagree is we need a lot of the same foundation stuff. iptables, ntp, cronspam, etc etc etc | 00:08 |
| adrian_otto | so roughly equivalent from a functionality perspective | 00:08 |
| clarkb | fungi: and writing that twice is silly | 00:08 |
| clarkb | and for long lived servers you definitely want something like puppet | 00:08 |
| zaro | jhesketh: yes, just tox. running on trusty. | 00:08 |
| fungi | we've already written a lot of it twice (see devstack) | 00:08 |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: Add support for disk-image-builder in nodepool https://review.openstack.org/88479 | 00:09 |
| clarkb | fungi: devstack doesn't do half the things we need | 00:09 |
| jhesketh | zaro: yeah, I've managed to reproduce locally so I should be able to fix it shortly... basically that was an experimental test anyway (playing around with JJB integration) so you should be able to disable it safely | 00:09 |
| clarkb | in any case latest patchset works except for this one issue | 00:09 |
| fungi | true. which is why i said | 00:09 |
| fungi | "a lot" | 00:10 |
| *** masayuki_ has joined #openstack-infra | 00:10 | |
| *** lcheng_ has quit IRC | 00:10 | |
| zaro | jhesketh: can you tell me what is loading plugin_config? | 00:10 |
| jhesketh | zaro: the reason it doesn't work is because you need infra/config slave scripts installed (which Jenkins nodes have) | 00:10 |
| *** starmer has quit IRC | 00:10 | |
| *** masayuki_ has quit IRC | 00:10 | |
| *** masayukig has quit IRC | 00:10 | |
| jhesketh | zaro: that'd be worker_server from memory | 00:11 |
| jhesketh | zaro: do you mind if I ask what you're doing :-) | 00:11 |
| *** ayoung has quit IRC | 00:11 | |
| zaro | jhesketh: i'm looking at seeing if we can decouple the xml generation so it will be easier for other apps to consume jjb outputs. | 00:12 |
| sdague | clarkb: if you split up the policy betweek prepare and run classes | 00:12 |
| zaro | jhesketh: using turbo hipster use case. | 00:12 |
| sdague | then you could do it all with puppet, it's just about building that convention in policy | 00:13 |
| sdague | packstack actually did some completely screwy things there | 00:13 |
| fungi | sdague: agreed, but a number of the puppet modules we're using from the forge currently lack that separation | 00:13 |
| jhesketh | zaro: ah awesome | 00:13 |
| clarkb | sdague: yes the problem is third party libs | 00:14 |
| clarkb | you can even have facter figure out if you are in a chroot for you | 00:14 |
| clarkb | (just have your dib script export FACTER_chroot=true | 00:14 |
| fungi | yeah, it would really mostly be wrapping tons of exec calls in conditionals for that | 00:14 |
| sdague | clarkb: sure | 00:14 |
| clarkb | fungi: to start its just service[rsyslog] and service[iptables-persistent] | 00:15 |
| jhesketh | zaro: so you might like that test to run.. in which case, you could install http://git.openstack.org/cgit/openstack-infra/config/tree/modules/openstack_project/files/slave_scripts into /usr/local/jenkins/slave_scripts/ | 00:15 |
| clarkb | so maybe I start there | 00:15 |
| fungi | clarkb: good point | 00:15 |
| *** ramashri has quit IRC | 00:15 | |
| jogo | pleia2: do you have any code to generate a list of bugs that were marked as in progress and have abandoned patches? | 00:16 |
| sdague | yeh, I went through all this about 10 years ago when writing tool exactly like dib for systemimager :) | 00:16 |
| fungi | clarkb: also, nodepool is back to being mostly idle with a good surplus of ready nodes, so we can probably safely restart it now | 00:16 |
| *** masayukig has joined #openstack-infra | 00:17 | |
| clarkb | fungi: cool I am ready when you are | 00:17 |
| *** penguinRaider has joined #openstack-infra | 00:17 | |
| sdague | I do like the idea of dib actually being able to run a guest for this, then you'd solve a ton of issues | 00:17 |
| fungi | i'll approve your test fix first so we can more reliably test fixes for the bugs we're about to discover | 00:17 |
| *** jerryz has joined #openstack-infra | 00:17 | |
| pleia2 | jogo: no, and not sure off the top of my head whether there is an abandoned patch option | 00:18 |
| pleia2 | jogo: well, I guess any patch? | 00:18 |
| jogo | pleia2: yeah does that exist? | 00:18 |
| clarkb | sdague: right so packer does that | 00:18 |
| jogo | pleia2: show bugs with patches of any state? | 00:18 |
| clarkb | sdague: which is why I made that comment | 00:18 |
| jogo | pleia2: if not I will start dinking on one | 00:19 |
| clarkb | but I am not sure how the image creation happens there if it will insist on talking to glance or what | 00:19 |
| sdague | well you can do it in kvm | 00:20 |
| clarkb | sdague: not nestedwith any performance | 00:20 |
| pleia2 | jogo: not as far as I know, I'd have to look at docs figure out how to add patch finding to https://github.com/pleia2/openstack-infra-scripts/blob/master/infra_bugday.py | 00:20 |
| jogo | pleia2: cool thanks | 00:21 |
| sdague | clarkb: it works in rackspace, so it can't be requiring glance | 00:22 |
| openstackgerrit | A change was merged to openstack-infra/nodepool: Increase watermark sleep in tests for reliability. https://review.openstack.org/98183 | 00:23 |
| *** praneshp has quit IRC | 00:24 | |
| *** starmer_ has joined #openstack-infra | 00:24 | |
| *** jhesketh has quit IRC | 00:25 | |
| sdague | clarkb: yeh, from my basic reading it's basically doing what we're doing in nodepool today | 00:25 |
| sdague | though reading go ... I seriously do not understand why this is the new hotness | 00:26 |
| clarkb | sdague: yup | 00:26 |
| clarkb | sdague: it really isn't anything special | 00:26 |
| clarkb | and the json files are puke | 00:26 |
| sdague | well packer looks cool | 00:26 |
| sdague | I meant 'go' itself | 00:26 |
| clarkb | oh | 00:26 |
| *** sarob_ has joined #openstack-infra | 00:26 | |
| clarkb | so I disagree. go is actually a neat language to program in and packer isn't that awesome :P | 00:27 |
| sdague | heh | 00:27 |
| clarkb | mostly because it does what nodepool does with a terrible config language | 00:27 |
| clarkb | it just happens to support all the things | 00:27 |
| sdague | I have too many battle scars from java type gynastics | 00:27 |
| dstufft | silent by default errors are the worst | 00:28 |
| *** starmer_ has quit IRC | 00:29 | |
| *** sarob has quit IRC | 00:29 | |
| *** wenlock has quit IRC | 00:29 | |
| *** sarob_ has quit IRC | 00:32 | |
| zaro | jhesketh : thanks. end of my day now. will try later | 00:32 |
| sdague | also, apparently something about the get-pip transition impacted the hyperv ci folks, I told them to come here. Hopefully they will. | 00:33 |
| *** starmer has joined #openstack-infra | 00:35 | |
| openstackgerrit | A change was merged to openstack-infra/elastic-recheck: Add query for ceilometer test_ordination_assign timeout bug 1326855 https://review.openstack.org/98160 | 00:36 |
| uvirtbot | Launchpad bug 1326855 in ceilometer ""tests.alarm.test_rpc.TestRPCAlarmPartitionCoordination.test_ordination_assign" race in the gate (dup-of: 1323524)" [Undecided,New] https://launchpad.net/bugs/1326855 | 00:36 |
| uvirtbot | Launchpad bug 1323524 in ceilometer "tests.alarm.test_rpc.TestRPCAlarmPartitionCoordination.test_ordination_allocate failed for timeout" [Medium,Confirmed] https://launchpad.net/bugs/1323524 | 00:36 |
| jogo | this new gerrit rest API is very nice | 00:37 |
| clarkb | there is a lot I like about dib though. the caching stuff is awesome | 00:37 |
| clarkb | and would be a little trickier with kvm | 00:37 |
| sdague | clarkb: honestly, my dib understanding is limitted. I will try to correctly this after the fires die out | 00:38 |
| clarkb | sdague: I am doing my best to make it easy for peopel to do that within the onctext of infra | 00:39 |
| clarkb | fungi: still want to restart that nodepool service? | 00:39 |
| *** starmer has quit IRC | 00:40 | |
| fungi | clarkb: yeah, was just catching up on e-mail. stopping it now | 00:40 |
| clarkb | cool I am here mostly hitting dib with a hammer but yell if you need anything | 00:40 |
| *** jhesketh has joined #openstack-infra | 00:40 | |
| clarkb | sdague: basically with dib you get run parts of install scrpts in a chroot which gets converted to qcow2 | 00:41 |
| clarkb | sdague: so for us one of those steps is run puppet | 00:41 |
| sdague | gotcha | 00:41 |
| fungi | clarkb: as luck would have it, puppet successfully installed newer nodepool such that pip freeze reports the right sha | 00:42 |
| fungi | also, the initscript's graceful stop worked instantly | 00:42 |
| *** aconrad has joined #openstack-infra | 00:42 | |
| clarkb | nice | 00:42 |
| clarkb | https://github.com/puppetlabs/puppetlabs-mysql/blob/master/manifests/server/service.pp so mysql does the right thing ish | 00:42 |
| clarkb | more like it gives you all the knobs | 00:42 |
| fungi | clarkb: what were we cleaning up before starting it again? just set the building nodes to delete? | 00:43 |
| clarkb | now to see how new of a version we need for that | 00:43 |
| Hunner | What knobs do you want taken away? | 00:43 |
| clarkb | Hunner: no we want more knobs I think | 00:43 |
| clarkb | fungi: actually it does that on its won right? | 00:43 |
| clarkb | fungi: I think we needed to check after it came up that nodepool didn't leak a bunch of nodes due to the race that should be fixed | 00:43 |
| Hunner | Moar knobs! http://img370.imageshack.us/img370/9343/28xp9ie0.jpg | 00:44 |
| clarkb | Hunner: we are running puppet in a chroot to build cloud images | 00:44 |
| clarkb | Hunner: and that means we can't have puppet try to start services | 00:44 |
| sdague | it would kind of be cool if you could monkey patch puppet to noop service | 00:45 |
| Hunner | clarkb: Usually that param is called $service_manage | 00:45 |
| fungi | clarkb: fair enough. started nodepool again | 00:45 |
| clarkb | Hunner: so that ensures stopped | 00:45 |
| fungi | watching logs now | 00:45 |
| clarkb | oh wait that is service_ensure | 00:45 |
| Hunner | Yeah, service_ensure => stopped doesn't work in weird things like docker, since it can't even connect to systemd to get the state of the service | 00:46 |
| Hunner | service_manage => false is usually used to case-out the service resource entirely | 00:46 |
| clarkb | right this would probably be a similar situation | 00:46 |
| Hunner | https://github.com/puppetlabs/puppetlabs-mysql#service_manage | 00:46 |
| clarkb | Hunner: hrm so you can't enable => true in that case | 00:46 |
| clarkb | whcih should be doable | 00:47 |
| Hunner | Oh, you want that | 00:47 |
| *** zns has joined #openstack-infra | 00:47 | |
| openstackgerrit | Ian Wienand proposed a change to openstack-infra/config: Promote devstack/f20 job to check job https://review.openstack.org/98280 | 00:47 |
| *** zns has quit IRC | 00:47 | |
| Hunner | Well, I guess you could always service_manage => false and declare your own service resource with enable => true, but that is icky | 00:47 |
| jogo | pleia2: thanks for the link that is a great starting point | 00:47 |
| clarkb | Hunner: it looks like having a service at all will fail | 00:48 |
| *** zns has joined #openstack-infra | 00:48 | |
| clarkb | but I need to do more debugging | 00:48 |
| Hunner | k | 00:48 |
| sdague | Hunner: what we actually want is the service resource to do everything it does , but go noop on the trying to start the service | 00:48 |
| clarkb | ya I just ran puppet with a service resource that does not ensure anything and it fialed | 00:48 |
| *** radez is now known as radez_g0n3 | 00:48 | |
| Hunner | sdague: Service <| title == 'mysql' |> { noop => true } is a hacky hack | 00:49 |
| sdague | Hunner: that would also make it not enable, right | 00:49 |
| Hunner | clarkb: noop mode will still try to `service mysqld status` | 00:49 |
| Hunner | Because noop says "should I start the service?" and then lets you know if it should | 00:49 |
| *** aconrad has quit IRC | 00:49 | |
| clarkb | fungi: how does it look? | 00:50 |
| clarkb | Hunner: noop mode isn't what we want either because that won't enable the srvice | 00:50 |
| openstackgerrit | Ian Wienand proposed a change to openstack-infra/config: Add devstack-f20 to hpcloud https://review.openstack.org/98024 | 00:50 |
| openstackgerrit | Ian Wienand proposed a change to openstack-infra/config: Promote devstack/f20 job to check job https://review.openstack.org/98280 | 00:50 |
| clarkb | we want to assert service enablement but not running state | 00:50 |
| clarkb | they are independent but apparently not in puppet | 00:50 |
| Hunner | sdague: Yep :/. And because all of these complicated edge cases, we have thus far stuck to the middle of the road and said "If you want to do fancy things, service_manage => false and then add your fancy stuff after" | 00:51 |
| *** starmer has joined #openstack-infra | 00:51 | |
| clarkb | Hunner: I think you can't do the fancy stuff with puppe though | 00:51 |
| clarkb | you would have to run an exec | 00:51 |
| sdague | Hunner: what about declaring a mixin and overriding the service providers? | 00:52 |
| clarkb | also this script is set -e and puppet failing on iptables restart didn't fail the whole image build | 00:52 |
| Hunner | clarkb: Because `service { 'mysql': enable => true }` still tries to ensure => running? (I don't know if it does) | 00:52 |
| clarkb | Hunner: yes that is what appears to happen | 00:52 |
| fungi | clarkb: looking fine so far. i'm assuming i won't be able to see the leaking right away | 00:52 |
| clarkb | fungi: ya probably not | 00:52 |
| clarkb | fungi: I think we want to nodepool list | grep hpcloud-b | wc -l and compare to nova list in that region | 00:52 |
| Hunner | clarkb: Then yeah, exec :(. We do use execs from time to time to reload a service rather than restart, or other such things (in postgresql for example) | 00:52 |
| Hunner | sdague: Well... then you're stuck with maintaining code that mucks with non-apis | 00:53 |
| clarkb | I think this is my cue to find beer | 00:53 |
| sdague | Hunner: sure, but our other option is to write out own puppet basically | 00:54 |
| fungi | clarkb: drink yuo up some beer | 00:54 |
| sdague | Hunner: would the puppet community accept spliting up the service class into 2 phases like that? | 00:54 |
| Hunner | sdague: or service_manage => false and add an exec resource | 00:54 |
| sdague | Hunner: that assumes all the policys support that convention | 00:55 |
| *** mfer has joined #openstack-infra | 00:55 | |
| *** starmer has quit IRC | 00:55 | |
| Hunner | sdague: splitting ensure/enable? Probably not, since you'd run into issues with duplicate resources or backwards-incompatible resources | 00:55 |
| sdague | well, basically a no_live | 00:55 |
| sdague | option | 00:55 |
| Hunner | sbadia: Ah, enhancing service providers for embedding in chroots, etc? | 00:56 |
| *** zns has quit IRC | 00:56 | |
| sdague | Hunner: right, that's what dib is | 00:56 |
| fungi | ensure => dry_running (and then we could just sed the manifest?) | 00:56 |
| Hunner | sdague: That might be more feasible, though I don't know if the general market has shifted far enough in that direction to get it past the "80/20" rule without some work :? | 00:56 |
| Hunner | fungi: That looks good. Are there other use cases than chroots that you can think of? | 00:57 |
| oomichi | Alex_Gaynor: hi | 00:58 |
| *** signed8bit has quit IRC | 00:58 | |
| Hunner | (docker and pacemaker are the two main cases that people don't want the service managed) | 00:58 |
| Alex_Gaynor | oomichi: hi | 00:58 |
| fungi | if service had an ensure state which was almost identical to running but doesn't start it... then we could just make it a variable switched from a conditional around a chroot fact | 00:58 |
| oomichi | Alex_Gaynor: about https://review.openstack.org/#/c/98012/ | 00:58 |
| clarkb | I am doing more testing of ^ | 00:58 |
| clarkb | I am going to make this work | 00:59 |
| fungi | Hunner: not off the top of my head, but chroots are... well... a classic use case going well back in unix history | 00:59 |
| oomichi | Alex_Gaynor: cannot we add GPL library into requirements? | 00:59 |
| *** marcoemorais has quit IRC | 00:59 | |
| Hunner | Sorry, I need to get offline now. ttyl :) | 00:59 |
| Alex_Gaynor | oomichi: Per https://wiki.openstack.org/wiki/Requirements and my understanding of the GPL, I don't think so, no. | 00:59 |
| sdague | oomichi: no, not GPL | 00:59 |
| sdague | LGPL is fine | 00:59 |
| fungi | later Hunner! | 00:59 |
| oomichi | Alex_Gaynor, sdague: I got it, thanks. | 01:00 |
| morganfainberg | sdague, ++ was just about to say that | 01:00 |
| clarkb | yall realize that python mysql is gpl right? | 01:00 |
| clarkb | so uh ya | 01:00 |
| morganfainberg | clarkb, shhhh | 01:00 |
| fungi | clarkb: lies. all lies | 01:00 |
| clarkb | I am pretty sure its ok | 01:00 |
| oomichi | Alex_Gaynor, sdague: I will consider the other workaround. | 01:00 |
| clarkb | as long as you aren't distributing the code >_> | 01:00 |
| clarkb | (the gpl bits) | 01:00 |
| morganfainberg | clarkb, GPLv3 might be an issue. | 01:00 |
| sdague | clarkb: isn't it gpl with the funny open source exception | 01:01 |
| sdague | like the rest of mysql | 01:01 |
| clarkb | morganfainberg: actually apache2 iscompat with gplv3 I thought | 01:01 |
| clarkb | its v2 that is more problematic iirc | 01:01 |
| morganfainberg | clarkb, is it? i thought it wasn't. wow brain might be fried | 01:01 |
| morganfainberg | clarkb, let me go check | 01:01 |
| *** dims__ has quit IRC | 01:01 | |
| fungi | "Copyleft licenses may be okay depending on circumstances" is nice and vague on the wiki | 01:02 |
| morganfainberg | clarkb, you're right v3 is compat | 01:02 |
| clarkb | so if we refer to apache foundation | 01:02 |
| clarkb | you can't copy pasta gpl code into your stuff | 01:02 |
| morganfainberg | clarkb, ASLv2 code can be accepted into GPLv3 codebases but GPL code can't be added to ASLv2 codebases | 01:02 |
| morganfainberg | clarkb, but requring the use of a GPL lib shouldn't be bad. | 01:02 |
| clarkb | right | 01:02 |
| clarkb | and that only covers copy pasta | 01:03 |
| clarkb | not linking | 01:03 |
| sdague | morganfainberg: GPL includes dyanmic linking | 01:03 |
| morganfainberg | otherwise we might be screwed with Linux Kernel headers :P let alone anything else | 01:03 |
| morganfainberg | sdague, it does? | 01:03 |
| sdague | yes, that's the whole reason for LGPL | 01:03 |
| clarkb | sdague: right so this becomes a distribution thing iirc | 01:03 |
| clarkb | sdague: openstack doesn't require mysql python so we are ok or some such | 01:03 |
| clarkb | there was a whole thread on the ml | 01:03 |
| sdague | ok, that predates me | 01:04 |
| lifeless | also 'linking' in C works by copying | 01:04 |
| sdague | I will not open that can of worms | 01:04 |
| lifeless | it doesn't work that way in Python | 01:04 |
| morganfainberg | sdague, yeah reading it now. | 01:04 |
| morganfainberg | does a python "import" constittute linking? | 01:04 |
| lifeless | and lets not think about interface copyright and all that implication | 01:04 |
| fungi | because we also work with other db connectivity libraries under an orm | 01:04 |
| clarkb | fungi: right | 01:04 |
| morganfainberg | IANAL and especially not an IP Lawyer | 01:04 |
| sdague | lifeless: I'm pretty sure import == dynamic link for these purposes | 01:04 |
| Alex_Gaynor | Depends which lawyer you ask. | 01:05 |
| lifeless | sdague: IANAL, but I know laywers on both sides of the answer. | 01:05 |
| fungi | possibly also exec under a pipe back into the code | 01:05 |
| morganfainberg | so.. i'll just go with ... we should ask a person smarter about legal stuff if this is a real issue. | 01:05 |
| sdague | Alex_Gaynor: curious. | 01:05 |
| morganfainberg | lifeless, the important question is which side does the foundation support | 01:05 |
| lifeless | morganfainberg: actually, its which side do our deployers support | 01:05 |
| clarkb | morganfainberg: the foundation apparently doesn't offer legal advice in this arena | 01:05 |
| *** starmer has joined #openstack-infra | 01:06 | |
| clarkb | morganfainberg: its crazy pants in some ways | 01:06 |
| clarkb | lifeless: and distributors | 01:06 |
| fungi | the foundation supports whatever its legal counsel recommends in specific cases | 01:06 |
| ianw | sdague: do you think we can get this one through soon? https://review.openstack.org/#/c/97532/ (Enable horizon on Fedoras) ... it's pretty straight forward | 01:06 |
| morganfainberg | sdague, to Alex_Gaynor 's point, i don't think anyone has had a precedent setting case on this front. | 01:06 |
| lifeless | clarkb: GPL is a use license, distribtors don't need to care (with respect to this) | 01:06 |
| sdague | morganfainberg: sure | 01:06 |
| lifeless | clarkb: but LALALALALALALAL | 01:06 |
| morganfainberg | sdague, it's a lot of hand waving | 01:06 |
| sdague | all the precidents for GPL infringement have been basically openwrt related | 01:07 |
| sdague | so it's all c code | 01:07 |
| lifeless | Ubuntu's boot-time linking of nvidia drivers is analogous to 'import foo is not linking *for GPL distribution aspects*' btw | 01:07 |
| lifeless | and that got all firey when it happened | 01:07 |
| sdague | lifeless: sure, but it also depends on the copyright holders | 01:07 |
| fungi | there need to be a crapton of gpl infringement cases around google android and hardware manufacturers shipping it with customized/patched kernels and no source | 01:08 |
| clarkb | gah I realize now one reason puppet may not be doing what I want it to do | 01:08 |
| sdague | ianw: dtroyer is back | 01:08 |
| clarkb | dib is using upstream puppet not my local puppet | 01:08 |
| clarkb | mordred: ^ if there is a way to fix that can you let me know | 01:08 |
| lifeless | clarkb: dib doesn't use puppet itself at all | 01:08 |
| clarkb | lifeless: well the dib elements mordred wrote | 01:08 |
| *** yamahata has joined #openstack-infra | 01:08 | |
| clarkb | puppte failing is also not causing the whole thing to fail despite my set -e | 01:09 |
| lifeless | clarkb: add an echo $? ? | 01:09 |
| fungi | clarkb: we have the same problem in nodepool image-update in some cases | 01:09 |
| clarkb | fungi: huh | 01:09 |
| ianw | dtroyer: re above https://review.openstack.org/#/c/97532/ (Enable horizon on Fedoras) would help with f20 jobs | 01:09 |
| *** starmer has quit IRC | 01:10 | |
| clarkb | lifeless: ya I will have to do a lot more digging | 01:10 |
| fungi | prep scripts are set -e but there are some puppet apply failures which just get ignored and we end up with a busted image instead | 01:10 |
| *** salv-orlando has quit IRC | 01:10 | |
| sdague | fungi: something turning it off? | 01:10 |
| clarkb | fungi: so maybe puppet service {} does the "right" thing afterall | 01:10 |
| clarkb | :P | 01:11 |
| sdague | heh | 01:11 |
| clarkb | those are the only two "err"s | 01:11 |
| clarkb | and if that doesn't make puppet fail then idunno | 01:11 |
| fungi | sdague: no, puppet is being called from the set -e script and is not aborting on its own when it hits some types of failures, then seems to exit 0 when done | 01:12 |
| openstackgerrit | Ian Wienand proposed a change to openstack-infra/config: Add f20 job to devstack-gate https://review.openstack.org/98281 | 01:12 |
| sdague | fungi: interesting | 01:12 |
| clarkb | http://projects.puppetlabs.com/issues/19033 | 01:13 |
| clarkb | its a bug | 01:13 |
| clarkb | we can apparently use the detailed exit codes flag and parse $? ourselves | 01:13 |
| fungi | sdague: we ran into it at one point a few months ago where puppet failed to install/configure some things for reasons i can't recall now, and we ended up with images in use from the resulting runs | 01:13 |
| sdague | clarkb: it's good that the bug was accepted a year ago... | 01:14 |
| *** yamahata has quit IRC | 01:15 | |
| *** yamahata has joined #openstack-infra | 01:15 | |
| fungi | so i guess --detailed-exit-codes is something we should probably be running with regardless? | 01:16 |
| clarkb | I guess | 01:18 |
| morganfainberg | fungi, so silent failure, exit on failure with success return code? just wow. | 01:18 |
| morganfainberg | fungi, that makes me cry a little inside. | 01:18 |
| fungi | morganfainberg: my tear ducts ran dry years ago | 01:19 |
| *** starmer has joined #openstack-infra | 01:20 | |
| *** nati_uen_ has quit IRC | 01:21 | |
| *** nati_ueno has joined #openstack-infra | 01:21 | |
| openstackgerrit | Ken'ichi Ohmichi proposed a change to openstack/requirements: Add license descriptions to a-z libraries https://review.openstack.org/98282 | 01:22 |
| harlowja | arg, do we really want to be doing that to the requirements file :-p | 01:22 |
| lifeless | ugh no | 01:22 |
| harlowja | https://github.com/openstack/requirements/blob/master/detail.py#L38 will suck that same info down from pypi | 01:22 |
| clarkb | lifeless: in dib if I want to clone from a location outside of the chroot I assume this is already a sovled rpoblem? | 01:23 |
| clarkb | ideally I could give git a filesystem path so that I don't need to host a thing with apache | 01:23 |
| clarkb | I think I want a line in a source-repository file | 01:23 |
| clarkb | but I am not sure what that should look like | 01:24 |
| jogo | so I am trying to use the gerrit REST API and having some issues | 01:24 |
| jogo | requests.get("https://review.openstack.org:443/changes/98265").json() | 01:24 |
| jogo | fails | 01:24 |
| jogo | with a json decode error | 01:24 |
| clarkb | jogo: because that is an html page | 01:24 |
| clarkb | I think | 01:24 |
| anteaya | pleia2: thanks, it was a good walk | 01:25 |
| clarkb | oh nevermind thats json | 01:25 |
| *** starmer has quit IRC | 01:25 | |
| clarkb | broken json | 01:25 |
| clarkb | but json | 01:25 |
| harlowja | 99% json, lol | 01:25 |
| harlowja | 1% a little extra | 01:25 |
| jogo | how does that happen? | 01:26 |
| clarkb | gerrit bug? | 01:26 |
| morganfainberg | harlowja, soooo close to json... you can almost taste the serialization | 01:26 |
| harlowja | json-next-gen, ha | 01:27 |
| *** sarob has joined #openstack-infra | 01:27 | |
| jogo | To prevent against Cross Site Script Inclusion (XSSI) attacks, the JSON response body starts with a magic prefix line that must be stripped before feeding the rest of the response body to a JSON parser: | 01:27 |
| sdague | go JAVA! | 01:27 |
| jogo | https://gerrit-review.googlesource.com/Documentation/rest-api.html | 01:27 |
| jogo | derp | 01:27 |
| harlowja | goo google | 01:28 |
| lifeless | same madness websockets has | 01:28 |
| harlowja | ya, there are some valid reasons for some of this, i can't remember what they are right now though, ha | 01:29 |
| *** zns has joined #openstack-infra | 01:30 | |
| anteaya | mriedem: this one failed again https://review.openstack.org/#/c/95831/ | 01:30 |
| harlowja | at least they didn't put in while(1); which is also common :-P | 01:32 |
| harlowja | * https://news.ycombinator.com/item?id=5168121 | 01:32 |
| *** sarob has quit IRC | 01:32 | |
| *** MarkAtwood has joined #openstack-infra | 01:32 | |
| openstackgerrit | Ian Wienand proposed a change to openstack-infra/devstack-gate: Use root to write syslog from journalctl https://review.openstack.org/98283 | 01:33 |
| jogo | pleia2: woot its working | 01:35 |
| jogo | pleia2: gonna push a patch up soon | 01:35 |
| jogo | pleia2: oh wait that isn't on stackforge | 01:35 |
| *** yaguang has joined #openstack-infra | 01:35 | |
| *** reed has quit IRC | 01:36 | |
| *** zz_gondoi is now known as gondoi | 01:37 | |
| openstackgerrit | Ian Wienand proposed a change to openstack-infra/devstack-gate: Use root to write syslog from journalctl https://review.openstack.org/98283 | 01:38 |
| *** aconrad has joined #openstack-infra | 01:40 | |
| clarkb | hrm another thing that may fall apart is facter | 01:40 |
| clarkb | maybe | 01:40 |
| jogo | http://paste.openstack.org/show/83027 | 01:41 |
| jogo | mikal: ^ | 01:41 |
| tchaypo | so when i run gertty should i expect it to just do nothing for a while? | 01:41 |
| tchaypo | it says "sync: 168" in the top right | 01:42 |
| clarkb | tchaypo: that number should fall to zero over time | 01:42 |
| jogo | mikal: since we are on a approval freeze, figured I would do this | 01:42 |
| fungi | tchaypo: ? for help (or f1), l to list all projects | 01:42 |
| clarkb | tchaypo: new gertty only syncs if changes are made | 01:42 |
| jogo | mikal: open bugs and associated patch statuses | 01:42 |
| jogo | pleia2 clarkb: its working pretty well | 01:42 |
| mestery | jogo: This one failed again, just did a re-check again (https://review.openstack.org/#/c/97245/) :( | 01:43 |
| fungi | tchaypo: it starts on the projects screen with the list of projects to which you've subscribed | 01:43 |
| fungi | tchaypo: which initially may be none | 01:43 |
| jogo | mestery: so many bugs | 01:43 |
| mestery | jogo: I know :( | 01:43 |
| mestery | jogo: It's been a rough week I think | 01:43 |
| mestery | jogo: Hopefully this one makes it through overnight, I think it will help once it lands. | 01:43 |
| clarkb | fungi: we still good with nodepool non leakage? | 01:44 |
| fungi | clarkb: seems so | 01:44 |
| clarkb | cool | 01:44 |
| clarkb | I think I sorted out how to have my local dev copy of config slurped into the dib image so I can run puppet against that | 01:44 |
| clarkb | so I may have actual useful test data soonish | 01:45 |
| clarkb | but I should head home soon | 01:45 |
| clarkb | there is dinner and beer at home | 01:45 |
| lifeless | clarkb: generally its 'export a ref to it' for source-repositories | 01:46 |
| *** nosnos has joined #openstack-infra | 01:46 | |
| lifeless | clarkb: theres some glue in adam_g's tripleo-end-to-end thing that you could riff off of | 01:46 |
| *** wenlock has joined #openstack-infra | 01:46 | |
| clarkb | lifeless: I found some rudimentary stuff mordred did and have hard coded for now | 01:47 |
| clarkb | but should add an optional ref at some point | 01:47 |
| tchaypo | I've subscribed to a bunch of projects but they're all 0/0 | 01:47 |
| clarkb | it failing now so I think the changes "worked" | 01:47 |
| tchaypo | no wait - infra just switched to 1/1 | 01:47 |
| ianw | does anyone know where the logs for ./prepare_node_devstack.sh go after it's run in http://nodepool.openstack.org/image.log? | 01:48 |
| clarkb | ianw: thats it | 01:48 |
| *** aconrad has quit IRC | 01:48 | |
| ianw | clarkb: so the output of that isn't captured? or am i missing something? | 01:49 |
| fungi | clarkb: spot checks comparing nova list in hpcloud-b with nodepool list for it come back within a couple percent | 01:49 |
| clarkb | ianw: I guess I don't understand the question | 01:49 |
| fungi | i assume any leak would be much more noticeable by now | 01:49 |
| clarkb | ianw: the image logs capture the output of the node preparation scripts | 01:49 |
| *** starmer has joined #openstack-infra | 01:50 | |
| fungi | ianw: if those logs aren't verbose enough about what's being run in that script, i guess the script could be made to set -x | 01:50 |
| jerryz | fungi: do we still use username, key file to authenticate for jenkins slaves? or just credential id? | 01:51 |
| *** MarkAtwood has quit IRC | 01:51 | |
| jogo | mestery: you hit another bug https://review.openstack.org/#/c/98265/ is needed first | 01:52 |
| clarkb | jerryz: I think we use credential id | 01:52 |
| clarkb | which I want to change because they are clunky but meh | 01:52 |
| mestery | jogo: Nuts, I figured that after checking a moment ago. :( | 01:52 |
| mestery | jogo: It's all about sequencing :) | 01:52 |
| fungi | jerryz: the jenkins master would be configured with a username and the path to a keyfile to use to connect to slaves via ssh, but that credential shows up as a uuid in the jenkins config.xml which needs to be referenced in the nodepool yaml config | 01:53 |
| mikal | jogo: how did you pick those bugs? | 01:53 |
| ianw | fungi: that's why it's a bit confusing, all those scripts have '#!/bin/bash -xe'? | 01:53 |
| fungi | ianw: perhaps you're missing that you need to filter the image.log by the image and provider name appearing in the prefix of each log line (they're all interleaved, building in parallel) | 01:54 |
| jogo | mikal novaclient bugs | 01:54 |
| anteaya | jogo mestery that patch failed jenkins: 2014-06-05 23:26:45,370 Creating ssh connection to '172.24.4.2' as 'cirros' with public key authentication | 01:54 |
| anteaya | mestery: is that the ssh bug? | 01:54 |
| jerryz | clarkb fungi: i am working on a change to allow nodepool to register slaves that jenkins master connects through a ssh jump host, but i have to use key file to authenticate to those slaves, credential id is not visable to the ssh command | 01:54 |
| * mestery checks | 01:54 | |
| anteaya | error: [Errno 113] No route to host | 01:54 |
| jogo | mikal: using https://github.com/jogo/openstack-infra-scripts | 01:54 |
| ianw | fungi: pretty sure that's what i'm doing http://paste.openstack.org/show/83030/ | 01:55 |
| clarkb | jerryz: credential id is just a keyfile + username set of options | 01:55 |
| jogo | anteaya: ohh that my be a different bug :) | 01:55 |
| clarkb | jerryz: You should be able to continue to use credential ids | 01:55 |
| anteaya | yay | 01:55 |
| mestery | anteaya: It is, but that test uses nova-network, not neutron | 01:55 |
| *** starmer has quit IRC | 01:55 | |
| anteaya | an ssh nova-net bug | 01:55 |
| mestery | anteaya: I think so, yes. | 01:55 |
| mestery | anteaya: It's possible it's the same cause as the neutron one though, but hard to tell. | 01:55 |
| jerryz | clarkb: yes, but the information is stored in jenkins credential manager | 01:56 |
| jerryz | clarkb: nodepool won't know that | 01:56 |
| clarkb | jerryz: it doesn't need to | 01:56 |
| jogo | mikal: full novaclient list http://paste.openstack.org/show/83031 | 01:56 |
| clarkb | you configure jenkins to jump host and you configure nodepool to jump host | 01:56 |
| clarkb | those should be independent problems | 01:56 |
| fungi | ianw: your paste looks like it's showing the commands being run from prepare_node_devstack.sh | 01:56 |
| ianw | fungi: bah, i see what's going on ... http://nodepool.openstack.org/image.log is truncated for some reason | 01:58 |
| ianw | i thought it would be a symlink to the latest | 01:58 |
| *** aconrad has joined #openstack-infra | 01:58 | |
| ianw | http://nodepool.openstack.org/image.log.2014-05-06 shows what i expect | 01:58 |
| fungi | ianw: oh, perhaps. checking on the server now | 01:58 |
| anteaya | is this an important part of the error log? chmod: cannot access `/opt/stack/logs/nosetests*.xml': No such file or directory | 01:58 |
| anteaya | I never know if it is important or a red herring | 01:58 |
| *** Fabio has quit IRC | 01:58 | |
| jerryz | clarkb: but i need to write scripts which run on master to launch jump host connected nodes. not hudson.plugins.sshslaves.SSHLauncher | 01:59 |
| clarkb | jerryz: can't you just set the appropriate values in .ssh/config? | 01:59 |
| jerryz | clarkb: sshlauncher doesn't hornor that | 01:59 |
| clarkb | wow | 02:00 |
| clarkb | you can make paramiko honor that even | 02:00 |
| anteaya | jogo mestery well there is the error via testr: http://logs.openstack.org/65/98265/1/check/check-grenade-dsvm/666a92e/logs/testr_results.html.gz | 02:00 |
| jerryz | clarkb: yeah | 02:00 |
| jerryz | it works for nodepool | 02:00 |
| fungi | ianw: yep, last modified Jun 5 14:42 and way smaller than the previous days | 02:00 |
| fungi | ianw: not sure what happened to today's image log, but if it happens again tomorrow i'll dig into it | 02:00 |
| *** mfer has quit IRC | 02:01 | |
| jerryz | clarkb: but i would still want to use -o ProxyCommand, not ssh config because host filter would not apply for every case in private clouds | 02:01 |
| anteaya | it is a new one: message:"FAIL: tempest.scenario.test_volume_boot_pattern.TestVolumeBootPatternV2.test_volume_boot_pattern[compute,image,volume]" | 02:01 |
| anteaya | no events matched for 7 days | 02:01 |
| anteaya | this is nova? | 02:01 |
| jerryz | clarkb: when ip range are the same for different private clouds | 02:02 |
| *** mfer has joined #openstack-infra | 02:03 | |
| * anteaya files a bug report in nova | 02:03 | |
| *** mfer has quit IRC | 02:04 | |
| *** starmer has joined #openstack-infra | 02:05 | |
| openstackgerrit | Dan Prince proposed a change to openstack-infra/nodepool: Add support for network labels https://review.openstack.org/98287 | 02:06 |
| anteaya | jogo: can you make sure these bugs are not the same? https://bugs.launchpad.net/tempest/+bug/1226337 and https://bugs.launchpad.net/nova/+bug/1249065 | 02:06 |
| uvirtbot | Launchpad bug 1226337 in cinder "tempest.scenario.test_volume_boot_pattern.TestVolumeBootPattern 'qemu-nbd: Failed to bdrv_open'" [Critical,Fix released] | 02:06 |
| jogo | anteaya: from a quick glance they look very different | 02:07 |
| anteaya | kk, I'll file a new bug | 02:07 |
| jogo | anteaya: I am distracted playing with http://paste.openstack.org/show/83029/ | 02:08 |
| openstackgerrit | Ian Wienand proposed a change to openstack-infra/devstack-gate: Fix ntp restart for Fedora https://review.openstack.org/98288 | 02:10 |
| anteaya | jogo: fine, just if it comes to me getting someone wagging a finger at me if I filed a dup, I did ask | 02:10 |
| *** starmer has quit IRC | 02:10 | |
| jogo | anteaya: hehe | 02:11 |
| openstackgerrit | Ian Wienand proposed a change to openstack-infra/devstack-gate: Fix ntp restart for Fedora https://review.openstack.org/98288 | 02:11 |
| *** morganfainberg is now known as morganfainberg_Z | 02:14 | |
| *** zns has quit IRC | 02:15 | |
| openstackgerrit | Anita Kuno proposed a change to openstack-infra/elastic-recheck: Add query for nova test_volume_boot_pattern bug 1327020 https://review.openstack.org/98290 | 02:16 |
| uvirtbot | Launchpad bug 1327020 in nova "FAIL: tempest.scenario.test_volume_boot_pattern.TestVolumeBootPatternV2.test_volume_boot_pattern[compute,image,volume]" [Undecided,New] https://launchpad.net/bugs/1327020 | 02:16 |
| *** sarob has joined #openstack-infra | 02:17 | |
| anteaya | so clarkb's patch is back in check, about an hour until we see how that does | 02:17 |
| *** sarob has quit IRC | 02:19 | |
| *** sarob has joined #openstack-infra | 02:19 | |
| *** starmer has joined #openstack-infra | 02:20 | |
| *** mmaglana has joined #openstack-infra | 02:21 | |
| *** praneshp has joined #openstack-infra | 02:23 | |
| *** sarob has quit IRC | 02:23 | |
| *** starmer has quit IRC | 02:25 | |
| *** praneshp_ has joined #openstack-infra | 02:26 | |
| *** praneshp has quit IRC | 02:27 | |
| *** praneshp_ is now known as praneshp | 02:27 | |
| *** morganfainberg_Z is now known as morganfainberg | 02:28 | |
| *** zns has joined #openstack-infra | 02:32 | |
| *** sweston has joined #openstack-infra | 02:32 | |
| *** zns has quit IRC | 02:32 | |
| *** zns has joined #openstack-infra | 02:33 | |
| *** starmer has joined #openstack-infra | 02:35 | |
| *** aconrad has quit IRC | 02:38 | |
| *** starmer has quit IRC | 02:40 | |
| *** morganfainberg is now known as morganfainberg_Z | 02:42 | |
| *** morganfainberg_Z is now known as morganfainberg | 02:43 | |
| adrian_otto | does someone know a way to change the status of a bunch of LP bugs from "Fix Committed" to "Fix Released" after a release? | 02:43 |
| adrian_otto | I have a whole mess of them that I don't want to touch individually | 02:43 |
| morganfainberg | adrian_otto, i think ttx has some magic scripts to do it | 02:44 |
| *** gyee has quit IRC | 02:44 | |
| morganfainberg | adrian_otto, https://github.com/openstack-infra/release-tools/blob/master/process_bugs.py | 02:45 |
| morganfainberg | adrian_otto, that looks like the script | 02:45 |
| adrian_otto | <3 | 02:46 |
| *** harlowja is now known as harlowja_away | 02:46 | |
| morganfainberg | anteaya, maybe you know, but is there a reason the pipermail archive is never the one that shows up in google when searching for dev-list threads? | 02:48 |
| *** plars-away is now known as plars | 02:50 | |
| *** starmer has joined #openstack-infra | 02:50 | |
| *** sweston has quit IRC | 02:53 | |
| anteaya | morganfainberg: I don't know the answer to that question, no | 02:54 |
| anteaya | I have noticed that though | 02:55 |
| *** starmer has quit IRC | 02:55 | |
| morganfainberg | anteaya, wonder if it's some setting we have or if it's something else. | 02:56 |
| morganfainberg | anteaya, I shall make it my goal to poke into that this weekend! :) | 02:56 |
| anteaya | i would be only speculating | 02:56 |
| anteaya | and to be honest I am too tired to do that | 02:56 |
| morganfainberg | hehe | 02:56 |
| anteaya | a worthy goal | 02:56 |
| morganfainberg | don't speculate then! | 02:56 |
| anteaya | thanks | 02:56 |
| anteaya | I shall not | 02:57 |
| anteaya | I have to do a family thing, still not sure if I am going to start weekending on friday or saturday | 02:57 |
| anteaya | and do the stuff on the meeting action items I said I would have done for next week | 02:57 |
| anteaya | and book flights | 02:58 |
| anteaya | I'm horrible at planning | 02:58 |
| morganfainberg | anteaya, good luck with it! | 02:58 |
| anteaya | I wish for a magic carpet that would just take me places | 02:58 |
| anteaya | thanks | 02:58 |
| anteaya | then I don't have to plan stuff in advance | 02:58 |
| anteaya | I never know how I'm going to feel before I feel it | 02:58 |
| anteaya | I read the queen has to select menus of events a year in advance | 02:59 |
| anteaya | a year | 02:59 |
| anteaya | I wouldn't survive | 02:59 |
| morganfainberg | lol, oh my a year in advance? | 03:00 |
| *** mmaglana has quit IRC | 03:01 | |
| anteaya | that is what I said | 03:01 |
| *** wenlock has quit IRC | 03:01 | |
| anteaya | I have no idea what I want in my salad next year | 03:01 |
| anteaya | I can't decide on tomorrow's salad | 03:01 |
| *** mmaglana has joined #openstack-infra | 03:01 | |
| anteaya | apparently she eats everything she selected and never complains | 03:02 |
| *** homeless has quit IRC | 03:04 | |
| clarkb | Im sure she picks 10 of everything | 03:04 |
| anteaya | ha ha ha | 03:04 |
| clarkb | then you have choices | 03:04 |
| *** starmer has joined #openstack-infra | 03:05 | |
| *** mmaglana has quit IRC | 03:05 | |
| *** ayoung has joined #openstack-infra | 03:06 | |
| *** mmaglana has joined #openstack-infra | 03:06 | |
| *** sweston has joined #openstack-infra | 03:06 | |
| fungi | morganfainberg: i just prefix my searches with site:lists.openstack.org | 03:07 |
| morganfainberg | fungi, ah good idea. | 03:07 |
| morganfainberg | fungi, still seems a little silly | 03:07 |
| *** sandywalsh_ has quit IRC | 03:08 | |
| *** unicell has joined #openstack-infra | 03:08 | |
| clarkb | I find it works just searcing openstack foo | 03:08 |
| fungi | morganfainberg: we don't go out of our way to fancy our dev systems up with "search engine optimization" as having it come up as the first hit for an unqualified search isn't much of a concern | 03:09 |
| *** starmer has quit IRC | 03:10 | |
| morganfainberg | fungi, i don't expect us to, but if it's a simple change to make it more likely to pop to the top, why not? | 03:11 |
| fungi | morganfainberg: if it's misconfigured, then sure | 03:11 |
| morganfainberg | fungi, my thought was - look at the simple stuff, if it's anything beyond simple, meh. | 03:11 |
| *** basha has joined #openstack-infra | 03:11 | |
| fungi | morganfainberg: problem is that a lot of the other mailing list aggregators want to sell eyeballs, so they jump through hoops to up their search engine rankings | 03:12 |
| anteaya | boo | 03:12 |
| anteaya | http://logs.openstack.org/65/98265/1/check/check-dg-tempest-dsvm-full/b8232ab/logs/testr_results.html.gz | 03:12 |
| morganfainberg | fungi, and if that is the whole reason we aren't there, not much we should be doing about it | 03:12 |
| morganfainberg | fungi. so, i am in vehement agreement with you :) | 03:12 |
| anteaya | clarkb: your patch hit this bug for the second time | 03:12 |
| *** matsuhashi has quit IRC | 03:13 | |
| *** nosnos has quit IRC | 03:13 | |
| anteaya | https://bugs.launchpad.net/nova/+bug/1327020 | 03:13 |
| uvirtbot | Launchpad bug 1327020 in nova "FAIL: tempest.scenario.test_volume_boot_pattern.TestVolumeBootPatternV2.test_volume_boot_pattern[compute,image,volume]" [Undecided,New] | 03:13 |
| fungi | morganfainberg: and i you. if there's something simple we're supposed to be doing and aren't, then we should probably do it | 03:13 |
| *** nosnos has joined #openstack-infra | 03:13 | |
| fungi | morganfainberg: but like, if i want to search debian or openbsd or whatever mailing list archives from a web search engine, i pretty much always run into the same exact issue | 03:13 |
| morganfainberg | fungi, yeah, selling ad space is a big deal and lots of eyes on mailing list archives | 03:14 |
| fungi | the aggregators always seem to pop up with tons of entries well before the actual official archives for free software project mailing lists | 03:14 |
| *** matsuhashi has joined #openstack-infra | 03:14 | |
| adrian_otto | morganfainberg: that script worked like a charm. | 03:14 |
| *** matsuhashi has quit IRC | 03:14 | |
| morganfainberg | adrian_otto, awesome, glad to help | 03:14 |
| *** mbacchi has quit IRC | 03:14 | |
| anteaya | okay I've had enough | 03:15 |
| anteaya | signing off and see you tomorrow | 03:15 |
| morganfainberg | adrian_otto, i wish i could have used that script on the cleanup i did yesterday for Keystone. | 03:15 |
| morganfainberg | adrian_otto, but... it wasn't simple like a release :P | 03:16 |
| *** nosnos has quit IRC | 03:17 | |
| clarkb | morganfainberg: you could just mark everything fix released | 03:17 |
| clarkb | I won't tell | 03:17 |
| * morganfainberg smirks. | 03:17 | |
| *** sweston has quit IRC | 03:18 | |
| *** basha has quit IRC | 03:18 | |
| *** starmer has joined #openstack-infra | 03:20 | |
| fungi | i was slightly disturbed to see updates to diablo-era bug reports | 03:24 |
| *** starmer has quit IRC | 03:25 | |
| *** julim has joined #openstack-infra | 03:27 | |
| *** rainya has quit IRC | 03:27 | |
| *** julim has quit IRC | 03:29 | |
| *** nosnos has joined #openstack-infra | 03:31 | |
| *** wenlock has joined #openstack-infra | 03:33 | |
| *** HenryG has quit IRC | 03:34 | |
| *** julim has joined #openstack-infra | 03:35 | |
| *** starmer_ has joined #openstack-infra | 03:35 | |
| *** talluri has joined #openstack-infra | 03:37 | |
| *** matsuhashi has joined #openstack-infra | 03:38 | |
| *** nosnos has quit IRC | 03:39 | |
| *** nosnos has joined #openstack-infra | 03:40 | |
| *** starmer_ has quit IRC | 03:40 | |
| *** rainya has joined #openstack-infra | 03:45 | |
| *** adrian_otto has quit IRC | 03:47 | |
| *** matsuhashi has quit IRC | 03:48 | |
| *** starmer has joined #openstack-infra | 03:50 | |
| *** starmer has quit IRC | 03:55 | |
| fabo | anyone familiar with jenkins-jobs-builder and the ssh publisher? | 03:56 |
| *** matsuhashi has joined #openstack-infra | 03:56 | |
| *** matsuhashi has quit IRC | 03:56 | |
| *** godp1301 has joined #openstack-infra | 03:57 | |
| clarkb | fabo: sure (though not ssh publisher) | 03:58 |
| clarkb | do you mean scp publisher? | 03:58 |
| *** nosnos has quit IRC | 04:00 | |
| fabo | clarkb: yes, publish over ssh plugin | 04:00 |
| clarkb | so I am not familiar with that plugin. we use the scp plugin | 04:01 |
| fabo | clarkb: current implementation is generating a post-build action, and doesn't aggregate multiple hosts | 04:02 |
| fabo | clarkb: I guess scp plugin is doing the same | 04:02 |
| *** gokrokve has joined #openstack-infra | 04:02 | |
| clarkb | can you just define multiple ssh publishers? | 04:02 |
| clarkb | that is what we do with scp | 04:02 |
| fabo | clarkb: I'm wondering if anyone thought about having it in build actions? | 04:03 |
| *** matsuhashi has joined #openstack-infra | 04:04 | |
| clarkb | does the ssh publisher plugin support build actions? | 04:04 |
| clarkb | if so you can definitely add it | 04:04 |
| fabo | clarkb: yes, it's supported | 04:05 |
| clarkb | fabo: can definitely add it then | 04:05 |
| *** starmer has joined #openstack-infra | 04:05 | |
| clarkb | I am not sure how that solves the multi host problem though | 04:05 |
| clarkb | you should be able to have multiple post build actions that talk to multiple hots | 04:06 |
| fabo | clarkb: any example on the syntax you would expect to differentiate when you want it in post-build vs build actions? | 04:06 |
| *** mmaglana has quit IRC | 04:06 | |
| fabo | clarkb: I'm willing to do the changes but not sure which syntax is expected | 04:07 |
| clarkb | fabo: yes in the yaml there is a builders section and a publishers section | 04:07 |
| *** mmaglana has joined #openstack-infra | 04:07 | |
| clarkb | you have to add it to the builders section if that is where you want it | 04:07 |
| fabo | clarkb: ah good point :) | 04:07 |
| fabo | clarkb: thanks, looking into that then | 04:07 |
| *** aconrad has joined #openstack-infra | 04:07 | |
| *** gondoi is now known as zz_gondoi | 04:08 | |
| *** starmer has quit IRC | 04:09 | |
| *** mmaglana has quit IRC | 04:12 | |
| *** gokrokve has quit IRC | 04:12 | |
| *** crc32 has quit IRC | 04:14 | |
| *** starmer has joined #openstack-infra | 04:20 | |
| *** nosnos has joined #openstack-infra | 04:21 | |
| *** HenryG has joined #openstack-infra | 04:24 | |
| *** starmer has quit IRC | 04:25 | |
| SergeyLukjanov | morning folks | 04:27 |
| SergeyLukjanov | clarkb, is anything awful happens today? :) | 04:27 |
| *** harlowja_at_home has joined #openstack-infra | 04:27 | |
| clarkb | SergeyLukjanov: it was much better today. gate is still cranky | 04:27 |
| openstackgerrit | afazekas proposed a change to openstack-infra/elastic-recheck: Add query for 1244457 https://review.openstack.org/98297 | 04:28 |
| *** zns has quit IRC | 04:29 | |
| *** zns has joined #openstack-infra | 04:30 | |
| *** zns has joined #openstack-infra | 04:31 | |
| SergeyLukjanov | clarkb, cool | 04:33 |
| SergeyLukjanov | just finished scrollback | 04:33 |
| clarkb | then I spent the afternoon beating dib with a stick and crying | 04:33 |
| SergeyLukjanov | clarkb, oh, I was thinking that it works good enough, we're using it for building sahara imagrs | 04:34 |
| *** gokrokve has joined #openstack-infra | 04:34 | |
| clarkb | SergeyLukjanov: the hard bit is we have existing puppet that we want to use and making that play nice has been an adventure | 04:34 |
| clarkb | SergeyLukjanov: a lot of the bits do work well like the caching | 04:34 |
| *** Longgeek has joined #openstack-infra | 04:35 | |
| *** starmer has joined #openstack-infra | 04:35 | |
| SergeyLukjanov | oh, yeah... got it | 04:35 |
| clarkb | SergeyLukjanov: https://review.openstack.org/#/c/88479/15 is the change | 04:36 |
| clarkb | I pushed a new patchset with almost working devstack images | 04:36 |
| clarkb | SergeyLukjanov: the current problem with them is puppet fails trying to restart services | 04:36 |
| clarkb | (but dib doesn't fail because puppet returns 0 anyways) | 04:36 |
| clarkb | need to make puppet not fail so that we can make puppet return detailed return codes and fail the dib build if there are any puppet failres | 04:37 |
| mattoliverau | clarkb: do you ever sleep or have you been replaced by some kind of robot? | 04:37 |
| clarkb | mattoliverau: I try to sleep | 04:38 |
| clarkb | and no robot unless it happened in my sleep | 04:38 |
| *** mburned is now known as mburned_out | 04:38 | |
| SergeyLukjanov | :) | 04:39 |
| *** starmer has quit IRC | 04:40 | |
| *** gokrokve has quit IRC | 04:40 | |
| *** amotoki has joined #openstack-infra | 04:43 | |
| *** godp1301 has quit IRC | 04:43 | |
| *** marcoemorais has joined #openstack-infra | 04:43 | |
| *** godp1301 has joined #openstack-infra | 04:44 | |
| *** trinaths has joined #openstack-infra | 04:44 | |
| *** Ryan_Lane has quit IRC | 04:44 | |
| *** marcoemorais1 has joined #openstack-infra | 04:46 | |
| *** marcoemorais has quit IRC | 04:48 | |
| *** godp1301 has quit IRC | 04:50 | |
| *** starmer has joined #openstack-infra | 04:50 | |
| *** harlowja_at_home has quit IRC | 04:53 | |
| *** basha has joined #openstack-infra | 04:55 | |
| *** starmer has quit IRC | 04:55 | |
| *** trinaths has left #openstack-infra | 04:56 | |
| *** ildikov has quit IRC | 04:56 | |
| *** trinaths has joined #openstack-infra | 04:57 | |
| *** ianw has quit IRC | 04:59 | |
| *** ianw has joined #openstack-infra | 04:59 | |
| *** ianw has quit IRC | 04:59 | |
| *** ianw has joined #openstack-infra | 05:00 | |
| jogo | lifeless: how do I specify the number of bugs to return in launchpadlib | 05:03 |
| jogo | it seems by default I only get the first 100 | 05:03 |
| *** Sam-I-Am has quit IRC | 05:03 | |
| *** monester has quit IRC | 05:04 | |
| *** cyeoh has quit IRC | 05:04 | |
| *** vishy has quit IRC | 05:04 | |
| *** wendar has quit IRC | 05:04 | |
| *** jvrbanac has quit IRC | 05:04 | |
| *** asadoughi has quit IRC | 05:04 | |
| *** wendar has joined #openstack-infra | 05:05 | |
| *** jvrbanac has joined #openstack-infra | 05:05 | |
| *** marcoemorais1 has quit IRC | 05:06 | |
| *** vishy has joined #openstack-infra | 05:07 | |
| *** Sam-I-Am has joined #openstack-infra | 05:10 | |
| *** briancli1e has joined #openstack-infra | 05:10 | |
| *** monester has joined #openstack-infra | 05:11 | |
| *** trinaths has quit IRC | 05:15 | |
| *** julim has quit IRC | 05:15 | |
| *** praneshp has quit IRC | 05:15 | |
| *** yamahata has quit IRC | 05:15 | |
| *** adalbas has quit IRC | 05:15 | |
| *** AaronGr has quit IRC | 05:15 | |
| *** Hunner has quit IRC | 05:15 | |
| *** dmakogon_ has quit IRC | 05:15 | |
| *** torgomatic has quit IRC | 05:15 | |
| *** unicell has quit IRC | 05:15 | |
| *** nati_ueno has quit IRC | 05:15 | |
| *** otherwiseguy has quit IRC | 05:15 | |
| *** dosaboy has quit IRC | 05:15 | |
| *** cyli has quit IRC | 05:15 | |
| *** mdenny has quit IRC | 05:15 | |
| *** mwagner_lap has quit IRC | 05:15 | |
| *** fifieldt_ has quit IRC | 05:15 | |
| *** matrohon has quit IRC | 05:15 | |
| *** gabriel-bezerra has quit IRC | 05:15 | |
| *** plars has quit IRC | 05:15 | |
| *** isviridov|away has quit IRC | 05:15 | |
| *** masayukig has quit IRC | 05:15 | |
| *** spiffxp has quit IRC | 05:15 | |
| *** devananda has quit IRC | 05:15 | |
| *** krotscheck has quit IRC | 05:15 | |
| *** gargola has quit IRC | 05:15 | |
| *** Alexei_987 has quit IRC | 05:15 | |
| *** marun has quit IRC | 05:15 | |
| *** achuprin has quit IRC | 05:15 | |
| *** mancdaz has quit IRC | 05:15 | |
| *** johnthetubaguy has quit IRC | 05:15 | |
| *** ilyashakhat has quit IRC | 05:15 | |
| *** viktors|afk has quit IRC | 05:15 | |
| *** bauzas has quit IRC | 05:15 | |
| *** rossella_s has quit IRC | 05:15 | |
| *** DinaBelova has quit IRC | 05:15 | |
| *** JayF has quit IRC | 05:15 | |
| *** sandywalsh has quit IRC | 05:15 | |
| *** gfa has quit IRC | 05:15 | |
| *** mrodden has quit IRC | 05:15 | |
| *** ctracey has quit IRC | 05:15 | |
| *** asselin has quit IRC | 05:15 | |
| *** yolanda has quit IRC | 05:15 | |
| *** pcrews has quit IRC | 05:15 | |
| *** sabari has quit IRC | 05:15 | |
| *** hogepodge has quit IRC | 05:15 | |
| *** rickerc has quit IRC | 05:15 | |
| *** sulochan has quit IRC | 05:15 | |
| *** dougwig has quit IRC | 05:15 | |
| *** serverascode has quit IRC | 05:15 | |
| *** jamielennox has quit IRC | 05:15 | |
| *** skraynev has quit IRC | 05:15 | |
| *** ChanServ has quit IRC | 05:15 | |
| *** HenryG has quit IRC | 05:15 | |
| *** wenlock has quit IRC | 05:15 | |
| *** yjiang5 has quit IRC | 05:15 | |
| *** W00die has quit IRC | 05:15 | |
| *** dkehn has quit IRC | 05:15 | |
| *** Kiall has quit IRC | 05:15 | |
| *** NobodyCam has quit IRC | 05:15 | |
| *** locke105 has quit IRC | 05:15 | |
| *** jesusaurus has quit IRC | 05:15 | |
| *** jeblair has quit IRC | 05:15 | |
| *** dansmith has quit IRC | 05:15 | |
| *** ekarlso has quit IRC | 05:15 | |
| *** timfreund has quit IRC | 05:15 | |
| *** jpeeler has quit IRC | 05:15 | |
| *** dangers_away has quit IRC | 05:15 | |
| *** NithyaG_afk has quit IRC | 05:15 | |
| *** flaper87|afk has quit IRC | 05:15 | |
| *** soren has quit IRC | 05:15 | |
| *** lifeless has quit IRC | 05:15 | |
| *** pbelanyi has quit IRC | 05:15 | |
| *** vladan has quit IRC | 05:15 | |
| *** mtreinish has quit IRC | 05:15 | |
| *** dtroyer has quit IRC | 05:15 | |
| *** fabo has quit IRC | 05:15 | |
| *** Shrews has quit IRC | 05:15 | |
| *** mikal has quit IRC | 05:15 | |
| *** shardy has quit IRC | 05:15 | |
| *** kashyap has quit IRC | 05:15 | |
| *** pleia2 has quit IRC | 05:15 | |
| *** therve has quit IRC | 05:15 | |
| *** hughsaunders has quit IRC | 05:15 | |
| *** Alex_Gaynor has quit IRC | 05:15 | |
| *** Ng has quit IRC | 05:15 | |
| *** psedlak has quit IRC | 05:15 | |
| *** SergeyLukjanov has quit IRC | 05:15 | |
| *** NikitaKonovalov has quit IRC | 05:15 | |
| *** gaelL has quit IRC | 05:15 | |
| *** ptoohill has quit IRC | 05:15 | |
| *** sbadia has quit IRC | 05:15 | |
| *** clayg has quit IRC | 05:15 | |
| *** jerryz has quit IRC | 05:15 | |
| *** boris-42 has quit IRC | 05:15 | |
| *** bookwar has quit IRC | 05:15 | |
| *** morganfainberg has quit IRC | 05:15 | |
| *** briancline has quit IRC | 05:15 | |
| *** portante has quit IRC | 05:15 | |
| *** gema has quit IRC | 05:15 | |
| *** jogo has quit IRC | 05:15 | |
| *** yaguang has quit IRC | 05:15 | |
| *** srenatus has quit IRC | 05:15 | |
| *** penguinRaider has quit IRC | 05:15 | |
| *** jp_at_hp has quit IRC | 05:15 | |
| *** changbl has quit IRC | 05:15 | |
| *** fbo_away has quit IRC | 05:15 | |
| *** dkliban_afk has quit IRC | 05:15 | |
| *** tteggel has quit IRC | 05:15 | |
| *** social__ has quit IRC | 05:15 | |
| *** vipul has quit IRC | 05:15 | |
| *** medieval1 has quit IRC | 05:15 | |
| *** katyafervent has quit IRC | 05:15 | |
| *** enikanorov has quit IRC | 05:15 | |
| *** gilliard has quit IRC | 05:15 | |
| *** bnemec has quit IRC | 05:15 | |
| *** belliott has quit IRC | 05:15 | |
| *** jd__ has quit IRC | 05:15 | |
| *** Guest15213 has quit IRC | 05:15 | |
| *** sdague has quit IRC | 05:15 | |
| *** loquacities has quit IRC | 05:15 | |
| *** mugsie has quit IRC | 05:15 | |
| *** mnaser has quit IRC | 05:15 | |
| *** msuriar has quit IRC | 05:15 | |
| *** amotoki has quit IRC | 05:15 | |
| *** aconrad has quit IRC | 05:15 | |
| *** mkoderer has quit IRC | 05:15 | |
| *** simonmcc has quit IRC | 05:15 | |
| *** cody-somerville has quit IRC | 05:15 | |
| *** jeremyb has quit IRC | 05:15 | |
| *** pfalleno1 has quit IRC | 05:15 | |
| *** _afazekas_slp has quit IRC | 05:15 | |
| *** bogdando has quit IRC | 05:15 | |
| *** BadCub has quit IRC | 05:15 | |
| *** vponomaryov has quit IRC | 05:15 | |
| *** enikanorov_ has quit IRC | 05:15 | |
| *** esmute has quit IRC | 05:15 | |
| *** dguerri has quit IRC | 05:15 | |
| *** kmartin has quit IRC | 05:15 | |
| *** greghaynes has quit IRC | 05:15 | |
| *** gnuoy has quit IRC | 05:15 | |
| *** phschwartz has quit IRC | 05:15 | |
| *** timrc has quit IRC | 05:15 | |
| *** ogelbukh has quit IRC | 05:15 | |
| *** ryanpetrello has quit IRC | 05:15 | |
| *** Daviey has quit IRC | 05:15 | |
| *** nijaba has quit IRC | 05:15 | |
| *** ianw has quit IRC | 05:15 | |
| *** zns has quit IRC | 05:15 | |
| *** rcarrillocruz has quit IRC | 05:15 | |
| *** yfried has quit IRC | 05:15 | |
| *** tcammann has quit IRC | 05:15 | |
| *** jraim has quit IRC | 05:15 | |
| *** jcooley has quit IRC | 05:15 | |
| *** YorikSar has quit IRC | 05:15 | |
| *** jamespd has quit IRC | 05:15 | |
| *** StevenK has quit IRC | 05:15 | |
| *** ityaptin has quit IRC | 05:15 | |
| *** ruhe has quit IRC | 05:15 | |
| *** sileht has quit IRC | 05:15 | |
| *** zul has quit IRC | 05:15 | |
| *** BobBall has quit IRC | 05:15 | |
| *** jomara has quit IRC | 05:15 | |
| *** rpodolyaka has quit IRC | 05:15 | |
| *** gpocentek has quit IRC | 05:15 | |
| *** Clabbe has quit IRC | 05:15 | |
| *** d0ugal has quit IRC | 05:15 | |
| *** comstud has quit IRC | 05:15 | |
| *** nibalizer has quit IRC | 05:15 | |
| *** tchaypo has quit IRC | 05:15 | |
| *** adam_g has quit IRC | 05:15 | |
| *** pabelanger has quit IRC | 05:15 | |
| *** roaet_ has quit IRC | 05:15 | |
| *** basha has quit IRC | 05:15 | |
| *** matsuhashi has quit IRC | 05:15 | |
| *** ayoung has quit IRC | 05:15 | |
| *** ociuhandu has quit IRC | 05:15 | |
| *** openstackgerrit has quit IRC | 05:15 | |
| *** primeministerp has quit IRC | 05:15 | |
| *** kevinbenton has quit IRC | 05:15 | |
| *** mika has quit IRC | 05:15 | |
| *** mattoliverau has quit IRC | 05:15 | |
| *** zaro has quit IRC | 05:15 | |
| *** yamahata_ has quit IRC | 05:15 | |
| *** gothicmindfood has quit IRC | 05:15 | |
| *** ccorrigan has quit IRC | 05:15 | |
| *** med_ has quit IRC | 05:15 | |
| *** cgoncalves has quit IRC | 05:15 | |
| *** EntropyWorks has quit IRC | 05:15 | |
| *** asettle has quit IRC | 05:18 | |
| *** SpamapS has quit IRC | 05:21 | |
| *** dripton has quit IRC | 05:21 | |
| *** grantbow has quit IRC | 05:21 | |
| *** grantbow has joined #openstack-infra | 05:22 | |
| *** zns_ has joined #openstack-infra | 05:28 | |
| *** SpamapS has joined #openstack-infra | 05:28 | |
| *** dripton has joined #openstack-infra | 05:28 | |
| *** asettle-afk has joined #openstack-infra | 05:28 | |
| *** bookwar has joined #openstack-infra | 05:28 | |
| *** mattoliverau has joined #openstack-infra | 05:28 | |
| *** marcoemorais has joined #openstack-infra | 05:28 | |
| *** asadough1 has joined #openstack-infra | 05:28 | |
| *** cyeoh has joined #openstack-infra | 05:28 | |
| *** ianw has joined #openstack-infra | 05:28 | |
| *** trinaths has joined #openstack-infra | 05:28 | |
| *** amotoki has joined #openstack-infra | 05:28 | |
| *** HenryG has joined #openstack-infra | 05:28 | |
| *** aconrad has joined #openstack-infra | 05:28 | |
| *** matsuhashi has joined #openstack-infra | 05:28 | |
| *** julim has joined #openstack-infra | 05:28 | |
| *** wenlock has joined #openstack-infra | 05:28 | |
| *** unicell has joined #openstack-infra | 05:28 | |
| *** ayoung has joined #openstack-infra | 05:28 | |
| *** praneshp has joined #openstack-infra | 05:28 | |
| *** yaguang has joined #openstack-infra | 05:28 | |
| *** nati_ueno has joined #openstack-infra | 05:28 | |
| *** yamahata has joined #openstack-infra | 05:28 | |
| *** jerryz has joined #openstack-infra | 05:28 | |
| *** penguinRaider has joined #openstack-infra | 05:28 | |
| *** masayukig has joined #openstack-infra | 05:28 | |
| *** spiffxp has joined #openstack-infra | 05:28 | |
| *** mrodden has joined #openstack-infra | 05:28 | |
| *** ociuhandu has joined #openstack-infra | 05:28 | |
| *** devananda has joined #openstack-infra | 05:28 | |
| *** yjiang5 has joined #openstack-infra | 05:28 | |
| *** otherwiseguy has joined #openstack-infra | 05:28 | |
| *** adalbas has joined #openstack-infra | 05:28 | |
| *** krotscheck has joined #openstack-infra | 05:28 | |
| *** W00die has joined #openstack-infra | 05:28 | |
| *** gargola has joined #openstack-infra | 05:28 | |
| *** Alexei_987 has joined #openstack-infra | 05:28 | |
| *** marun has joined #openstack-infra | 05:28 | |
| *** ctracey has joined #openstack-infra | 05:28 | |
| *** srenatus has joined #openstack-infra | 05:28 | |
| *** dosaboy has joined #openstack-infra | 05:28 | |
| *** cyli has joined #openstack-infra | 05:28 | |
| *** asselin has joined #openstack-infra | 05:28 | |
| *** rcarrillocruz has joined #openstack-infra | 05:28 | |
| *** openstackgerrit has joined #openstack-infra | 05:28 | |
| *** yolanda has joined #openstack-infra | 05:28 | |
| *** achuprin has joined #openstack-infra | 05:28 | |
| *** primeministerp has joined #openstack-infra | 05:28 | |
| *** yfried has joined #openstack-infra | 05:28 | |
| *** d0ugal has joined #openstack-infra | 05:28 | |
| *** pcrews has joined #openstack-infra | 05:28 | |
| *** dmakogon_ has joined #openstack-infra | 05:28 | |
| *** rpodolyaka has joined #openstack-infra | 05:28 | |
| *** dkehn has joined #openstack-infra | 05:28 | |
| *** mkoderer has joined #openstack-infra | 05:28 | |
| *** viktors|afk has joined #openstack-infra | 05:28 | |
| *** tcammann has joined #openstack-infra | 05:28 | |
| *** jp_at_hp has joined #openstack-infra | 05:28 | |
| *** AaronGr has joined #openstack-infra | 05:28 | |
| *** simonmcc has joined #openstack-infra | 05:28 | |
| *** mancdaz has joined #openstack-infra | 05:28 | |
| *** jraim has joined #openstack-infra | 05:28 | |
| *** johnthetubaguy has joined #openstack-infra | 05:28 | |
| *** mdenny has joined #openstack-infra | 05:28 | |
| *** Hunner has joined #openstack-infra | 05:28 | |
| *** cody-somerville has joined #openstack-infra | 05:28 | |
| *** jcooley has joined #openstack-infra | 05:28 | |
| *** mwagner_lap has joined #openstack-infra | 05:28 | |
| *** YorikSar has joined #openstack-infra | 05:28 | |
| *** jamespd has joined #openstack-infra | 05:28 | |
| *** sabari has joined #openstack-infra | 05:28 | |
| *** hogepodge has joined #openstack-infra | 05:28 | |
| *** fifieldt_ has joined #openstack-infra | 05:28 | |
| *** ilyashakhat has joined #openstack-infra | 05:28 | |
| *** matrohon has joined #openstack-infra | 05:28 | |
| *** torgomatic has joined #openstack-infra | 05:28 | |
| *** sulochan has joined #openstack-infra | 05:28 | |
| *** StevenK has joined #openstack-infra | 05:28 | |
| *** rickerc has joined #openstack-infra | 05:28 | |
| *** changbl has joined #openstack-infra | 05:28 | |
| *** ityaptin has joined #openstack-infra | 05:28 | |
| *** bauzas has joined #openstack-infra | 05:28 | |
| *** Kiall has joined #openstack-infra | 05:28 | |
| *** kevinbenton has joined #openstack-infra | 05:28 | |
| *** rossella_s has joined #openstack-infra | 05:28 | |
| *** ruhe has joined #openstack-infra | 05:28 | |
| *** DinaBelova has joined #openstack-infra | 05:28 | |
| *** JayF has joined #openstack-infra | 05:28 | |
| *** jeremyb has joined #openstack-infra | 05:28 | |
| *** pfalleno1 has joined #openstack-infra | 05:28 | |
| *** mika has joined #openstack-infra | 05:28 | |
| *** sandywalsh has joined #openstack-infra | 05:28 | |
| *** _afazekas_slp has joined #openstack-infra | 05:28 | |
| *** gabriel-bezerra has joined #openstack-infra | 05:28 | |
| *** bogdando has joined #openstack-infra | 05:28 | |
| *** plars has joined #openstack-infra | 05:28 | |
| *** NobodyCam has joined #openstack-infra | 05:28 | |
| *** BadCub has joined #openstack-infra | 05:28 | |
| *** zaro has joined #openstack-infra | 05:28 | |
| *** sileht has joined #openstack-infra | 05:28 | |
| *** zul has joined #openstack-infra | 05:28 | |
| *** social__ has joined #openstack-infra | 05:28 | |
| *** BobBall has joined #openstack-infra | 05:28 | |
| *** fbo_away has joined #openstack-infra | 05:28 | |
| *** vponomaryov has joined #openstack-infra | 05:28 | |
| *** shardy has joined #openstack-infra | 05:28 | |
| *** locke105 has joined #openstack-infra | 05:28 | |
| *** dkliban_afk has joined #openstack-infra | 05:28 | |
| *** jomara has joined #openstack-infra | 05:28 | |
| *** fabo has joined #openstack-infra | 05:28 | |
| *** enikanorov_ has joined #openstack-infra | 05:28 | |
| *** gfa has joined #openstack-infra | 05:28 | |
| *** gpocentek has joined #openstack-infra | 05:28 | |
| *** yamahata_ has joined #openstack-infra | 05:28 | |
| *** jesusaurus has joined #openstack-infra | 05:28 | |
| *** tteggel has joined #openstack-infra | 05:28 | |
| *** jeblair has joined #openstack-infra | 05:28 | |
| *** gothicmindfood has joined #openstack-infra | 05:28 | |
| *** sdague has joined #openstack-infra | 05:28 | |
| *** belliott has joined #openstack-infra | 05:28 | |
| *** esmute has joined #openstack-infra | 05:28 | |
| *** dansmith has joined #openstack-infra | 05:28 | |
| *** dguerri has joined #openstack-infra | 05:28 | |
| *** vipul has joined #openstack-infra | 05:28 | |
| *** kmartin has joined #openstack-infra | 05:28 | |
| *** dougwig has joined #openstack-infra | 05:28 | |
| *** ccorrigan has joined #openstack-infra | 05:28 | |
| *** isviridov|away has joined #openstack-infra | 05:28 | |
| *** medieval1 has joined #openstack-infra | 05:28 | |
| *** greghaynes has joined #openstack-infra | 05:28 | |
| *** ekarlso has joined #openstack-infra | 05:28 | |
| *** boris-42 has joined #openstack-infra | 05:28 | |
| *** gnuoy has joined #openstack-infra | 05:28 | |
| *** Clabbe has joined #openstack-infra | 05:28 | |
| *** med_ has joined #openstack-infra | 05:28 | |
| *** katyafervent has joined #openstack-infra | 05:28 | |
| *** kashyap has joined #openstack-infra | 05:28 | |
| *** enikanorov has joined #openstack-infra | 05:28 | |
| *** pleia2 has joined #openstack-infra | 05:28 | |
| *** timfreund has joined #openstack-infra | 05:28 | |
| *** jpeeler has joined #openstack-infra | 05:28 | |
| *** therve has joined #openstack-infra | 05:28 | |
| *** serverascode has joined #openstack-infra | 05:28 | |
| *** dangers_away has joined #openstack-infra | 05:28 | |
| *** gilliard has joined #openstack-infra | 05:28 | |
| *** bnemec has joined #openstack-infra | 05:28 | |
| *** NithyaG_afk has joined #openstack-infra | 05:28 | |
| *** phschwartz has joined #openstack-infra | 05:28 | |
| *** hughsaunders has joined #openstack-infra | 05:28 | |
| *** portante has joined #openstack-infra | 05:28 | |
| *** timrc has joined #openstack-infra | 05:28 | |
| *** cgoncalves has joined #openstack-infra | 05:28 | |
| *** gema has joined #openstack-infra | 05:28 | |
| *** jamielennox has joined #openstack-infra | 05:28 | |
| *** flaper87|afk has joined #openstack-infra | 05:28 | |
| *** soren has joined #openstack-infra | 05:28 | |
| *** lifeless has joined #openstack-infra | 05:28 | |
| *** skraynev has joined #openstack-infra | 05:28 | |
| *** ogelbukh has joined #openstack-infra | 05:28 | |
| *** jd__ has joined #openstack-infra | 05:28 | |
| *** Guest15213 has joined #openstack-infra | 05:28 | |
| *** pbelanyi has joined #openstack-infra | 05:28 | |
| *** Alex_Gaynor has joined #openstack-infra | 05:28 | |
| *** vladan has joined #openstack-infra | 05:28 | |
| *** mtreinish has joined #openstack-infra | 05:28 | |
| *** comstud has joined #openstack-infra | 05:28 | |
| *** EntropyWorks has joined #openstack-infra | 05:28 | |
| *** ryanpetrello has joined #openstack-infra | 05:28 | |
| *** dtroyer has joined #openstack-infra | 05:28 | |
| *** nibalizer has joined #openstack-infra | 05:28 | |
| *** jogo has joined #openstack-infra | 05:28 | |
| *** tchaypo has joined #openstack-infra | 05:28 | |
| *** ChanServ has joined #openstack-infra | 05:28 | |
| *** Ng has joined #openstack-infra | 05:28 | |
| *** morganfainberg has joined #openstack-infra | 05:28 | |
| *** psedlak has joined #openstack-infra | 05:28 | |
| *** mikal has joined #openstack-infra | 05:28 | |
| *** Shrews has joined #openstack-infra | 05:28 | |
| *** sbadia has joined #openstack-infra | 05:28 | |
| *** gaelL has joined #openstack-infra | 05:28 | |
| *** NikitaKonovalov has joined #openstack-infra | 05:28 | |
| *** SergeyLukjanov has joined #openstack-infra | 05:28 | |
| *** pabelanger has joined #openstack-infra | 05:28 | |
| *** roaet_ has joined #openstack-infra | 05:28 | |
| *** dickson.freenode.net sets mode: +o ChanServ | 05:28 | |
| *** msuriar has joined #openstack-infra | 05:28 | |
| *** adam_g has joined #openstack-infra | 05:28 | |
| *** mugsie has joined #openstack-infra | 05:28 | |
| *** nijaba has joined #openstack-infra | 05:28 | |
| *** loquacities has joined #openstack-infra | 05:28 | |
| *** mnaser has joined #openstack-infra | 05:28 | |
| *** ptoohill has joined #openstack-infra | 05:28 | |
| *** Daviey has joined #openstack-infra | 05:28 | |
| *** clayg has joined #openstack-infra | 05:28 | |
| *** dripton has quit IRC | 05:28 | |
| *** mattoliverau has quit IRC | 05:28 | |
| *** trinaths has quit IRC | 05:28 | |
| *** julim has quit IRC | 05:28 | |
| *** praneshp has quit IRC | 05:28 | |
| *** yamahata has quit IRC | 05:28 | |
| *** adalbas has quit IRC | 05:28 | |
| *** AaronGr has quit IRC | 05:28 | |
| *** Hunner has quit IRC | 05:28 | |
| *** dmakogon_ has quit IRC | 05:28 | |
| *** torgomatic has quit IRC | 05:28 | |
| *** unicell has quit IRC | 05:28 | |
| *** nati_ueno has quit IRC | 05:28 | |
| *** otherwiseguy has quit IRC | 05:28 | |
| *** dosaboy has quit IRC | 05:28 | |
| *** cyli has quit IRC | 05:28 | |
| *** mdenny has quit IRC | 05:28 | |
| *** mwagner_lap has quit IRC | 05:28 | |
| *** fifieldt_ has quit IRC | 05:28 | |
| *** matrohon has quit IRC | 05:28 | |
| *** gabriel-bezerra has quit IRC | 05:28 | |
| *** plars has quit IRC | 05:28 | |
| *** isviridov|away has quit IRC | 05:28 | |
| *** SpamapS has quit IRC | 05:28 | |
| *** masayukig has quit IRC | 05:28 | |
| *** spiffxp has quit IRC | 05:28 | |
| *** devananda has quit IRC | 05:28 | |
| *** krotscheck has quit IRC | 05:28 | |
| *** gargola has quit IRC | 05:28 | |
| *** Alexei_987 has quit IRC | 05:28 | |
| *** marun has quit IRC | 05:28 | |
| *** achuprin has quit IRC | 05:28 | |
| *** mancdaz has quit IRC | 05:28 | |
| *** johnthetubaguy has quit IRC | 05:28 | |
| *** ilyashakhat has quit IRC | 05:28 | |
| *** viktors|afk has quit IRC | 05:28 | |
| *** bauzas has quit IRC | 05:28 | |
| *** rossella_s has quit IRC | 05:28 | |
| *** DinaBelova has quit IRC | 05:28 | |
| *** JayF has quit IRC | 05:28 | |
| *** sandywalsh has quit IRC | 05:28 | |
| *** gfa has quit IRC | 05:28 | |
| *** marcoemorais has quit IRC | 05:28 | |
| *** mrodden has quit IRC | 05:28 | |
| *** ctracey has quit IRC | 05:28 | |
| *** asselin has quit IRC | 05:28 | |
| *** yolanda has quit IRC | 05:28 | |
| *** pcrews has quit IRC | 05:28 | |
| *** sabari has quit IRC | 05:28 | |
| *** hogepodge has quit IRC | 05:28 | |
| *** rickerc has quit IRC | 05:28 | |
| *** sulochan has quit IRC | 05:28 | |
| *** dougwig has quit IRC | 05:28 | |
| *** serverascode has quit IRC | 05:28 | |
| *** jamielennox has quit IRC | 05:28 | |
| *** skraynev has quit IRC | 05:28 | |
| *** ChanServ has quit IRC | 05:28 | |
| *** HenryG has quit IRC | 05:28 | |
| *** wenlock has quit IRC | 05:28 | |
| *** yjiang5 has quit IRC | 05:28 | |
| *** W00die has quit IRC | 05:28 | |
| *** dkehn has quit IRC | 05:28 | |
| *** Kiall has quit IRC | 05:28 | |
| *** NobodyCam has quit IRC | 05:28 | |
| *** locke105 has quit IRC | 05:28 | |
| *** jesusaurus has quit IRC | 05:28 | |
| *** jeblair has quit IRC | 05:28 | |
| *** dansmith has quit IRC | 05:28 | |
| *** ekarlso has quit IRC | 05:28 | |
| *** timfreund has quit IRC | 05:28 | |
| *** jpeeler has quit IRC | 05:28 | |
| *** dangers_away has quit IRC | 05:28 | |
| *** NithyaG_afk has quit IRC | 05:28 | |
| *** flaper87|afk has quit IRC | 05:28 | |
| *** soren has quit IRC | 05:28 | |
| *** lifeless has quit IRC | 05:28 | |
| *** pbelanyi has quit IRC | 05:28 | |
| *** vladan has quit IRC | 05:28 | |
| *** mtreinish has quit IRC | 05:28 | |
| *** dtroyer has quit IRC | 05:28 | |
| *** fabo has quit IRC | 05:28 | |
| *** Shrews has quit IRC | 05:28 | |
| *** mikal has quit IRC | 05:28 | |
| *** shardy has quit IRC | 05:28 | |
| *** kashyap has quit IRC | 05:28 | |
| *** pleia2 has quit IRC | 05:28 | |
| *** therve has quit IRC | 05:28 | |
| *** hughsaunders has quit IRC | 05:28 | |
| *** Alex_Gaynor has quit IRC | 05:28 | |
| *** Ng has quit IRC | 05:28 | |
| *** psedlak has quit IRC | 05:28 | |
| *** SergeyLukjanov has quit IRC | 05:28 | |
| *** NikitaKonovalov has quit IRC | 05:28 | |
| *** gaelL has quit IRC | 05:28 | |
| *** ptoohill has quit IRC | 05:28 | |
| *** sbadia has quit IRC | 05:28 | |
| *** clayg has quit IRC | 05:28 | |
| *** jerryz has quit IRC | 05:28 | |
| *** bookwar has quit IRC | 05:28 | |
| *** boris-42 has quit IRC | 05:28 | |
| *** morganfainberg has quit IRC | 05:28 | |
| *** portante has quit IRC | 05:28 | |
| *** gema has quit IRC | 05:28 | |
| *** jogo has quit IRC | 05:28 | |
| *** asadough1 has quit IRC | 05:28 | |
| *** yaguang has quit IRC | 05:28 | |
| *** srenatus has quit IRC | 05:28 | |
| *** penguinRaider has quit IRC | 05:28 | |
| *** jp_at_hp has quit IRC | 05:28 | |
| *** changbl has quit IRC | 05:28 | |
| *** fbo_away has quit IRC | 05:28 | |
| *** dkliban_afk has quit IRC | 05:28 | |
| *** tteggel has quit IRC | 05:28 | |
| *** social__ has quit IRC | 05:28 | |
| *** vipul has quit IRC | 05:28 | |
| *** medieval1 has quit IRC | 05:28 | |
| *** katyafervent has quit IRC | 05:28 | |
| *** enikanorov has quit IRC | 05:28 | |
| *** gilliard has quit IRC | 05:28 | |
| *** bnemec has quit IRC | 05:28 | |
| *** belliott has quit IRC | 05:28 | |
| *** jd__ has quit IRC | 05:28 | |
| *** Guest15213 has quit IRC | 05:28 | |
| *** sdague has quit IRC | 05:28 | |
| *** loquacities has quit IRC | 05:28 | |
| *** mugsie has quit IRC | 05:28 | |
| *** mnaser has quit IRC | 05:28 | |
| *** msuriar has quit IRC | 05:28 | |
| *** zns_ has quit IRC | 05:28 | |
| *** cyeoh has quit IRC | 05:28 | |
| *** amotoki has quit IRC | 05:28 | |
| *** aconrad has quit IRC | 05:28 | |
| *** mkoderer has quit IRC | 05:28 | |
| *** simonmcc has quit IRC | 05:28 | |
| *** cody-somerville has quit IRC | 05:28 | |
| *** jeremyb has quit IRC | 05:28 | |
| *** pfalleno1 has quit IRC | 05:28 | |
| *** _afazekas_slp has quit IRC | 05:28 | |
| *** bogdando has quit IRC | 05:28 | |
| *** BadCub has quit IRC | 05:28 | |
| *** vponomaryov has quit IRC | 05:28 | |
| *** enikanorov_ has quit IRC | 05:28 | |
| *** esmute has quit IRC | 05:28 | |
| *** dguerri has quit IRC | 05:28 | |
| *** kmartin has quit IRC | 05:28 | |
| *** greghaynes has quit IRC | 05:28 | |
| *** gnuoy has quit IRC | 05:28 | |
| *** phschwartz has quit IRC | 05:28 | |
| *** timrc has quit IRC | 05:28 | |
| *** ogelbukh has quit IRC | 05:28 | |
| *** ryanpetrello has quit IRC | 05:28 | |
| *** Daviey has quit IRC | 05:28 | |
| *** nijaba has quit IRC | 05:28 | |
| *** ianw has quit IRC | 05:28 | |
| *** rcarrillocruz has quit IRC | 05:28 | |
| *** yfried has quit IRC | 05:28 | |
| *** tcammann has quit IRC | 05:28 | |
| *** jraim has quit IRC | 05:28 | |
| *** jcooley has quit IRC | 05:28 | |
| *** YorikSar has quit IRC | 05:28 | |
| *** jamespd has quit IRC | 05:28 | |
| *** StevenK has quit IRC | 05:28 | |
| *** ityaptin has quit IRC | 05:28 | |
| *** ruhe has quit IRC | 05:28 | |
| *** sileht has quit IRC | 05:28 | |
| *** zul has quit IRC | 05:28 | |
| *** BobBall has quit IRC | 05:28 | |
| *** jomara has quit IRC | 05:28 | |
| *** rpodolyaka has quit IRC | 05:28 | |
| *** gpocentek has quit IRC | 05:28 | |
| *** Clabbe has quit IRC | 05:28 | |
| *** d0ugal has quit IRC | 05:28 | |
| *** comstud has quit IRC | 05:28 | |
| *** nibalizer has quit IRC | 05:28 | |
| *** tchaypo has quit IRC | 05:28 | |
| *** adam_g has quit IRC | 05:28 | |
| *** pabelanger has quit IRC | 05:28 | |
| *** roaet_ has quit IRC | 05:28 | |
| *** asettle-afk has quit IRC | 05:28 | |
| *** matsuhashi has quit IRC | 05:28 | |
| *** ayoung has quit IRC | 05:28 | |
| *** ociuhandu has quit IRC | 05:28 | |
| *** openstackgerrit has quit IRC | 05:28 | |
| *** primeministerp has quit IRC | 05:28 | |
| *** kevinbenton has quit IRC | 05:28 | |
| *** mika has quit IRC | 05:28 | |
| *** zaro has quit IRC | 05:28 | |
| *** yamahata_ has quit IRC | 05:28 | |
| *** gothicmindfood has quit IRC | 05:28 | |
| *** ccorrigan has quit IRC | 05:28 | |
| *** med_ has quit IRC | 05:28 | |
| *** cgoncalves has quit IRC | 05:28 | |
| *** EntropyWorks has quit IRC | 05:28 | |
| *** basha has joined #openstack-infra | 05:34 | |
| *** SpamapS has joined #openstack-infra | 05:34 | |
| *** dripton has joined #openstack-infra | 05:34 | |
| *** asettle-afk has joined #openstack-infra | 05:34 | |
| *** bookwar has joined #openstack-infra | 05:34 | |
| *** mattoliverau has joined #openstack-infra | 05:34 | |
| *** marcoemorais has joined #openstack-infra | 05:34 | |
| *** asadough1 has joined #openstack-infra | 05:34 | |
| *** cyeoh has joined #openstack-infra | 05:34 | |
| *** ianw has joined #openstack-infra | 05:34 | |
| *** trinaths has joined #openstack-infra | 05:34 | |
| *** amotoki has joined #openstack-infra | 05:34 | |
| *** HenryG has joined #openstack-infra | 05:34 | |
| *** aconrad has joined #openstack-infra | 05:34 | |
| *** matsuhashi has joined #openstack-infra | 05:34 | |
| *** julim has joined #openstack-infra | 05:34 | |
| *** wenlock has joined #openstack-infra | 05:34 | |
| *** unicell has joined #openstack-infra | 05:34 | |
| *** ayoung has joined #openstack-infra | 05:34 | |
| *** praneshp has joined #openstack-infra | 05:34 | |
| *** yaguang has joined #openstack-infra | 05:34 | |
| *** nati_ueno has joined #openstack-infra | 05:34 | |
| *** yamahata has joined #openstack-infra | 05:34 | |
| *** jerryz has joined #openstack-infra | 05:34 | |
| *** penguinRaider has joined #openstack-infra | 05:34 | |
| *** masayukig has joined #openstack-infra | 05:34 | |
| *** spiffxp has joined #openstack-infra | 05:34 | |
| *** mrodden has joined #openstack-infra | 05:34 | |
| *** ociuhandu has joined #openstack-infra | 05:34 | |
| *** devananda has joined #openstack-infra | 05:34 | |
| *** yjiang5 has joined #openstack-infra | 05:34 | |
| *** otherwiseguy has joined #openstack-infra | 05:34 | |
| *** adalbas has joined #openstack-infra | 05:34 | |
| *** krotscheck has joined #openstack-infra | 05:34 | |
| *** W00die has joined #openstack-infra | 05:34 | |
| *** gargola has joined #openstack-infra | 05:34 | |
| *** Alexei_987 has joined #openstack-infra | 05:34 | |
| *** marun has joined #openstack-infra | 05:34 | |
| *** ctracey has joined #openstack-infra | 05:34 | |
| *** srenatus has joined #openstack-infra | 05:34 | |
| *** dosaboy has joined #openstack-infra | 05:34 | |
| *** cyli has joined #openstack-infra | 05:34 | |
| *** asselin has joined #openstack-infra | 05:34 | |
| *** rcarrillocruz has joined #openstack-infra | 05:34 | |
| *** openstackgerrit has joined #openstack-infra | 05:34 | |
| *** yolanda has joined #openstack-infra | 05:34 | |
| *** achuprin has joined #openstack-infra | 05:34 | |
| *** primeministerp has joined #openstack-infra | 05:34 | |
| *** yfried has joined #openstack-infra | 05:34 | |
| *** d0ugal has joined #openstack-infra | 05:34 | |
| *** pcrews has joined #openstack-infra | 05:34 | |
| *** dmakogon_ has joined #openstack-infra | 05:34 | |
| *** rpodolyaka has joined #openstack-infra | 05:34 | |
| *** dkehn has joined #openstack-infra | 05:34 | |
| *** mkoderer has joined #openstack-infra | 05:34 | |
| *** viktors|afk has joined #openstack-infra | 05:34 | |
| *** tcammann has joined #openstack-infra | 05:34 | |
| *** jp_at_hp has joined #openstack-infra | 05:34 | |
| *** AaronGr has joined #openstack-infra | 05:34 | |
| *** simonmcc has joined #openstack-infra | 05:34 | |
| *** mancdaz has joined #openstack-infra | 05:34 | |
| *** jraim has joined #openstack-infra | 05:34 | |
| *** johnthetubaguy has joined #openstack-infra | 05:34 | |
| *** mdenny has joined #openstack-infra | 05:34 | |
| *** Hunner has joined #openstack-infra | 05:34 | |
| *** cody-somerville has joined #openstack-infra | 05:34 | |
| *** jcooley has joined #openstack-infra | 05:34 | |
| *** mwagner_lap has joined #openstack-infra | 05:34 | |
| *** YorikSar has joined #openstack-infra | 05:34 | |
| *** jamespd has joined #openstack-infra | 05:34 | |
| *** sabari has joined #openstack-infra | 05:34 | |
| *** hogepodge has joined #openstack-infra | 05:34 | |
| *** fifieldt_ has joined #openstack-infra | 05:34 | |
| *** ilyashakhat has joined #openstack-infra | 05:34 | |
| *** matrohon has joined #openstack-infra | 05:34 | |
| *** torgomatic has joined #openstack-infra | 05:34 | |
| *** sulochan has joined #openstack-infra | 05:34 | |
| *** StevenK has joined #openstack-infra | 05:34 | |
| *** rickerc has joined #openstack-infra | 05:34 | |
| *** changbl has joined #openstack-infra | 05:34 | |
| *** ityaptin has joined #openstack-infra | 05:34 | |
| *** bauzas has joined #openstack-infra | 05:34 | |
| *** Kiall has joined #openstack-infra | 05:34 | |
| *** kevinbenton has joined #openstack-infra | 05:34 | |
| *** rossella_s has joined #openstack-infra | 05:34 | |
| *** ruhe has joined #openstack-infra | 05:34 | |
| *** DinaBelova has joined #openstack-infra | 05:34 | |
| *** JayF has joined #openstack-infra | 05:34 | |
| *** jeremyb has joined #openstack-infra | 05:34 | |
| *** pfalleno1 has joined #openstack-infra | 05:34 | |
| *** mika has joined #openstack-infra | 05:34 | |
| *** sandywalsh has joined #openstack-infra | 05:34 | |
| *** _afazekas_slp has joined #openstack-infra | 05:34 | |
| *** gabriel-bezerra has joined #openstack-infra | 05:34 | |
| *** bogdando has joined #openstack-infra | 05:34 | |
| *** plars has joined #openstack-infra | 05:34 | |
| *** NobodyCam has joined #openstack-infra | 05:34 | |
| *** BadCub has joined #openstack-infra | 05:34 | |
| *** zaro has joined #openstack-infra | 05:34 | |
| *** sileht has joined #openstack-infra | 05:34 | |
| *** zul has joined #openstack-infra | 05:34 | |
| *** social__ has joined #openstack-infra | 05:34 | |
| *** BobBall has joined #openstack-infra | 05:34 | |
| *** fbo_away has joined #openstack-infra | 05:34 | |
| *** vponomaryov has joined #openstack-infra | 05:34 | |
| *** shardy has joined #openstack-infra | 05:34 | |
| *** locke105 has joined #openstack-infra | 05:34 | |
| *** dkliban_afk has joined #openstack-infra | 05:34 | |
| *** jomara has joined #openstack-infra | 05:34 | |
| *** fabo has joined #openstack-infra | 05:34 | |
| *** enikanorov_ has joined #openstack-infra | 05:34 | |
| *** gfa has joined #openstack-infra | 05:34 | |
| *** gpocentek has joined #openstack-infra | 05:34 | |
| *** yamahata_ has joined #openstack-infra | 05:34 | |
| *** jesusaurus has joined #openstack-infra | 05:34 | |
| *** tteggel has joined #openstack-infra | 05:34 | |
| *** jeblair has joined #openstack-infra | 05:34 | |
| *** gothicmindfood has joined #openstack-infra | 05:34 | |
| *** sdague has joined #openstack-infra | 05:34 | |
| *** belliott has joined #openstack-infra | 05:34 | |
| *** esmute has joined #openstack-infra | 05:34 | |
| *** dansmith has joined #openstack-infra | 05:34 | |
| *** dguerri has joined #openstack-infra | 05:34 | |
| *** vipul has joined #openstack-infra | 05:34 | |
| *** kmartin has joined #openstack-infra | 05:34 | |
| *** dougwig has joined #openstack-infra | 05:34 | |
| *** ccorrigan has joined #openstack-infra | 05:34 | |
| *** isviridov|away has joined #openstack-infra | 05:34 | |
| *** medieval1 has joined #openstack-infra | 05:34 | |
| *** greghaynes has joined #openstack-infra | 05:34 | |
| *** ekarlso has joined #openstack-infra | 05:34 | |
| *** boris-42 has joined #openstack-infra | 05:34 | |
| *** gnuoy has joined #openstack-infra | 05:34 | |
| *** Clabbe has joined #openstack-infra | 05:34 | |
| *** med_ has joined #openstack-infra | 05:34 | |
| *** katyafervent has joined #openstack-infra | 05:34 | |
| *** kashyap has joined #openstack-infra | 05:34 | |
| *** enikanorov has joined #openstack-infra | 05:34 | |
| *** pleia2 has joined #openstack-infra | 05:34 | |
| *** timfreund has joined #openstack-infra | 05:34 | |
| *** jpeeler has joined #openstack-infra | 05:34 | |
| *** therve has joined #openstack-infra | 05:34 | |
| *** serverascode has joined #openstack-infra | 05:34 | |
| *** dangers_away has joined #openstack-infra | 05:34 | |
| *** gilliard has joined #openstack-infra | 05:34 | |
| *** bnemec has joined #openstack-infra | 05:34 | |
| *** NithyaG_afk has joined #openstack-infra | 05:34 | |
| *** phschwartz has joined #openstack-infra | 05:34 | |
| *** hughsaunders has joined #openstack-infra | 05:34 | |
| *** portante has joined #openstack-infra | 05:34 | |
| *** timrc has joined #openstack-infra | 05:34 | |
| *** cgoncalves has joined #openstack-infra | 05:34 | |
| *** gema has joined #openstack-infra | 05:34 | |
| *** jamielennox has joined #openstack-infra | 05:34 | |
| *** flaper87|afk has joined #openstack-infra | 05:34 | |
| *** soren has joined #openstack-infra | 05:34 | |
| *** lifeless has joined #openstack-infra | 05:34 | |
| *** skraynev has joined #openstack-infra | 05:34 | |
| *** ogelbukh has joined #openstack-infra | 05:34 | |
| *** jd__ has joined #openstack-infra | 05:34 | |
| *** Guest15213 has joined #openstack-infra | 05:34 | |
| *** pbelanyi has joined #openstack-infra | 05:34 | |
| *** Alex_Gaynor has joined #openstack-infra | 05:34 | |
| *** vladan has joined #openstack-infra | 05:34 | |
| *** mtreinish has joined #openstack-infra | 05:34 | |
| *** comstud has joined #openstack-infra | 05:34 | |
| *** EntropyWorks has joined #openstack-infra | 05:34 | |
| *** ryanpetrello has joined #openstack-infra | 05:34 | |
| *** dtroyer has joined #openstack-infra | 05:34 | |
| *** nibalizer has joined #openstack-infra | 05:34 | |
| *** jogo has joined #openstack-infra | 05:34 | |
| *** tchaypo has joined #openstack-infra | 05:34 | |
| *** ChanServ has joined #openstack-infra | 05:34 | |
| *** Ng has joined #openstack-infra | 05:34 | |
| *** morganfainberg has joined #openstack-infra | 05:34 | |
| *** psedlak has joined #openstack-infra | 05:34 | |
| *** mikal has joined #openstack-infra | 05:34 | |
| *** Shrews has joined #openstack-infra | 05:34 | |
| *** sbadia has joined #openstack-infra | 05:34 | |
| *** gaelL has joined #openstack-infra | 05:34 | |
| *** NikitaKonovalov has joined #openstack-infra | 05:34 | |
| *** SergeyLukjanov has joined #openstack-infra | 05:34 | |
| *** pabelanger has joined #openstack-infra | 05:34 | |
| *** roaet_ has joined #openstack-infra | 05:34 | |
| *** dickson.freenode.net sets mode: +o ChanServ | 05:34 | |
| *** msuriar has joined #openstack-infra | 05:34 | |
| *** adam_g has joined #openstack-infra | 05:34 | |
| *** mugsie has joined #openstack-infra | 05:34 | |
| *** nijaba has joined #openstack-infra | 05:34 | |
| *** loquacities has joined #openstack-infra | 05:34 | |
| *** mnaser has joined #openstack-infra | 05:34 | |
| *** ptoohill has joined #openstack-infra | 05:34 | |
| *** Daviey has joined #openstack-infra | 05:34 | |
| *** clayg has joined #openstack-infra | 05:34 | |
| *** briancli1e is now known as briancline | 05:34 | |
| *** grantbow has quit IRC | 05:34 | |
| boris-42 | jhesketh could you pls take a look one more time on my patch https://review.openstack.org/#/c/97778/ I addressed your comment | 05:34 |
| boris-42 | jhesketh thanks | 05:34 |
| *** starmer has joined #openstack-infra | 05:35 | |
| *** grantbow has joined #openstack-infra | 05:36 | |
| *** grantbow has quit IRC | 05:36 | |
| *** grantbow has joined #openstack-infra | 05:36 | |
| jhesketh | boris-42: yep, will do | 05:37 |
| boris-42 | jhesketh thank you, after openstack CI, i just forgot how to work on github =( | 05:38 |
| jhesketh | haha, same here.. it makes me realise how much better openstack's process is though | 05:39 |
| *** starmer has quit IRC | 05:40 | |
| *** doude has joined #openstack-infra | 05:41 | |
| *** praneshp has quit IRC | 05:44 | |
| *** starmer has joined #openstack-infra | 05:50 | |
| *** jp_at_hp has quit IRC | 05:53 | |
| *** unicell has quit IRC | 05:53 | |
| *** starmer has quit IRC | 05:55 | |
| openstackgerrit | Christian Berendt proposed a change to openstack-dev/hacking: Add H804: enforce title case in commit messages https://review.openstack.org/98059 | 06:00 |
| tchaypo | i lolled | 06:02 |
| clarkb | tchaypo: its rules like that why we don't use hacking round here | 06:03 |
| tchaypo | and then I realised that the commit message does not use title case | 06:04 |
| tchaypo | and i lolled even harder | 06:04 |
| tchaypo | and then i read the full commit message where it says that they only want to enforce the title case *for the first word* of the commit message. | 06:05 |
| tchaypo | so no more messages that start with "UNDERCLOUD_IP needs to be set before it's used" I guess | 06:05 |
| * clarkb goes to leave a nice comment | 06:05 | |
| *** starmer has joined #openstack-infra | 06:05 | |
| *** maxbit has quit IRC | 06:06 | |
| YorikSar | fungi, clarkb: I don't see any particular benefit from making OpenStack-specific feature non-OpenStack-specific... | 06:06 |
| *** maxbit has joined #openstack-infra | 06:06 | |
| YorikSar | fungi, clarkb: I'll review logs from infra meeting to see if anything there will convince me. | 06:07 |
| ianw | tchaypo: i wonder if somebody can write a "commit message should make this understandable to people who did not write the patch" check | 06:08 |
| clarkb | YorikSar: mostly so that git review doesn't get spun off into a corner | 06:08 |
| clarkb | YorikSar: I like the idea of making -D do special things if you want | 06:08 |
| *** _nadya_ has joined #openstack-infra | 06:08 | |
| *** sc68cal_ has joined #openstack-infra | 06:09 | |
| *** sc68cal has quit IRC | 06:09 | |
| YorikSar | clarkb: But it's not draft... It'll make people think "ok, I'll upload smth so that people won't see it intentionaly and I won't get embarased of bubble sort in that method" but essentially it'll just get uploaded for everyone to see. | 06:10 |
| *** starmer has quit IRC | 06:10 | |
| clarkb | YorikSar: so you should never think that about drafts | 06:10 |
| clarkb | because everyone can see drafts | 06:10 |
| clarkb | this is why we disabled drafts | 06:10 |
| *** praneshp has joined #openstack-infra | 06:10 | |
| clarkb | and have essentially replaced them with WIP | 06:10 |
| YorikSar | clarkb: I know it. But noone will see draft unless one want it ;) | 06:10 |
| *** ildikov has joined #openstack-infra | 06:11 | |
| clarkb | so I don't think its crazy to let the -D behavior be configurable | 06:11 |
| *** andreykurilin_ has joined #openstack-infra | 06:12 | |
| kevinbenton | can this patch get bumped to the top of the queue? It’s supposed to address the recent surge in SSH timeout failures | 06:13 |
| kevinbenton | https://review.openstack.org/#/c/97245/ | 06:13 |
| clarkb | kevinbenton: it cannot be, it hasn't gone into the gate yet | 06:14 |
| YorikSar | clarkb: Ok... From the meeting log it's not clear where -D came from. | 06:15 |
| kevinbenton | clarkb: ok. i just started the recheck on it. after it passes verification can i ping you to bump it up? | 06:15 |
| clarkb | kevinbenton: not me I am about to fall asleep | 06:16 |
| clarkb | SergeyLukjanov: should be able to help though | 06:16 |
| YorikSar | clarkb: May be we should leave -D alone and allow -W to be configurable? | 06:16 |
| clarkb | YorikSar: that was my suggestion | 06:16 |
| tchaypo | ianw: it's funny you should say that, check out the message I wrote before I saw your comment :) | 06:16 |
| kevinbenton | clarkb: sounds good | 06:16 |
| clarkb | SergeyLukjanov: you just run the `zuul promote --pipeline gate --changes 97245,2` command on zuul.o.o | 06:17 |
| clarkb | SergeyLukjanov: you might need to do it as root so that it can read the zuul.conf | 06:17 |
| YorikSar | clarkb: Ok. I won't keep you from falling asleep. Let's discuss it more at night (my time) :) | 06:18 |
| clarkb | YorikSar: I am happy with -W being configurable too | 06:18 |
| clarkb | either way works for me | 06:18 |
| YorikSar | clarkb: I guess I won't be touching it on this week anyway. I haven't given any love to by rootwrap daemon baby for a long time... | 06:19 |
| *** tkelsey has joined #openstack-infra | 06:20 | |
| *** starmer has joined #openstack-infra | 06:20 | |
| SlickNik | Would appreciate it if I could get some eyeballs on https://review.openstack.org/#/c/97608/ when folks get a chance. Much thanks! | 06:21 |
| *** starmer has quit IRC | 06:25 | |
| *** sc68cal_ has quit IRC | 06:26 | |
| * SergeyLukjanov here | 06:26 | |
| SergeyLukjanov | clarkb, yup, I know, already done it about 10 times yesterday :) | 06:27 |
| openstackgerrit | Christian Berendt proposed a change to openstack-dev/hacking: Add H804: first word of commit message should be capitalized https://review.openstack.org/98059 | 06:27 |
| SergeyLukjanov | clarkb, and we've been used zuul enqueue to move jobs to the gate pipeline :) | 06:29 |
| *** _nadya_ has quit IRC | 06:29 | |
| *** sc68cal has joined #openstack-infra | 06:29 | |
| SergeyLukjanov | kevinbenton, so, the https://review.openstack.org/#/c/97245/ should fix Bug 1323658 - SSH EOFError - Public network connectivity check failed | 06:31 |
| uvirtbot | Launchpad bug 1323658 in neutron "SSH EOFError - Public network connectivity check failed" [Undecided,New] https://launchpad.net/bugs/1323658 | 06:31 |
| *** Ryan_Lane has joined #openstack-infra | 06:31 | |
| *** Longgeek has quit IRC | 06:31 | |
| kevinbenton | SergeyLukjanov: not quite. it should just reduce it back down to a much rarer event | 06:31 |
| kevinbenton | SergeyLukjanov: people on the neutron team are still working on figuring out the root cause | 06:31 |
| openstackgerrit | Christian Berendt proposed a change to openstack-dev/hacking: Add H804: first word of commit message should be capitalized https://review.openstack.org/98059 | 06:31 |
| openstackgerrit | Joshua Hesketh proposed a change to openstack-infra/nodepool: Remove the need for zmq in proper demand calcs. https://review.openstack.org/98306 | 06:31 |
| SergeyLukjanov | kevinbenton, okay | 06:32 |
| *** Ryan_Lane has joined #openstack-infra | 06:32 | |
| SergeyLukjanov | kevinbenton, there is a bunch of near-to-merge CRs in top of the gate, I'll promote this one after them finished | 06:33 |
| SergeyLukjanov | to not reset them again | 06:33 |
| kevinbenton | SergeyLukjanov: sounds good. this one is still in the check queue anyway | 06:33 |
| *** unicell has joined #openstack-infra | 06:34 | |
| *** starmer has joined #openstack-infra | 06:35 | |
| SergeyLukjanov | kevinbenton, I've pushed it to the top of gate | 06:39 |
| SergeyLukjanov | (it's now a good time for it - gate reset again :( ) | 06:39 |
| StevenK | Heh, I was wondering why we just had a gate reset again | 06:39 |
| kevinbenton | SergeyLukjanov: hopefully this will help a bit (assuming even it can pass) | 06:40 |
| SergeyLukjanov | StevenK, we have it each several jobs passed | 06:40 |
| *** starmer has quit IRC | 06:40 | |
| *** andreaf has joined #openstack-infra | 06:42 | |
| *** _afazekas_slp has quit IRC | 06:44 | |
| *** marcoemorais has quit IRC | 06:44 | |
| openstackgerrit | Boris Pavlovic proposed a change to openstack-infra/config: Finally add osprofiler to stackforge https://review.openstack.org/97778 | 06:44 |
| *** jpich has joined #openstack-infra | 06:45 | |
| *** yfried has quit IRC | 06:47 | |
| openstackgerrit | Fathi Boudra proposed a change to openstack-infra/jenkins-job-builder: general: add support for childCustomWorkspace https://review.openstack.org/97108 | 06:48 |
| boris-42 | jhesketh lol hope now it is ok=) | 06:50 |
| *** starmer has joined #openstack-infra | 06:50 | |
| *** aconrad has quit IRC | 06:52 | |
| *** asettle-afk is now known as asettle | 06:52 | |
| *** starmer has quit IRC | 06:55 | |
| openstackgerrit | Fathi Boudra proposed a change to openstack-infra/jenkins-job-builder: general: add support for childCustomWorkspace https://review.openstack.org/97108 | 06:55 |
| openstackgerrit | Fathi Boudra proposed a change to openstack-infra/jenkins-job-builder: matrix project: add support for dynamic axis https://review.openstack.org/97111 | 06:57 |
| *** denis_makogon has joined #openstack-infra | 06:58 | |
| *** jgallard has joined #openstack-infra | 07:02 | |
| *** Longgeek has joined #openstack-infra | 07:02 | |
| *** _nadya_ has joined #openstack-infra | 07:03 | |
| *** Longgeek has quit IRC | 07:06 | |
| *** Longgeek has joined #openstack-infra | 07:06 | |
| lifeless | jogo: :) | 07:07 |
| openstackgerrit | A change was merged to openstack-infra/zuul: Pass pngcrush on status png files https://review.openstack.org/89865 | 07:08 |
| *** mestery has quit IRC | 07:10 | |
| *** jgallard has quit IRC | 07:10 | |
| *** _nadya_ has quit IRC | 07:10 | |
| *** jgallard has joined #openstack-infra | 07:10 | |
| *** mestery has joined #openstack-infra | 07:12 | |
| *** rdopiera has joined #openstack-infra | 07:12 | |
| *** _nadya_ has joined #openstack-infra | 07:13 | |
| *** bookwar has left #openstack-infra | 07:14 | |
| jogo | lifeless: I used it to build http://paste.openstack.org/show/83055/ | 07:16 |
| *** nati_ueno has quit IRC | 07:17 | |
| *** praneshp has quit IRC | 07:17 | |
| *** andreykurilin_ has quit IRC | 07:19 | |
| *** starmer has joined #openstack-infra | 07:20 | |
| openstackgerrit | A change was merged to openstack-infra/nodepool: Remove libzmq-dev from dependency list https://review.openstack.org/98218 | 07:22 |
| *** Ryan_Lane has quit IRC | 07:24 | |
| lifeless | jogo: almost like reviewday :) | 07:24 |
| *** starmer has quit IRC | 07:25 | |
| *** andreykurilin_ has joined #openstack-infra | 07:25 | |
| *** ihrachyshka has joined #openstack-infra | 07:26 | |
| jogo | lifeless: but backwards | 07:26 |
| *** yfried has joined #openstack-infra | 07:26 | |
| jogo | reviewday won't show abandoned bugs | 07:27 |
| jogo | err patches | 07:27 |
| openstackgerrit | Le Tian Ren proposed a change to openstack-infra/config: Add acl to create branches on powervc-driver https://review.openstack.org/97726 | 07:29 |
| *** _nadya_ has quit IRC | 07:29 | |
| *** lcheng has joined #openstack-infra | 07:30 | |
| openstackgerrit | A change was merged to openstack-infra/nodepool: Add warnings about the installation of libzmq1 https://review.openstack.org/97835 | 07:30 |
| *** amcrn has joined #openstack-infra | 07:31 | |
| *** mkerrin1 has quit IRC | 07:31 | |
| *** ihrachyshka has quit IRC | 07:31 | |
| *** ihrachyshka has joined #openstack-infra | 07:32 | |
| *** ominakov has joined #openstack-infra | 07:33 | |
| *** praneshp has joined #openstack-infra | 07:34 | |
| *** _nadya_ has joined #openstack-infra | 07:35 | |
| *** starmer has joined #openstack-infra | 07:36 | |
| *** mkerrin has joined #openstack-infra | 07:38 | |
| *** starmer has quit IRC | 07:40 | |
| *** mrda is now known as mrda-longweekend | 07:43 | |
| *** wenlock has quit IRC | 07:45 | |
| *** terryw has joined #openstack-infra | 07:45 | |
| *** jlibosva has joined #openstack-infra | 07:46 | |
| *** otherwiseguy has quit IRC | 07:48 | |
| *** starmer has joined #openstack-infra | 07:50 | |
| *** penguinRaider has quit IRC | 07:51 | |
| *** Ryan_Lane has joined #openstack-infra | 07:55 | |
| *** starmer has quit IRC | 07:55 | |
| *** Ryan_Lane1 has joined #openstack-infra | 07:57 | |
| *** isviridov|away is now known as isviridov | 07:57 | |
| *** viktors|afk is now known as viktors | 07:58 | |
| *** jp_at_hp has joined #openstack-infra | 07:59 | |
| *** Ryan_Lane has quit IRC | 07:59 | |
| openstackgerrit | Fathi Boudra proposed a change to openstack-infra/jenkins-job-builder: matrix project: add support for dynamic axis https://review.openstack.org/97111 | 08:01 |
| *** Ryan_Lane1 has quit IRC | 08:01 | |
| *** Guest93241 has joined #openstack-infra | 08:02 | |
| openstackgerrit | Fathi Boudra proposed a change to openstack-infra/jenkins-job-builder: matrix project: add support for dynamic axis https://review.openstack.org/97111 | 08:03 |
| *** e0ne has joined #openstack-infra | 08:03 | |
| *** wenlock has joined #openstack-infra | 08:04 | |
| *** hashar has joined #openstack-infra | 08:05 | |
| *** wenlock has quit IRC | 08:05 | |
| *** starmer has joined #openstack-infra | 08:05 | |
| *** jistr has joined #openstack-infra | 08:07 | |
| *** pelix has joined #openstack-infra | 08:08 | |
| *** derekh_ has joined #openstack-infra | 08:08 | |
| *** marun has quit IRC | 08:09 | |
| *** starmer has quit IRC | 08:10 | |
| *** _nadya_ has quit IRC | 08:11 | |
| *** bookwar has joined #openstack-infra | 08:13 | |
| *** e0ne has quit IRC | 08:15 | |
| *** e0ne has joined #openstack-infra | 08:16 | |
| *** e0ne has quit IRC | 08:20 | |
| *** starmer has joined #openstack-infra | 08:20 | |
| openstackgerrit | Yuriy Taraday proposed a change to openstack-infra/git-review: Ignore newline in bp/bug search in commit message https://review.openstack.org/98324 | 08:24 |
| *** srenatus has quit IRC | 08:24 | |
| *** starmer has quit IRC | 08:25 | |
| YorikSar | clarkb, fungi: ^ - might be another bugfix to land before 1.24 | 08:26 |
| *** praneshp has quit IRC | 08:26 | |
| *** penguinRaider has joined #openstack-infra | 08:30 | |
| *** starmer has joined #openstack-infra | 08:35 | |
| *** andreykurilin_ has quit IRC | 08:38 | |
| *** salv-orlando has joined #openstack-infra | 08:40 | |
| *** starmer has quit IRC | 08:40 | |
| vponomaryov | Hello everyone, I have question about incubation requirements | 08:42 |
| vponomaryov | There is one requirement "Project must have a basic devstack-gate job set up" - should it be "votable"? Preffered? | 08:43 |
| vponomaryov | info got from http://git.openstack.org/cgit/openstack/governance/tree/reference/incubation-integration-requirements.rst | 08:44 |
| openstackgerrit | Bob Ball proposed a change to openstack-infra/nodepool: Check for stale PID lock when starting https://review.openstack.org/90052 | 08:44 |
| mattoliverau | ok, I'm calling it a night! Have a great weekend all, it's a long weekend in Oz so be back full steam ahead on Tuesday :) | 08:46 |
| *** starmer has joined #openstack-infra | 08:51 | |
| *** starmer has quit IRC | 08:55 | |
| *** pblaho has joined #openstack-infra | 08:59 | |
| *** terryw has quit IRC | 09:00 | |
| *** otherwiseguy has joined #openstack-infra | 09:00 | |
| *** starmer has joined #openstack-infra | 09:05 | |
| SergeyLukjanov | vponomaryov, hey, IIRC it's better to have the dsvm job work and voting | 09:08 |
| SergeyLukjanov | vponomaryov, in fact, it's demonstration that your project could run and work with openstack | 09:09 |
| vponomaryov | SergeyLukjanov: will it be OK, If exist two nonvoting tempest jobs and add voting dsvm without tempest? | 09:10 |
| *** starmer has quit IRC | 09:10 | |
| *** dizquierdo has joined #openstack-infra | 09:17 | |
| *** starmer has joined #openstack-infra | 09:20 | |
| *** starmer has quit IRC | 09:25 | |
| SergeyLukjanov | vponomaryov, I think so, but note that I'm not a TC member | 09:26 |
| SergeyLukjanov | :) | 09:26 |
| vponomaryov | SergeyLukjanov: I am absorbing any experience, thanks | 09:26 |
| vponomaryov | SergeyLukjanov: If you have time, could you please review this: https://review.openstack.org/#/c/97747/ it has already five +1's | 09:29 |
| *** unicell has quit IRC | 09:31 | |
| *** GheRivero has quit IRC | 09:33 | |
| *** hashar has quit IRC | 09:33 | |
| *** hashar has joined #openstack-infra | 09:35 | |
| SergeyLukjanov | vponomaryov, /me commented | 09:35 |
| *** starmer has joined #openstack-infra | 09:35 | |
| *** flaper87|afk is now known as flaper87 | 09:39 | |
| *** lcheng has quit IRC | 09:39 | |
| *** starmer has quit IRC | 09:40 | |
| vponomaryov | SergeyLukjanov: you mean set expected parameter that would say what is it by its name? | 09:40 |
| *** ihrachyshka has quit IRC | 09:43 | |
| trinaths | fungi: ping | 09:44 |
| *** Guest93241 has quit IRC | 09:45 | |
| openstackgerrit | Darragh Bailey proposed a change to openstack-infra/jenkins-job-builder: Use yaml local tags to support including files https://review.openstack.org/48783 | 09:47 |
| openstackgerrit | Darragh Bailey proposed a change to openstack-infra/jenkins-job-builder: Support lazy resolving of include yaml tags https://review.openstack.org/63580 | 09:47 |
| trinaths | anteaya: ping | 09:48 |
| *** rlandy has joined #openstack-infra | 09:50 | |
| *** starmer has joined #openstack-infra | 09:50 | |
| mordred | vponomaryov: what would the non-tempest dsvm job do? | 09:51 |
| vponomaryov | mordred: devstack installation with project's plugin | 09:52 |
| mordred | hrm | 09:55 |
| *** starmer has quit IRC | 09:55 | |
| vponomaryov | mordred: the most big reason - the way we use tempest | 09:55 |
| mordred | vponomaryov: yeah - that's probably fine as part of incubation | 09:55 |
| SergeyLukjanov | mordred, hey, why are you not in a bar or not sleeping? :) | 09:56 |
| *** IvanBerezovskiy has joined #openstack-infra | 09:57 | |
| mordred | SergeyLukjanov: how do you know I'm not doing one of those things? | 09:58 |
| mordred | SergeyLukjanov: (jetlag - I woke up at 4:30 and have walked to a 24 hour diner) | 09:58 |
| SergeyLukjanov | mordred, you're not mordred_phone ;) | 09:58 |
| mordred | SergeyLukjanov: excellent powers of reasoning! :) | 09:59 |
| *** Hal has joined #openstack-infra | 09:59 | |
| SergeyLukjanov | mordred, heh | 09:59 |
| *** Hal is now known as Guest86489 | 09:59 | |
| *** Guest86489 has quit IRC | 10:03 | |
| *** ociuhandu has quit IRC | 10:05 | |
| *** starmer has joined #openstack-infra | 10:06 | |
| *** amcrn has quit IRC | 10:06 | |
| vponomaryov | SergeyLukjanov: answered to your comment | 10:08 |
| *** starmer has quit IRC | 10:10 | |
| *** katyafervent is now known as katyafervent_awa | 10:12 | |
| *** jgallard has quit IRC | 10:13 | |
| *** e0ne has joined #openstack-infra | 10:15 | |
| openstackgerrit | Yuriy Taraday proposed a change to openstack-infra/git-review: Retry command on "hash mismatch" error in tests https://review.openstack.org/98354 | 10:17 |
| *** GheRivero has joined #openstack-infra | 10:19 | |
| *** starmer has joined #openstack-infra | 10:20 | |
| mordred | SergeyLukjanov: I just tried and failed to make my first gertty patch | 10:21 |
| mordred | not because I don't understand the code - but beceause the thing I wrote makes things WAY worse :) | 10:21 |
| mordred | I guess jeblair knows what he's doing | 10:22 |
| gilliard | It says something about gerrit when a curses version of its UI looks nicer. | 10:22 |
| mordred | gilliard: :) | 10:25 |
| *** starmer has quit IRC | 10:25 | |
| mordred | gilliard: the offline reviewing feature of gertty is possibly the best invention ever | 10:26 |
| mordred | although yeah - the online mode is still better too | 10:27 |
| *** Alexei_987 has left #openstack-infra | 10:28 | |
| pelix | gertty? does this mean full cmdline reviewing finally possible? | 10:29 |
| *** e0ne_ has joined #openstack-infra | 10:30 | |
| mordred | pelix: yes | 10:30 |
| gilliard | https://github.com/stackforge/gertty | 10:30 |
| mordred | pelix: we haven't announced because it's still got a few bugs and probably isn't ready for ALL of the devs | 10:30 |
| mordred | pelix: but it's the best thing you'll ever encounter in your life | 10:30 |
| pelix | don't care if it's not announced, this is what I've been hoping for since git-review was developed :) | 10:31 |
| mordred | pelix: well, jump on the train - the water's fine | 10:31 |
| *** flaper87 is now known as flaper87|afk | 10:31 | |
| gilliard | Takes a while to sync when you first subscribe to a project, I guess it's pulling each patch thru ssh | 10:32 |
| *** isviridov is now known as isviridov|away | 10:32 | |
| SergeyLukjanov | mordred, I've tried to contrib something to gertty too, but I've fall asleep | 10:32 |
| mordred | yes, it does | 10:32 |
| pelix | mordred: already cloned! | 10:32 |
| SergeyLukjanov | mordred, offline mode is awesome | 10:32 |
| mordred | gilliard, pelix: it does a git fetch of all of the relevant refs so taht you can check them out in your repo is needed while offline | 10:32 |
| mordred | so it also makes the equiv of "git review -d" really easy | 10:33 |
| *** e0ne has quit IRC | 10:33 | |
| pelix | so equivalent of 'git config --add remote.gerrit.fetch +refs/changes/*:refs/remotes/gerrit-changes/* && git fetch gerrit' but without polluting your remote branches view | 10:34 |
| *** starmer has joined #openstack-infra | 10:35 | |
| mordred | pelix: that's right | 10:36 |
| *** ociuhandu has joined #openstack-infra | 10:36 | |
| *** starmer has quit IRC | 10:40 | |
| openstackgerrit | Monty Taylor proposed a change to stackforge/gertty: Move contributing section to its own file https://review.openstack.org/98363 | 10:43 |
| openstackgerrit | Monty Taylor proposed a change to stackforge/gertty: Remove unneeded bit from setup.cfg https://review.openstack.org/98364 | 10:43 |
| *** markmc has joined #openstack-infra | 10:43 | |
| *** nosnos has quit IRC | 10:45 | |
| *** tkelsey has quit IRC | 10:46 | |
| *** sandywalsh has quit IRC | 10:46 | |
| openstackgerrit | Andrea Frittoli proposed a change to openstack-infra/devstack-gate: Support for alternate trunk name (not master) https://review.openstack.org/98365 | 10:46 |
| *** mattoliverau has quit IRC | 10:47 | |
| *** mattoliverau has joined #openstack-infra | 10:50 | |
| *** ihrachyshka has joined #openstack-infra | 10:50 | |
| *** starmer has joined #openstack-infra | 10:50 | |
| openstackgerrit | Christian Berendt proposed a change to openstack-infra/config: except ImportError when importing cPickle on Python3 https://review.openstack.org/98367 | 10:52 |
| *** ihrachyshka has quit IRC | 10:52 | |
| *** ihrachyshka has joined #openstack-infra | 10:52 | |
| *** starmer has quit IRC | 10:55 | |
| openstackgerrit | Andrea Frittoli proposed a change to openstack-infra/devstack-gate: Support for alternate trunk name (not master) https://review.openstack.org/98365 | 10:56 |
| *** apevec has joined #openstack-infra | 10:58 | |
| *** mburned_out is now known as mburned | 10:58 | |
| *** ihrachyshka_ has joined #openstack-infra | 10:59 | |
| *** ihrachyshka has quit IRC | 11:02 | |
| *** starmer has joined #openstack-infra | 11:05 | |
| *** mbacchi has joined #openstack-infra | 11:07 | |
| *** denis_makogon has quit IRC | 11:09 | |
| *** starmer has quit IRC | 11:10 | |
| *** e0ne_ has quit IRC | 11:11 | |
| yolanda | hi mordred, is there any reason for using v1.1 nova client version in nodepool patch, instead of v3? | 11:11 |
| *** e0ne has joined #openstack-infra | 11:11 | |
| mordred_phone | yolanda: nothing specific ... is v3 better? | 11:12 |
| yolanda | mordred, just looking at v1 client, fails because it isn't expected password argument | 11:12 |
| mordred_phone | (and can v3 talk to older things too?) | 11:12 |
| yolanda | mordred, don't know about that, i'll need to look at it | 11:13 |
| mordred_phone | oh, well, that should work ... we do password autg in production | 11:13 |
| yolanda | mm, maybe it's not called password | 11:13 |
| yolanda | i'll dig on it | 11:13 |
| yolanda | it may be the api key | 11:13 |
| *** tkelsey has joined #openstack-infra | 11:14 | |
| *** dims__ has joined #openstack-infra | 11:15 | |
| *** e0ne has quit IRC | 11:15 | |
| *** matsuhashi has quit IRC | 11:16 | |
| *** e0ne has joined #openstack-infra | 11:17 | |
| *** _nadya_ has joined #openstack-infra | 11:18 | |
| *** dims_ has joined #openstack-infra | 11:19 | |
| *** starmer has joined #openstack-infra | 11:20 | |
| *** matsuhashi has joined #openstack-infra | 11:21 | |
| *** dims__ has quit IRC | 11:21 | |
| *** fifieldt_ has quit IRC | 11:23 | |
| *** fifieldt_ has joined #openstack-infra | 11:24 | |
| *** starmer has quit IRC | 11:25 | |
| sdague | mordred_phone / SergeyLukjanov : anyone want to just ninja merge this havana item in - https://review.openstack.org/#/c/98269/ ? | 11:26 |
| sdague | because I definitely don't have the patience to fix the rest of havana to land that fix :) | 11:26 |
| sdague | man, what's up with the log server? | 11:27 |
| mordred_phone | sdague: sure, just a sec | 11:28 |
| jp_at_hp | mordred_phone: looking for this change to land https://review.openstack.org/#/c/97757 - it seems to have been in the gate queue for 13 hours. Is there anything that can be done to unblock it? The top of that queue seems to be 43 hours or so - is that just backlog? | 11:28 |
| *** yaguang has quit IRC | 11:28 | |
| openstackgerrit | Sean Dague proposed a change to openstack-infra/elastic-recheck: add bug for havana get-pip failure https://review.openstack.org/98378 | 11:33 |
| *** tkelsey has quit IRC | 11:33 | |
| sdague | jp_at_hp: it's because there are a lot of races in openstack exposed by our system | 11:33 |
| sdague | and until they get fixed, the gate queue is going to remain basically that long | 11:34 |
| *** lbragstad has joined #openstack-infra | 11:34 | |
| jp_at_hp | sdague: that doesn't seem to answer my question - ah - you skipped the "yes, the queue is just that long" and gave me the reason. Thanks :D | 11:34 |
| sdague | oh, sorry :) | 11:35 |
| sdague | yes the queue is that long | 11:35 |
| jp_at_hp | So it looks like that means ~ 30 hours until that gate job runs... | 11:35 |
| sdague | yep | 11:35 |
| jp_at_hp | Great to know - thanks. | 11:35 |
| sdague | maybe more | 11:35 |
| *** starmer has joined #openstack-infra | 11:35 | |
| sdague | estimating the tail duration is kind of hard | 11:35 |
| jp_at_hp | Always more - but hey, that's the weekend, so as long as it's under 72 hours... | 11:35 |
| sdague | yeh, right now we're promoting things that we think will fix key issues to make it better. | 11:36 |
| *** matsuhashi has quit IRC | 11:36 | |
| jp_at_hp | sdague: well that review I referenced is causing very many gate failures... | 11:36 |
| *** matsuhashi has joined #openstack-infra | 11:36 | |
| sdague | jp_at_hp: only for ironic | 11:37 |
| *** e0ne has quit IRC | 11:37 | |
| jp_at_hp | so it is a definite candidate for promotion. Well - that's all I care about ata the minute - you mean there's more to this openstack thing? ;-) | 11:37 |
| sdague | it's not causing "gate" failures | 11:37 |
| sdague | it's preventing ironic changes from moving from check -> gate | 11:37 |
| *** e0ne has joined #openstack-infra | 11:37 | |
| *** adalbas has quit IRC | 11:38 | |
| jp_at_hp | I think it's having knock-on effects in other areas too, escpecially around tripleo, but upshot is it's not necessarily your priority, and that's fine. | 11:38 |
| *** starmer has quit IRC | 11:40 | |
| *** matsuhashi has quit IRC | 11:40 | |
| sdague | for reference - these are the bugs we need to tackle - http://status.openstack.org/elastic-recheck/gate.html - they are the ones that actually caused the resets - if you want to help dive in | 11:42 |
| sdague | once those are lessened, code will flow quick again | 11:42 |
| *** mwagner_lap has quit IRC | 11:42 | |
| openstackgerrit | A change was merged to openstack-infra/elastic-recheck: add bug for havana get-pip failure https://review.openstack.org/98378 | 11:43 |
| mordred | sdague: ninja merge done | 11:43 |
| BobBall | I hope you made some scary hand gestures while you typed that mordred | 11:44 |
| mordred | BobBall: I did indeed | 11:45 |
| BobBall | *phew* | 11:45 |
| sdague | mordred: awesome | 11:45 |
| sdague | now every icehouse change won't explode imediately! :) | 11:45 |
| mordred | sdague: the top fails there seem to be infra fails - am I reading that wrong? | 11:46 |
| sdague | so this one that you just merged wasn't classified, I just landed the fingerprint, I expect that to be top of list | 11:47 |
| mordred | ok. | 11:47 |
| sdague | the top 2 have a suspicious similarity in shape | 11:47 |
| sdague | which means I think they are the same bug | 11:48 |
| sdague | which is rax deb mirror going offline | 11:48 |
| sdague | or unreachable | 11:48 |
| sdague | oh, they are actually the same bug | 11:48 |
| sdague | let me delete the expired one | 11:48 |
| openstackgerrit | Sean Dague proposed a change to openstack-infra/elastic-recheck: this is the expired version of https://review.openstack.org/98381 | 11:49 |
| mordred | what causes us to hit the rax mirror during tests- we should have most everything cached, no? | 11:49 |
| mordred | (I mean, I know that in theory we need to account for something new existing) | 11:50 |
| *** adalbas has joined #openstack-infra | 11:50 | |
| sdague | we seem to always need something | 11:50 |
| *** isviridov|away is now known as isviridov | 11:50 | |
| *** starmer has joined #openstack-infra | 11:50 | |
| sdague | or there's an update to something | 11:50 |
| sdague | I think that's fallout from the fact that the nodepool images often fail to build | 11:51 |
| mordred | well, I guess what I'm saying is - what if we didn't do apt-get update and just did the apt-get installs - that should lessen the amount that we go out to the mirror to only be when there is explicitly a new package that we never had before | 11:51 |
| sdague | we don't error on apt-get update | 11:52 |
| mordred | sdague: and/or we could wrap the install with "apt-get install $foo || (apt-get update && apt-get install $foo)" | 11:52 |
| mordred | I know we don't | 11:52 |
| mordred | but apt-get update causes apt to think it needs to get something that's not in the apt cache | 11:52 |
| mordred | which means that doing the update could increase the amount of times we go to the internet where we could potentially serve from local cache | 11:53 |
| sdague | so, I don't think it does | 11:55 |
| sdague | reading through this log http://logs.openstack.org/02/84202/7/gate/gate-tempest-dsvm-postgres-full/d0d1dec/logs/devstacklog.txt.gz#_2014-06-06_10_59_02_345 (that's the fail point) | 11:55 |
| sdague | if you go up to the rabbit install, it's pulling them off the disk | 11:55 |
| sdague | not hitting the network | 11:55 |
| *** starmer has quit IRC | 11:55 | |
| sdague | also... since when did we start pulling packages from amazon? | 11:55 |
| openstackgerrit | Christian Berendt proposed a change to openstack-dev/hacking: Add new check H905 https://review.openstack.org/98384 | 11:55 |
| mordred | ubuntu mirrors | 11:55 |
| mordred | sdague: but we run apt-get update in setup_host | 11:56 |
| mordred | in devstack-gate | 11:56 |
| mordred | sdague: that means apt-get update was run before that log file | 11:57 |
| sdague | I thought we set it explicitly? | 11:57 |
| mordred | it's in "use latest libvirt" | 11:57 |
| openstackgerrit | Christian Berendt proposed a change to openstack-dev/hacking: Add new check H905 https://review.openstack.org/98384 | 11:57 |
| mordred | enable_latest_libvirt, that is | 11:57 |
| openstackgerrit | A change was merged to openstack-infra/elastic-recheck: this is the expired version of https://review.openstack.org/98381 | 11:59 |
| *** zz_gondoi is now known as gondoi | 11:59 | |
| mordred | sdague: that was a temp workaround anyway: | 11:59 |
| mordred | https://bugs.launchpad.net/nova/+bug/1254872 | 11:59 |
| sdague | mordred: src pointer? | 11:59 |
| uvirtbot | Launchpad bug 1254872 in libvirt "libvirtError: Timed out during operation: cannot acquire state change lock" [High,Fix released] | 11:59 |
| mordred | sdague: and the package has not hit ubuntu stable | 11:59 |
| mordred | sdague: a319d3528fa7b695bc7b05b0d6768fcda29a5490 | 12:00 |
| mordred | Ib85769e9a3470bede836ac0b87861ce76c0aedcf | 12:00 |
| mordred | sdague: oh - my local repo is stale. one sec | 12:01 |
| *** trinaths has left #openstack-infra | 12:01 | |
| sdague | yeh | 12:01 |
| sdague | I deleted that bit a while ago :) | 12:01 |
| sdague | because the package is in stable | 12:01 |
| mordred | yah. awesome | 12:01 |
| mordred | so now why is it trying to go to the internets for postgres there | 12:02 |
| sdague | 93c26c1ae421dd4f57252ec436d1315d3ce702c5 | 12:02 |
| sdague | because there is a security update I think | 12:02 |
| *** gondoi is now known as zz_gondoi | 12:02 | |
| sdague | the only part of the apt-get update that takes any time is ... | 12:02 |
| sdague | http://logs.openstack.org/02/84202/7/gate/gate-tempest-dsvm-postgres-full/d0d1dec/logs/devstacklog.txt.gz#_2014-06-06_10_51_31_780 | 12:02 |
| sdague | so, we could make a patch to devstack to let you disable package update | 12:03 |
| mordred | well, you can already set NO_UPDATE_REPOS=True | 12:03 |
| sdague | the package update was in there from a development reason so that we don't have to debug people with crufty systems | 12:03 |
| mordred | yah | 12:04 |
| sdague | oh, yeh, sure | 12:04 |
| sdague | that too :) | 12:04 |
| sdague | any downside of doing that in devstack gate? | 12:05 |
| mordred | sdague: so we could just set that in devstack-gate and see if it helps | 12:05 |
| sdague | it means we'd be as stale as the nodepool builds | 12:05 |
| mordred | I mean, there is a chance that a new package that completely didn't exist in ubuntu at all | 12:05 |
| mordred | gets added to devstack | 12:05 |
| mordred | and it's not on the local machine or in the indexes | 12:05 |
| *** starmer has joined #openstack-infra | 12:05 | |
| sdague | what's cached? | 12:05 |
| sdague | it might just move a failure and not really help | 12:06 |
| mordred | or, rather, there is a chance that a package that's in the index but not cached gets added to devstack, and something about the dep list for that package which is pulled from the internets causes it to bomb out on cached versions | 12:06 |
| sdague | because if we add a new package to devstack, that ubuntu knows about | 12:06 |
| sdague | and it gets a sec update | 12:06 |
| mordred | this is why I think it should be coupled with a apt-get install $foo || (apt-get update && apt-get install $foo) | 12:06 |
| sdague | but the nodepool image is a couple days old | 12:06 |
| mordred | yah | 12:06 |
| mordred | that's why ^^ - because then the install will fail, but if it does, it's simple to try one time to run an update and then re-run the install | 12:07 |
| mordred | for that matter | 12:07 |
| *** dprince has joined #openstack-infra | 12:07 | |
| mordred | we could do that _anyway_ | 12:07 |
| sdague | yeh | 12:07 |
| mordred | because even the transient repo errors now | 12:07 |
| mordred | would 99% be fixed by just trying again after an update | 12:07 |
| sdague | their outages are long enough that I doubt it | 12:07 |
| sdague | but it's an interesting idea | 12:07 |
| *** ArxCruz has joined #openstack-infra | 12:09 | |
| sdague | ok, let me see if I can get this logic without being completely crazy pants in here, because the actual devstack code to do this is slightly more complex because of multi distro and flag support | 12:09 |
| mordred | sdague: working on patch - one sec | 12:10 |
| sdague | ok | 12:10 |
| *** salv-orlando has quit IRC | 12:10 | |
| sdague | even bette | 12:10 |
| *** starmer has quit IRC | 12:10 | |
| *** yfried has quit IRC | 12:11 | |
| *** rfolco has joined #openstack-infra | 12:12 | |
| mordred | sdague: before I pretty it up - this is sort of what I'm thinking: http://paste.openstack.org/show/83096 | 12:13 |
| *** alaski is now known as lascii | 12:16 | |
| sdague | so NO_UPDATE_REPO is also there to support an offline mode | 12:16 |
| sdague | I feel like we need UPDATE_RETRY or something | 12:16 |
| mordred | sdague: yeah. you're right | 12:16 |
| sdague | before you dive back in | 12:17 |
| sdague | https://review.openstack.org/#/c/98265/ could use to be moved to the gate and promoted | 12:17 |
| openstackgerrit | Ilya Sviridov proposed a change to openstack-infra/config: Added new python-magnetodbclient project to Stackforge https://review.openstack.org/91050 | 12:17 |
| sdague | turns out... all our cached files don't exist on the old/ side of grenade | 12:17 |
| mordred | sdague: I do not know the move-to-gate cantrip, although I do know the promote cantrip | 12:18 |
| mordred | SergeyLukjanov: ^^ do you know how to do this? | 12:18 |
| sdague | SergeyLukjanov definitely knows it | 12:18 |
| *** e0ne has quit IRC | 12:19 | |
| *** e0ne has joined #openstack-infra | 12:20 | |
| mordred | sdague: http://paste.openstack.org/show/83097 | 12:20 |
| *** Hal has joined #openstack-infra | 12:20 | |
| *** starmer has joined #openstack-infra | 12:20 | |
| *** Hal is now known as Guest33002 | 12:21 | |
| sdague | so that never updates right? | 12:23 |
| mordred | sdague: right. unless it fails to install something | 12:23 |
| sdague | no, I mean never | 12:23 |
| mordred | really? | 12:23 |
| sdague | because it short circuits on retry | 12:24 |
| mordred | oh. hahahaha. jesus | 12:24 |
| mordred | one sec. thank you | 12:24 |
| mordred | (this is why I'm pastebining it before submitting into the gate, btw) | 12:24 |
| *** e0ne has quit IRC | 12:24 | |
| sdague | I think you whant RETRY_UPDATE != "True" | 12:25 |
| mordred | yup. agree | 12:25 |
| *** starmer has quit IRC | 12:25 | |
| sdague | and don't set it on the real_package_install line at all | 12:25 |
| Shrews | mordred: are you suggesting we use pastebin as our new review system? :) | 12:25 |
| sdague | though, honestly, we have a ton of capacity in check | 12:25 |
| mordred | sdague: I need to set it on the real_package_install line | 12:25 |
| sdague | so sticking it there isn't a bad idea | 12:25 |
| sdague | mordred: why? | 12:25 |
| mordred | sdague: hang on - the logic is all screwed. let me take another pass | 12:26 |
| sdague | oh, you'll never got to it | 12:26 |
| sdague | yeh | 12:26 |
| sdague | this is why I said it was kind of squirely | 12:26 |
| mordred | yah | 12:26 |
| sdague | so actually, I think you want to use RETRY_UPDATE to decide if you set NO_UPDATE_REPOS | 12:26 |
| sdague | and that will be actually quite simple | 12:27 |
| mordred | sdague: I don't follow - you wanna do that? | 12:27 |
| sdague | actually, never mind, that's not quite right either | 12:27 |
| openstackgerrit | Christian Berendt proposed a change to openstack-dev/hacking: Add new check H905 https://review.openstack.org/98384 | 12:27 |
| mordred | sdague: https://etherpad.openstack.org/p/OA0XgB0HHO | 12:28 |
| sdague | I have a hard time thinking about it in etherpad, let me noodle a little in emacs to see if I see it | 12:29 |
| mordred | ok. | 12:29 |
| mordred | sdague: I think my latest change may have it | 12:29 |
| mordred | http://paste.openstack.org/show/83098 if that's easier | 12:30 |
| *** godp1301 has joined #openstack-infra | 12:31 | |
| sdague | so I think the real logic challenge here is we are using NO_UPDATE_REPOS as a config and state flag | 12:32 |
| sdague | if we use it only as a config flag | 12:32 |
| *** pdmars has joined #openstack-infra | 12:32 | |
| *** yamahata has quit IRC | 12:33 | |
| SergeyLukjanov | I'm here again | 12:33 |
| sdague | and have REPOS_UPDATED if we did it | 12:33 |
| sdague | then have RETRY_UPDATE basically be "ignore repos updated" | 12:33 |
| SergeyLukjanov | do you guys still need ninja push to gate? | 12:33 |
| sdague | SergeyLukjanov: we need a promote | 12:33 |
| SergeyLukjanov | sdague, I'll do it | 12:33 |
| SergeyLukjanov | sdague, which one? | 12:33 |
| sdague | SergeyLukjanov: well actually both of - https://etherpad.openstack.org/p/gatetriage-june2014 | 12:34 |
| sdague | in the order I just changed them to | 12:34 |
| SergeyLukjanov | sdague, okay | 12:34 |
| *** smarcet has joined #openstack-infra | 12:35 | |
| *** starmer has joined #openstack-infra | 12:35 | |
| *** prad_ has joined #openstack-infra | 12:36 | |
| *** talluri has quit IRC | 12:37 | |
| SergeyLukjanov | sdague, promoted | 12:37 |
| *** talluri has joined #openstack-infra | 12:37 | |
| sdague | SergeyLukjanov: thank you | 12:37 |
| *** dims_ has quit IRC | 12:37 | |
| otherwiseguy | Ok, so I've just watched the tests restart twice on my review. Something weird going on? | 12:37 |
| *** Shrews has quit IRC | 12:37 | |
| *** katyafervent_awa is now known as katyafervent | 12:38 | |
| mordred | sdague: ok. let me try that. | 12:38 |
| sdague | I get slightly amused on every gate storm that people show up on the list and provide suggestions on how to do the gate better... which are already how the gate works :) | 12:38 |
| sdague | otherwiseguy: change? | 12:38 |
| *** tkelsey has joined #openstack-infra | 12:38 | |
| *** Shrews has joined #openstack-infra | 12:39 | |
| *** thomasbiege has joined #openstack-infra | 12:39 | |
| sdague | otherwiseguy: what is the project, change, and what queue is it in | 12:39 |
| otherwiseguy | openstack/neutron 96525. I just noticed that there were things that showed passed and in progress, and they stopped twice. | 12:39 |
| SergeyLukjanov | sdague, :) | 12:39 |
| SergeyLukjanov | I'll be on meetings for the next 4 hours | 12:39 |
| sdague | otherwiseguy: right, that's in the gate queue | 12:39 |
| SergeyLukjanov | but able to promote something on demand | 12:39 |
| sdague | so if anything in front of it fails, it needs to get restarted | 12:39 |
| otherwiseguy | sdague: the gate would work much better if it were magic. | 12:40 |
| sdague | because that's a speculative execution test | 12:40 |
| otherwiseguy | :p | 12:40 |
| otherwiseguy | sdague: ah, good to know. :) | 12:40 |
| *** starmer has quit IRC | 12:40 | |
| sdague | so your change 96525 is actually getting tested with *all* the changes above it as a bundle | 12:40 |
| sdague | under the assumption they will all pass their tests | 12:40 |
| ttx | anteaya: http://fnords.wordpress.com/2014/06/06/analysis-of-april-2014-tc-election/ | 12:40 |
| sdague | but if they don't, the assumption is broken, and we need to reset | 12:40 |
| otherwiseguy | sdague: I wonder what the statistics would be like for that assumption being correct. | 12:41 |
| *** talluri has quit IRC | 12:42 | |
| * otherwiseguy stops watching his patch and just waits for an email | 12:42 | |
| mordred | sdague: http://paste.openstack.org/show/83099 | 12:42 |
| *** medieval1 has quit IRC | 12:43 | |
| sdague | otherwiseguy: when they are bad, that's when we get backlogs like this | 12:43 |
| sdague | and the reason they are bad is because we've got all these race conditions in openstack itself that aren't getting addressed | 12:43 |
| *** medieval1 has joined #openstack-infra | 12:43 | |
| sdague | which our test system fails the run on because things like starting a compute that you can ssh to doesn't happen | 12:44 |
| openstackgerrit | Andrea Frittoli proposed a change to openstack-infra/devstack-gate: Support for alternate trunk name (not master) https://review.openstack.org/98365 | 12:44 |
| *** aysyd has joined #openstack-infra | 12:44 | |
| otherwiseguy | sdague: Wait, are you telling me that there is crappy code inside openstack? I work on neutron, so I've never noticed this. /s | 12:44 |
| mordred | otherwiseguy: see. you learn things every day | 12:45 |
| sdague | mordred: that looks pretty sane | 12:45 |
| sdague | I *think* it's right | 12:45 |
| markmc | ttx, the foodie party, heh | 12:45 |
| *** zul has quit IRC | 12:45 | |
| mordred | sdague: do we have a thing like is_ubuntu that's actually is_ubuntu || is_debian ? | 12:46 |
| mordred | sdague: because honestly this logic should apply for all debian derivitives | 12:46 |
| ttx | markmc: just edited to say that bias is actually decresing | 12:46 |
| ttx | +a | 12:46 |
| markmc | ttx, even better | 12:47 |
| markmc | ttx, the debate about such things continued a little in london | 12:47 |
| sdague | mordred: is_ubuntu is true on debian | 12:48 |
| *** signed8bit has joined #openstack-infra | 12:48 | |
| *** medieval1 has quit IRC | 12:48 | |
| sdague | for historical reasons | 12:48 |
| markmc | ttx, suggestions including "all PTLs should be employed by the foundation", sigh | 12:48 |
| sdague | but it's worked out fine so far | 12:48 |
| sdague | markmc: WAT | 12:48 |
| ttx | sdague: sabdfl bullshit | 12:49 |
| sdague | man, I might need to come on these road tours some time for the craziness | 12:49 |
| mordred | ttx: that's some excellent blogging my friend | 12:49 |
| mordred | sdague: k. I pushed up the change to gerrit - it's far enough along | 12:49 |
| *** zul has joined #openstack-infra | 12:49 | |
| ttx | mordred: I don't dispair to be able to clean up my analysis scripts so that they can be run by everyone | 12:50 |
| ttx | but that nvd3 graph is a real bitch | 12:50 |
| *** starmer has joined #openstack-infra | 12:50 | |
| mordred | sdague: I should partial-bug it on 1286818 yeah? | 12:50 |
| sdague | Related-Bug is what I tend to use | 12:51 |
| sdague | mostly for laziness | 12:51 |
| mordred | k. commit message amended | 12:52 |
| *** starmer has quit IRC | 12:55 | |
| *** dizquierdo has quit IRC | 12:55 | |
| sdague | ttx: that blog up on your twitters yet? | 12:55 |
| ttx | yep just a sec | 12:56 |
| ttx | sdague: https://twitter.com/tcarrez/status/474897745950830592 | 12:57 |
| *** miqui has joined #openstack-infra | 12:58 | |
| openstackgerrit | Dan Prince proposed a change to openstack-infra/nodepool: Add support for network labels https://review.openstack.org/98287 | 13:02 |
| anteaya | the Foodie party? | 13:03 |
| anteaya | you find good creperies, that I do admit | 13:04 |
| *** starmer has joined #openstack-infra | 13:05 | |
| ttx | anteaya: I just couldn't find any other bias that high score could reveal :) | 13:05 |
| anteaya | guys with really short haircuts? | 13:06 |
| *** dims has joined #openstack-infra | 13:08 | |
| *** amotoki has quit IRC | 13:08 | |
| *** gabriel-bezerra has left #openstack-infra | 13:09 | |
| *** starmer has quit IRC | 13:10 | |
| *** dmakogon_ is now known as denis_makogon | 13:13 | |
| *** e0ne has joined #openstack-infra | 13:14 | |
| *** olaph has joined #openstack-infra | 13:14 | |
| mordred | dprince: the change above ^^ - you're using a novaclient extension - a) how stable is that and b) does a deployer have to do something special for that to work, or could one fairly well assume that if they've deployed neutron that novaclient extension will work? | 13:14 |
| dprince | mordred: if 'nova net-list' works on that provider this is the same extension | 13:15 |
| dprince | mordred: this is an extension, yes | 13:16 |
| mordred | mordred@camelot:~$ source hpcloud-monty-1.1.sh | 13:16 |
| mordred | mordred@camelot:~$ nova net-list | 13:16 |
| mordred | ERROR: Not found (HTTP 404) | 13:16 |
| dprince | mordred: the alternative is to use Neutron API's directly which seems out of scope for nodelist | 13:16 |
| mordred | dprince: that's using it on hpcloud - so I guess as a user I'd just need to know that my cloud doesn't suppor that feature and not configure a network by name | 13:16 |
| dprince | mordred: So that just means we can use net-label for HP. I really want this for the TripleO clouds... | 13:16 |
| mordred | ++ | 13:17 |
| *** basha has quit IRC | 13:17 | |
| dprince | mordred: perhaps I just doc that... and we can use it selectively | 13:17 |
| mordred | totally - just making sure I understand the ramifications | 13:17 |
| dprince | mordred: So how does HP let you get your network UUID's then? A web UI? | 13:17 |
| dprince | mordred: to be fair I haven't checked this with any of the public cloud providers... | 13:18 |
| mordred | dprince: neutron net-list | 13:18 |
| dprince | mordred: Aha. So we could try both... | 13:19 |
| mordred | dprince: I've got another patch coming that tries to add neutronclient and glanceclient objects to the nodepool provider - I could probably split that part out into its own patch and then we could refactor this to use neutronclient directly | 13:19 |
| dprince | mordred: but that would require us to bring in neutronclient | 13:19 |
| mordred | dprince: BUT - I don't think we need to block on that | 13:19 |
| mordred | unless you really just feel like fixing that part of my other patch :) | 13:19 |
| dprince | mordred: hmmm. So I functionally tested this against my TripleO Overcloud. I know what is there already works | 13:20 |
| mordred | dprince: https://review.openstack.org/#/c/46482 for reference <--- patch does not work | 13:20 |
| mordred | dprince: awesome | 13:20 |
| *** starmer has joined #openstack-infra | 13:20 | |
| mordred | yeah - I don't think we need to block on this- it's more a note of what the next version of the patch wants to look like | 13:21 |
| mordred | like, if we get neutronclient, we should replace the nova extension call with it | 13:21 |
| dprince | mordred: the only reason I used novaclient for this was because we already had it in nodepool | 13:21 |
| *** dkliban_afk is now known as dkliban | 13:22 | |
| mordred | yah | 13:22 |
| dprince | mordred: I could actually see a provider doing it either way. It could very well be a provider doesn't publically expose the Neutron API, but they do provide the Nova extension. | 13:22 |
| dprince | mordred: similar for glance... | 13:22 |
| mordred | dprince: good point | 13:23 |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/config: PyPI upload jobs should not take slave offline https://review.openstack.org/98405 | 13:23 |
| dprince | mordred: now hopefully that sort of thing will go out of style and people will just expose all the API's | 13:23 |
| mordred | dprince: right? I mean, it's only been four years ... ;) | 13:24 |
| openstackgerrit | Matthew Treinish proposed a change to openstack-infra/devstack-gate: Capture old tempest artifacts too https://review.openstack.org/98406 | 13:24 |
| * dprince thinks wishfully | 13:24 | |
| *** doug-fish has joined #openstack-infra | 13:24 | |
| *** starmer has quit IRC | 13:25 | |
| *** salv-orlando has joined #openstack-infra | 13:27 | |
| sdague | it might help once there is a unified client because then people will expect to have access to the apis | 13:29 |
| fungi | mordred: dhellmann: is it expected that projects with hyphens in their names (and thus in the beginning of their sdist tarball filenames) would have underscores instead in their wheel filenames? | 13:30 |
| sdague | the proxy api for nova actually kind of discourages providers from exposing the real apis in a way | 13:30 |
| mordred | sdague: or it can figure out whether to use glanceclient or nova extensions | 13:30 |
| sdague | or we just abolish extensions :) | 13:30 |
| fungi | mordred: dhellmann: by way of example, http://logs.openstack.org/4e/4e5fee2de947e047e3551a42800d9826de9d110d/release/diskimage-builder-tarball/96d71ca/console.html#_2014-06-06_11_44_14_931 | 13:31 |
| sdague | and make it all mandatory | 13:31 |
| mordred | fungi: yes | 13:31 |
| *** salv-orlando_ has joined #openstack-infra | 13:31 | |
| mordred | os_apply_config-0.1.10-py27-none-any.whl <-- just made that locally | 13:31 |
| mestery | Morning infra folks | 13:31 |
| mordred | hey mestery | 13:31 |
| * mestery catches up on the state of the gate | 13:31 | |
| mriedem | anteaya: thanks, that tempest patch of mine seems to be failing on something new, but i think we've seen before | 13:31 |
| mriedem | http://logs.openstack.org/31/95831/4/check/check-tempest-dsvm-full/136b7b0/console.html#_2014-06-06_00_45_35_223 | 13:31 |
| mordred | mestery: "poopy" | 13:31 |
| mestery | mordred: :( | 13:31 |
| mordred | mestery: but with cautious optimism? | 13:32 |
| mestery | mordred: Glass half full, I like it! :) | 13:32 |
| *** e0ne has quit IRC | 13:32 | |
| mordred | mestery: half full of vodka, I always say | 13:32 |
| mestery | Ha! | 13:32 |
| dhellmann | fungi: yes, I think because the name is set off from the version with a hyphen | 13:32 |
| fungi | mordred: okay, right now the pypi-wheel-upload.sh script uses pypi-extract-name.py to determine the wheel name, which gets hyphens instead of underscores. is just a simple s/-/_/g sufficient or are there other transformations we should also apply to be more forward-looking? | 13:32 |
| fungi | dhellmann: ^ | 13:33 |
| dhellmann | fungi: I don't know the answer to that. Maybe dstufft will? | 13:33 |
| *** salv-orlando has quit IRC | 13:33 | |
| yolanda | hi, trying to push corrections to a patch, but i'm getting this error: ! [remote rejected] HEAD -> refs/publish/master/dib (you are not allowed to upload merges) | 13:33 |
| fungi | mordred: dhellmann: keeping in mind that we should not actually import the module for security reasons | 13:33 |
| yolanda | i just got problems on the rebase, fixed them, and now the review gives that error | 13:33 |
| yolanda | is that something i could solve by some way, without redoing it? | 13:34 |
| mordred | fungi: there is a function in setuptools somewhere that knows how to do a transformation | 13:34 |
| mordred | fungi: into a "safe name" | 13:34 |
| fungi | mordred: oh, perfect! i'll see if i can locate it | 13:34 |
| mordred | fungi: I'm thrilled that the tarball and the wheel have different naming conventions | 13:35 |
| fungi | mordred: so am i. picking this apart has made my morning | 13:35 |
| mordred | fungi: so we might have to make a --wheel optino to pypi-extract-name? | 13:35 |
| *** starmer has joined #openstack-infra | 13:35 | |
| fungi | mordred: that's the plan | 13:35 |
| *** changbl has quit IRC | 13:36 | |
| fungi | maybe not that exact option name, but yes some cli flag to switch the output | 13:36 |
| mestery | So, regarding bug 1326510, this patch (https://review.openstack.org/#/c/97218/) should help to troubleshoot the failure cases in the gate. | 13:36 |
| uvirtbot | Launchpad bug 1326510 in neutron "tempest.api.network.test_fwaas_extensions.FWaaSExtensionTestJSON fails with 409" [Undecided,New] https://launchpad.net/bugs/1326510 | 13:36 |
| anteaya | mriedem: https://review.openstack.org/#/c/98290/ | 13:36 |
| mordred | fungi: pkg_resources.safe_name | 13:36 |
| anteaya | I found that last night, it looks like the same bug | 13:37 |
| mriedem | anteaya: ah cool, hadn't gotten that far yet | 13:37 |
| openstackgerrit | Yuriy Taraday proposed a change to openstack-infra/git-review: Ignore newline in bp/bug search in commit message https://review.openstack.org/98324 | 13:37 |
| *** matrohon has quit IRC | 13:37 | |
| fungi | mordred: you just saved me 10 minutes ;) thanks! | 13:37 |
| mordred | fungi: wheel.bdist_wheel.safer_name | 13:37 |
| anteaya | yeah, I am just reading backscroll myself | 13:37 |
| *** pblaho has quit IRC | 13:37 | |
| mordred | fungi: actually is what's in bdist_wheel | 13:37 |
| *** salv-orlando_ has quit IRC | 13:37 | |
| fungi | ooh | 13:37 |
| *** masayukig has quit IRC | 13:38 | |
| fungi | dhellmann: on a related note, we missed this when we renamed/split the pypi job names... https://review.openstack.org/98405 | 13:38 |
| openstackgerrit | A change was merged to openstack-infra/devstack-gate: Put cached files in old/ dirs too https://review.openstack.org/98265 | 13:39 |
| *** otherwiseguy has quit IRC | 13:39 | |
| dhellmann | fungi: that tells zuul it can reuse the node safely? | 13:39 |
| anteaya | okay finished ttx's blog post and scrollback | 13:40 |
| mordred | dhellmann: yes, and not to tell jenkins to disable the node | 13:40 |
| fungi | dhellmann: right | 13:40 |
| mordred | morning anteaya | 13:40 |
| anteaya | thanks for the blog post, ttx, I always do like the math | 13:40 |
| fungi | dhellmann: otherwise it does this after each run... https://jenkins.openstack.org/computer/pypi.slave.openstack.org/ | 13:40 |
| anteaya | mordred: good morning | 13:40 |
| *** starmer has quit IRC | 13:40 | |
| fungi | "Offline due to Gearman request" | 13:40 |
| anteaya | mordred: you appear to have been up all night, or a good chunk of it | 13:40 |
| dhellmann | got it | 13:40 |
| *** bknudson has joined #openstack-infra | 13:41 | |
| fungi | easy to spot when you know what you're looking for, but otherwise confusing behavior ;) | 13:41 |
| anteaya | yolanda: the system thinks you are submitting a merge, which you can't do | 13:41 |
| mordred | anteaya: I got at least 3 hours of sleep | 13:41 |
| anteaya | mordred: luxury | 13:41 |
| yolanda | hi anteaya, but is there any way to solve that? | 13:41 |
| *** salv-orlando has joined #openstack-infra | 13:41 | |
| mriedem | anteaya: there are actually already 4 bugs for that test_volume_boot_pattern failure | 13:42 |
| ttx | DAMN YOU LAUNCHPAD | 13:42 |
| mriedem | anteaya: so i'm going to start making duplicates | 13:42 |
| *** e0ne has joined #openstack-infra | 13:42 | |
| yolanda | i didn't do a merge,, just trying to rebase properly with git review | 13:42 |
| mordred | yolanda: you may need to do some fancy things with git reset and/or commit --amend - but I'm about to step AFK so I'm a bad person to try to help | 13:42 |
| mordred | yolanda: somehow in the process something has decided to do a merge | 13:42 |
| *** homeless has joined #openstack-infra | 13:42 | |
| anteaya | mriedem: really? I checked last night before I created mine | 13:42 |
| ttx | You can't create a blueprint from the API... so I can't write my spec2bp file-a-blueprint-from-this-psec-please helper | 13:42 |
| ttx | spec* | 13:42 |
| yolanda | may be better to just preserve the files and redo it | 13:42 |
| anteaya | mriedem: was there one in existance before I created my bug report? | 13:42 |
| mordred | yolanda: did you happen to ever run the command "git pull" without the --ff-only flag? | 13:42 |
| fungi | yolanda: it's also possible you pulled from a remote into your local branch when there was some divergence between them, and didn't specify --ff-only | 13:43 |
| fungi | that | 13:43 |
| mestery | Yay! https://review.openstack.org/#/c/97245/ just merged! | 13:43 |
| yolanda | mordred, no, i was going to commit new changes to nodepool, then git review complained, i just fixed the problem and git a rebase --continue | 13:43 |
| mordred | mestery: is that good? | 13:43 |
| *** apevec has quit IRC | 13:43 | |
| anteaya | mestery: yay | 13:43 |
| yolanda | i suppose i did something wrong in the middle | 13:43 |
| mestery | mordred: Yes! We think that was the cause of the "ssh timeout" increases with neutroin. | 13:43 |
| * anteaya releases white doves | 13:43 | |
| mestery | anteaya: Ha! | 13:43 |
| mordred | mestery: ooh! lovely | 13:43 |
| anteaya | neutroin | 13:43 |
| mestery | mordred: Now we can debug the real failure, but hopefully the gate will settle down with those while we do it. | 13:44 |
| anteaya | the drug we all need to be taiking | 13:44 |
| mestery | anteaya: :) | 13:44 |
| * fungi 's doin' the neutron dance | 13:44 | |
| * mestery ^5s fungi. | 13:44 | |
| * ttx sobs heavily | 13:44 | |
| *** andreaf has quit IRC | 13:44 | |
| * fungi 's also showing his age | 13:44 | |
| mordred | ttx: that's pretty awful | 13:44 |
| *** andreaf has joined #openstack-infra | 13:44 | |
| mordred | fungi: I wrote a devstack patch earlier related to apt-get update you may want to weight in on | 13:45 |
| fungi | ttx: i bet the planned storyboard api has that, plus unicorns AND rainbows | 13:45 |
| mordred | fungi: https://review.openstack.org/98398 | 13:45 |
| fungi | mordred: cool, i'll find it once i finish fixing this | 13:45 |
| mordred | fungi: I bet the current storyboard api has that | 13:45 |
| fungi | oh, no need to find ;) | 13:45 |
| anteaya | I need to get some breakfast, biab | 13:45 |
| mordred | fungi: as it's not possible to do anything in storyboard if it doesn't have an api :) | 13:46 |
| ttx | fungi: well, storyboard can't do anything that is not in the REST API, for one thing. | 13:46 |
| fungi | mordred: the current storyboard api has unicorns and rainbows? i'm so there | 13:46 |
| *** zns has joined #openstack-infra | 13:46 | |
| mordred | fungi: POST https://storyboard.openstack.org/api/unicorn | 13:46 |
| ttx | damn, we are already beyond LP | 13:46 |
| mordred | ttx: yup | 13:47 |
| mordred | ttx: does that mean you're declaring victory? | 13:47 |
| *** rcarrill` has joined #openstack-infra | 13:47 | |
| ttx | Bug 1 solved! | 13:47 |
| fungi | time to plant that flag | 13:47 |
| *** salv-orlando_ has joined #openstack-infra | 13:48 | |
| *** zz_gondoi is now known as gondoi | 13:48 | |
| *** jgallard has joined #openstack-infra | 13:48 | |
| *** rcarrillocruz has quit IRC | 13:48 | |
| openstackgerrit | Yuriy Taraday proposed a change to openstack-infra/git-review: Ignore newline in bp/bug search in commit message https://review.openstack.org/98324 | 13:49 |
| *** salv-orlando_ has quit IRC | 13:49 | |
| *** salv-orlando has quit IRC | 13:50 | |
| *** starmer has joined #openstack-infra | 13:50 | |
| *** Hal has joined #openstack-infra | 13:51 | |
| *** Hal has quit IRC | 13:51 | |
| *** salv-orlando has joined #openstack-infra | 13:52 | |
| openstackgerrit | Yuriy Taraday proposed a change to openstack-infra/git-review: Retry command on "hash mismatch" error in tests https://review.openstack.org/98354 | 13:52 |
| openstackgerrit | yolanda.robla proposed a change to openstack-infra/nodepool: Build images using diskimage-builder https://review.openstack.org/46482 | 13:54 |
| *** salv-orlando has quit IRC | 13:54 | |
| yolanda | mordred, pushed some changes to nodepool dib stuff, but it's not yet properly working | 13:54 |
| *** starmer has quit IRC | 13:55 | |
| *** yfried has joined #openstack-infra | 13:56 | |
| *** james_li has joined #openstack-infra | 13:57 | |
| *** gondoi is now known as zz_gondoi | 13:59 | |
| openstackgerrit | Ilya Sviridov proposed a change to openstack-infra/config: Added new python-magnetodbclient project to Stackforge https://review.openstack.org/91050 | 14:03 |
| *** changbl has joined #openstack-infra | 14:05 | |
| *** starmer has joined #openstack-infra | 14:05 | |
| openstackgerrit | Mark McLoughlin proposed a change to openstack-infra/elastic-recheck: Add query for volume detach failure https://review.openstack.org/98421 | 14:06 |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/config: Determine wheel names for PyPI uploading https://review.openstack.org/98422 | 14:06 |
| fungi | mordred: dhellmann: ^ | 14:06 |
| openstackgerrit | Dan Prince proposed a change to openstack-infra/nodepool: Add support for network labels https://review.openstack.org/98287 | 14:07 |
| *** salv-orlando has joined #openstack-infra | 14:08 | |
| *** yolanda has quit IRC | 14:08 | |
| *** markmcclain has joined #openstack-infra | 14:09 | |
| *** salv-orlando has quit IRC | 14:09 | |
| *** starmer has quit IRC | 14:10 | |
| openstackgerrit | A change was merged to openstack-infra/elastic-recheck: Add query for volume detach failure https://review.openstack.org/98421 | 14:12 |
| *** jistr has quit IRC | 14:12 | |
| krotscheck | I’m totally putting that up as a patch | 14:12 |
| *** jgrimm has joined #openstack-infra | 14:13 | |
| *** jistr has joined #openstack-infra | 14:14 | |
| anteaya | krotscheck: the unicorn patch? | 14:14 |
| mordred_phone | krotscheck: ++ | 14:14 |
| krotscheck | All I need is a CC-licensed ASCII art unicorn | 14:14 |
| fungi | but /api/unicorn is nothing without /api/rainbow to go with it | 14:16 |
| slamont | krotscheck: http://www.ascii-art.de/ascii/uvw/unicorn.txt I don`t if they are CC-licensed :) | 14:16 |
| slamont | know* | 14:16 |
| *** aburaschi has joined #openstack-infra | 14:16 | |
| anteaya | I like the first one | 14:17 |
| openstackgerrit | A change was merged to openstack-infra/config: PyPI upload jobs should not take slave offline https://review.openstack.org/98405 | 14:17 |
| *** bnemec is now known as beekneemech | 14:17 | |
| anteaya | the last one looks like it has been in a fight | 14:17 |
| *** salv-orlando has joined #openstack-infra | 14:17 | |
| *** g_306396236 has joined #openstack-infra | 14:17 | |
| *** g_306396236 has left #openstack-infra | 14:17 | |
| *** derekh_ is now known as derekh_underscor | 14:17 | |
| *** Lingo__ has joined #openstack-infra | 14:19 | |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/config: Determine wheel names for PyPI uploading https://review.openstack.org/98422 | 14:19 |
| fungi | dhellmann: ^ with your suggestion implemented | 14:19 |
| krotscheck | Ok, enough looking at unicorns, back to figuring out oslo.messaging | 14:19 |
| *** salv-orlando has quit IRC | 14:20 | |
| dhellmann | fungi: LGTM | 14:20 |
| *** starmer has joined #openstack-infra | 14:20 | |
| fungi | actually, the ocd in me wants to invert the conditional logic in that patch. one more iteration | 14:21 |
| openstackgerrit | Michael Krotscheck proposed a change to openstack-infra/storyboard-webclient: Added simple logged-in dashboard https://review.openstack.org/92662 | 14:21 |
| *** wenlock has joined #openstack-infra | 14:22 | |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/config: Determine wheel names for PyPI uploading https://review.openstack.org/98422 | 14:22 |
| dhellmann | fungi: you could also use argparse and not handle the missing option explicitly yourself | 14:22 |
| *** salv-orlando has joined #openstack-infra | 14:22 | |
| ttx | now I need to reuse a damn browser window to get that blueprint created. | 14:22 |
| *** Lingo__ has quit IRC | 14:23 | |
| fungi | dhellmann: yeah, though that gets into being at least as much code as just dealing with command-line arguments directly when it's only got two options and normally only gets called from other scripts in the same directory anyway | 14:23 |
| *** salv-orlando has quit IRC | 14:23 | |
| dhellmann | true | 14:24 |
| fungi | if it grew five or six options, then totally | 14:24 |
| *** salv-orlando has joined #openstack-infra | 14:25 | |
| *** starmer has quit IRC | 14:25 | |
| *** james_li has quit IRC | 14:26 | |
| openstackgerrit | Brant Knudson proposed a change to openstack-infra/config: For requirements, run job to verify that requirements overlap https://review.openstack.org/98426 | 14:27 |
| openstackgerrit | Brant Knudson proposed a change to openstack-infra/config: For requirements, run job to verify that requirements overlap https://review.openstack.org/98426 | 14:28 |
| openstackgerrit | Nikita Konovalov proposed a change to openstack-infra/storyboard-webclient: Tests updated to check StoryBoard spelling https://review.openstack.org/98428 | 14:28 |
| openstackgerrit | Brant Knudson proposed a change to openstack-infra/config: For requirements, run job to verify that requirements overlap https://review.openstack.org/98426 | 14:29 |
| krotscheck | NikitaKonovalov++ | 14:29 |
| *** fbo_away is now known as fbo | 14:29 | |
| *** gilliard has quit IRC | 14:29 | |
| *** salv-orlando has quit IRC | 14:30 | |
| *** otherwiseguy has joined #openstack-infra | 14:30 | |
| *** trinaths has joined #openstack-infra | 14:31 | |
| *** yamahata has joined #openstack-infra | 14:33 | |
| anteaya | ttx :( | 14:33 |
| ttx | will be ugly be still better than all manually creating it, i guess | 14:34 |
| * ttx closes shop early today | 14:34 | |
| *** thomasbiege has quit IRC | 14:35 | |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/config: Determine wheel names for PyPI uploading https://review.openstack.org/98422 | 14:35 |
| fungi | dhellmann: production tests turned up one more problem with the original script ^ | 14:35 |
| *** starmer has joined #openstack-infra | 14:35 | |
| fungi | should be ...0.1.19-py2.py3... rather than ...0.1.19.py2.py3... | 14:36 |
| anteaya | ttx I hear that | 14:36 |
| anteaya | I just ate breakfast on the deck in the shade, listening to the twittering birds and the chain saws, and watching the sunlight play on the waves of the lake | 14:36 |
| anteaya | and decided my weekend starts tonight | 14:37 |
| *** salv-orlando has joined #openstack-infra | 14:37 | |
| *** e0ne_ has joined #openstack-infra | 14:38 | |
| dhellmann | fungi: fiddly :-) | 14:38 |
| *** changbl has quit IRC | 14:38 | |
| fungi | dhellmann: and with that in place (tested with a puppet dev env), there is now a wheel at https://pypi.python.org/pypi/diskimage-builder | 14:39 |
| dhellmann | \o/ | 14:39 |
| *** bknudson has quit IRC | 14:39 | |
| dhellmann | fungi: does that mean we're ready to test alpha releases of oslo libs? | 14:40 |
| *** vhoward has joined #openstack-infra | 14:40 | |
| fungi | dhellmann: that would probably be the next step, yes | 14:40 |
| *** starmer has quit IRC | 14:41 | |
| fungi | first i'm bringing the pypi slave back online to let the pent up release jobs finally run free from the queue again | 14:41 |
| *** e0ne has quit IRC | 14:41 | |
| dhellmann | fungi: ok, I have a couple of different meetings today, but I should be able to find a window to test. I know harlowja_away also wanted to test with taskflow. | 14:41 |
| openstackgerrit | Anita Kuno proposed a change to openstack-infra/elastic-recheck: Add query for nova test_volume_boot_pattern bug 1298472 https://review.openstack.org/98290 | 14:42 |
| uvirtbot | Launchpad bug 1298472 in tempest "tempest.scenario.test_volume_boot_pattern.TestVolumeBootPattern.test_volume_boot_pattern fails with TimeoutException on SSH connection" [Undecided,New] https://launchpad.net/bugs/1298472 | 14:42 |
| openstackgerrit | Roger Luethi proposed a change to openstack-dev/hacking: Fix a typo in HACKING.rst. https://review.openstack.org/98435 | 14:42 |
| fungi | grr... new problem exposed by https://jenkins.openstack.org/job/os-cloud-config-pypi-both-upload/1/console | 14:43 |
| fungi | it tried to get os_cloud_config-0.0.4-py27-none-any.whl instead of os_cloud_config-0.0.4-py2-none-any.whl | 14:43 |
| *** derekh_underscor is now known as derekh_ | 14:43 | |
| * dhellmann headdesk | 14:44 | |
| fungi | dhellmann: mordred_phone: have a moment to school me in the difference between a py27 wheel and a py2 wheel? | 14:44 |
| dhellmann | I don't know the answer to that either. :-/ | 14:44 |
| *** rgerganov has quit IRC | 14:44 | |
| dhellmann | Alex_Gaynor, dstufft : ^^ | 14:44 |
| * Guest33002 is away: Occupé | 14:44 | |
| Alex_Gaynor | fungi: py2 wheel works on any python2 (e.g. 2.6) and py27 wheel is only python 2.7 | 14:45 |
| dstufft | py27 or higher techincally | 14:45 |
| fungi | Alex_Gaynor: i sort of assumed that much... more wondering how i guess which will be built | 14:45 |
| fungi | is that encoded in the setup.cfg? | 14:45 |
| dstufft | I think it used to default to pyXY | 14:45 |
| dstufft | now it defaults to pyX | 14:45 |
| dstufft | and you can control that now IIRC | 14:46 |
| Alex_Gaynor | fungi: if there's no [universal] wheel = 1 you get a wheel specific to your python, not sure how specific, if it does have that you get a py2.py3 wheel | 14:46 |
| fungi | it's possible we have py27 hard-coded as a fallback in one of our scripts. checking | 14:47 |
| *** thedodd has joined #openstack-infra | 14:47 | |
| *** bknudson has joined #openstack-infra | 14:47 | |
| fungi | yep, fix incoming | 14:48 |
| *** eharney has joined #openstack-infra | 14:48 | |
| *** yamahata has quit IRC | 14:48 | |
| *** yamahata has joined #openstack-infra | 14:49 | |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/config: Determine wheel names for PyPI uploading https://review.openstack.org/98422 | 14:49 |
| fungi | https://review.openstack.org/#/c/98422/4..5/modules/openstack_project/files/slave_scripts/pypi-extract-universal.py should take care of it | 14:50 |
| *** mmaglana has joined #openstack-infra | 14:50 | |
| fungi | mordred_phone: is that ^ an overly-simple assumption? | 14:50 |
| *** starmer has joined #openstack-infra | 14:50 | |
| *** isviridov is now known as isviridov|meetin | 14:50 | |
| *** virmitio has joined #openstack-infra | 14:51 | |
| openstackgerrit | Anita Kuno proposed a change to openstack-infra/elastic-recheck: Add query for nova test_volume_boot_pattern bug 1298472 https://review.openstack.org/98290 | 14:52 |
| *** rdopiera has quit IRC | 14:52 | |
| uvirtbot | Launchpad bug 1298472 in tempest "tempest.scenario.test_volume_boot_pattern.TestVolumeBootPattern.test_volume_boot_pattern fails with TimeoutException on SSH connection" [Undecided,New] https://launchpad.net/bugs/1298472 | 14:52 |
| *** atarakt has joined #openstack-infra | 14:52 | |
| openstackgerrit | Roger Luethi proposed a change to openstack-dev/hacking: Fix a typo in HACKING.rst https://review.openstack.org/98435 | 14:54 |
| openstackgerrit | Fathi Boudra proposed a change to openstack-infra/jenkins-job-builder: builders: add 'publish over ssh' support as a build step https://review.openstack.org/98437 | 14:54 |
| *** starmer has quit IRC | 14:54 | |
| *** gokrokve has joined #openstack-infra | 14:56 | |
| *** gokrokve_ has joined #openstack-infra | 14:57 | |
| *** nati_ueno has joined #openstack-infra | 14:57 | |
| openstackgerrit | Kiall Mac Innes proposed a change to openstack-infra/config: Enable voting for recently added Designate jobs https://review.openstack.org/98439 | 14:58 |
| *** bruff has joined #openstack-infra | 14:58 | |
| openstackgerrit | Christian Berendt proposed a change to openstack-infra/zuul: replace dict.iteritems() with six.iteritems(dict) https://review.openstack.org/98440 | 14:58 |
| *** gokrokv__ has joined #openstack-infra | 14:59 | |
| *** _nadya_ has quit IRC | 14:59 | |
| *** _nadya_ has joined #openstack-infra | 14:59 | |
| fungi | dhellmann: another new problem! tripleo_image_elements-0.7.7-cp27-none-linux_x86_64.whl (but pypi-wheel-upload.sh assumes -none-any.whl) | 15:00 |
| fungi | ideas? | 15:00 |
| *** e0ne has joined #openstack-infra | 15:00 | |
| Kiall | SergeyLukjanov / mordred_phone / fungi - could I get a +2/+A on a trivial Zuul layout change? Enables voting for some recently added Designate jobs - TC waiting to see these in action before they vote on incubation. Thanks a million :) https://review.openstack.org/#/c/98439/ | 15:00 |
| *** markmcclain has quit IRC | 15:00 | |
| *** gokrokve has quit IRC | 15:01 | |
| dhellmann | fungi: I think PyPI doesn't allow uploading platform-specific wheels for linux? | 15:01 |
| *** e0ne_ has quit IRC | 15:01 | |
| *** gokrokve_ has quit IRC | 15:01 | |
| dhellmann | because the encoding doesn't support a notion of "which" linux, and it's easier to get the dev tools for linux than other platforms where it's easy to answer the "which" question | 15:02 |
| openstackgerrit | Christian Berendt proposed a change to openstack-infra/zuul: replace dict.iteritems() with six.iteritems(dict) https://review.openstack.org/98440 | 15:02 |
| dhellmann | at least that's what I remember, but that's old info and could be stale | 15:02 |
| *** ominakov_ has joined #openstack-infra | 15:02 | |
| openstackgerrit | yolanda.robla proposed a change to openstack-infra/nodepool: Build images using diskimage-builder https://review.openstack.org/46482 | 15:03 |
| Kiall | Awesome - thanks fungi | 15:03 |
| fungi | dhellmann: yeah, so we should probably consider this to be a bug in tripleo-image-elements setup? | 15:03 |
| dhellmann | fungi: could the upload job report that it found a wheel that is platform-specific, and then skip uploading it without triggering an error? | 15:03 |
| dhellmann | fungi: that's another possible interpretation :-) | 15:03 |
| fungi | dhellmann: it would need to scrape the tarballs site to find other names besides the one it's looking for | 15:03 |
| dhellmann | I assumed they had a reason for making it platform-specific, but maybe that's just some default setting that isn't being overridden | 15:03 |
| dhellmann | fungi: oh, right, this is downloading the file from somewhere else | 15:04 |
| fungi | i'll see if i can spot the issue in tie | 15:04 |
| *** rlandy has quit IRC | 15:05 | |
| *** ominakov has quit IRC | 15:05 | |
| *** esp has joined #openstack-infra | 15:06 | |
| fungi | huh... is has wheel.universal=1 | 15:06 |
| esp | good morning folks, can anyone assist me with a question about a patch waiting to merge? | 15:07 |
| *** basha has joined #openstack-infra | 15:07 | |
| *** adalbas has quit IRC | 15:07 | |
| openstackgerrit | A change was merged to openstack-infra/config: Enable voting for recently added Designate jobs https://review.openstack.org/98439 | 15:07 |
| anteaya | esp if your question is relatted to how long it is taking to merge, check out http://status.openstack.org/zuul/ | 15:08 |
| *** james_li has joined #openstack-infra | 15:08 | |
| anteaya | esp search for your patch id number and you will find the spot in the queue, along with how long it has been sitting there | 15:08 |
| openstackgerrit | Christian Berendt proposed a change to openstack-infra/elastic-recheck: Prettified all HTML files https://review.openstack.org/98445 | 15:08 |
| *** briancurtin has left #openstack-infra | 15:08 | |
| esp | thx anteaya! | 15:09 |
| anteaya | esp my pleasure | 15:09 |
| esp | anteaya: https://review.openstack.org/#/c/97997/ | 15:09 |
| openstackgerrit | Christian Berendt proposed a change to openstack-infra/zuul: replace dict.iteritems() with six.iteritems(dict) https://review.openstack.org/98440 | 15:10 |
| anteaya | esp: well better than halfway | 15:11 |
| *** atiwari has joined #openstack-infra | 15:11 | |
| esp | anteaya: lol | 15:11 |
| anteaya | esp: I don't see it being tested before the weekend | 15:11 |
| esp | anteaya: I hope I didn’t mess anything up | 15:11 |
| *** mmaglana has quit IRC | 15:11 | |
| anteaya | esp you aren't on the bottom | 15:11 |
| anteaya | esp what do you mean by mess anything up? | 15:11 |
| anteaya | esp we are dealing with race conditions in the gate | 15:12 |
| *** mmaglana has joined #openstack-infra | 15:12 | |
| anteaya | http://status.openstack.org/elastic-recheck/gate.html | 15:12 |
| esp | anteaya: well there were a lot of things going in, I wonder if it was contributing to the slowness | 15:12 |
| dhellmann | fungi: it appears to be caused by having only data files, and not a pure python package, included in setup.cfg | 15:12 |
| esp | anteaya: understood. | 15:12 |
| anteaya | so while you are waiting, select any bug on that page and dive in | 15:12 |
| esp | anteaya: k, I’ll go dumpster diving for some bugs :) | 15:13 |
| fungi | dhellmann: ooh, good catch. i couldn't find any smoking gun in the package config | 15:13 |
| anteaya | folks are standing by in -qa to answer questions from anyone willing to help address these bugs | 15:13 |
| anteaya | esp welcome to the party | 15:13 |
| dhellmann | fungi: hang on, I might be wrong there, trying again | 15:13 |
| dhellmann | fungi: no, just a dirty workspace | 15:14 |
| dhellmann | fungi: adding "packages = elements" to the files section of setup.cfg, removing the existing build artifacts and egg-info dir, and rebuilding gives me tripleo_image_elements-0.7.7-py2.py3-none-any.whl | 15:14 |
| esp | thx anteaya, I’ll preach patience to folks on our side! | 15:14 |
| anteaya | esp and share the gospel of the gate bugs for the impatient | 15:15 |
| dhellmann | fungi: of course, that will install a package called "elements" in dist-packages, which isn't a terribly unique name and includes stuff that really ought to go in the data directory | 15:15 |
| dhellmann | fungi: I think I'd call this a setuptools bug, but we might be able to work around it in some safe way by adding a little mini package to that repo or renaming the elements package to something that isn't going to collide with other libraries | 15:16 |
| *** mmaglana has quit IRC | 15:16 | |
| dhellmann | fungi: https://pypi.python.org/pypi/Elements | 15:16 |
| *** salv-orlando has quit IRC | 15:16 | |
| *** salv-orlando_ has joined #openstack-infra | 15:16 | |
| esp | anteaya: lol, ok will do | 15:17 |
| *** dizquierdo has joined #openstack-infra | 15:17 | |
| *** basha has quit IRC | 15:18 | |
| anteaya | esp thanks | 15:18 |
| *** isviridov|meetin is now known as isviridov | 15:19 | |
| *** salv-orlando_ has quit IRC | 15:19 | |
| *** starmer has joined #openstack-infra | 15:20 | |
| fungi | dhellmann: yeah, it's likely we're going to run into more of this with devs packaging non-python things for upload to pypi | 15:20 |
| * dhellmann nods | 15:21 | |
| *** adalbas has joined #openstack-infra | 15:21 | |
| *** tnurlygayanov has quit IRC | 15:22 | |
| *** e0ne has quit IRC | 15:22 | |
| *** tnurlygayanov has joined #openstack-infra | 15:22 | |
| *** openstackgerrit has quit IRC | 15:22 | |
| fungi | dhellmann: yeah, the sdist tarball ends up with Platform: UNKNOWN in its PKG-INFO | 15:23 |
| anteaya | come back openstackgerrit | 15:24 |
| *** mfer has joined #openstack-infra | 15:24 | |
| *** starmer has quit IRC | 15:25 | |
| *** medieval1 has joined #openstack-infra | 15:25 | |
| *** nati_ueno has quit IRC | 15:26 | |
| anteaya | SergeyLukjanov fungi mordred_phone openstackgerrit has disappeared | 15:26 |
| mestery | mtreinish: Regarding this one: https://review.openstack.org/#/c/97269/, I think it's still necessary even in the face of the other two reviews referenced. | 15:26 |
| fungi | mordred_phone: do you happen to have an opinion on the proper solution there? (wheel building for pure data packages, and tripleo-image-elements in particular) | 15:26 |
| mestery | mtreinish: https://review.openstack.org/#/c/97269/ is dumping the console logs for the VM, which is really helpful for debugging "ssh timeout" issues. Thoughts? | 15:27 |
| *** jpich has quit IRC | 15:28 | |
| mordred_phone | fungi: not yet, but I'll come up with one | 15:28 |
| dhellmann | fungi: if the wheel only contains data, is the data installed properly when the wheel is installed | 15:28 |
| *** ildikov has quit IRC | 15:28 | |
| *** esp has left #openstack-infra | 15:28 | |
| *** openstackgerrit has joined #openstack-infra | 15:29 | |
| arunkant | Hi..any update on gate issue mentioned in https://review.openstack.org/#/c/95842/.. waiting for this approval and merge. | 15:29 |
| bruff | Question around a failing pip install: I'm installing zuul using pip install and it appears to be some proxy issue. I'm using an ansible playbook which works OK from home but not at work. Plus I'm doing the same thing for Jenkins Job Builder which is completing at ok work. I've seen 'no route to host' and connection timeout errors. Is there any general advice around pip and proxies? | 15:29 |
| *** e0ne has joined #openstack-infra | 15:30 | |
| *** mburned has left #openstack-infra | 15:30 | |
| fungi | anteaya: restarted and it's back now | 15:30 |
| *** yolanda has joined #openstack-infra | 15:30 | |
| fungi | dhellmann: good question. not sure, since i don't really know what tie is expected to do | 15:31 |
| *** zns has quit IRC | 15:31 | |
| mtreinish | mestery: the other reviews I referenced I thought unified those connectivity check methods and both had the same _log_console call | 15:31 |
| *** zns has joined #openstack-infra | 15:32 | |
| yolanda | mordred, looks as disk-image-builder needs dkms package to properly finish installing, what will be the right place to install that dependency, or at least, document it? | 15:32 |
| *** zns has quit IRC | 15:32 | |
| mestery | mtreinish: I just ran with those other two, and never saw a console dump, and sure enough, it's not there. :( | 15:32 |
| fungi | dhellmann: when installing the tarball into a venv it seems to end up with things like share/tripleo-image-elements/tuskar/os-config-applier/etc/tuskar/tuskar.conf | 15:32 |
| *** zns has joined #openstack-infra | 15:33 | |
| fungi | dhellmann: so presumably if the wheel does the same... | 15:33 |
| dhellmann | fungi: that seems like a reasonable test | 15:33 |
| openstackgerrit | Sean Dague proposed a change to openstack-infra/elastic-recheck: fix 1327184 https://review.openstack.org/98452 | 15:33 |
| anteaya | fungi: thanks | 15:33 |
| *** ominakov_ has quit IRC | 15:33 | |
| zaro | morning | 15:33 |
| *** Guest33002 has quit IRC | 15:33 | |
| fungi | dhellmann: share/tripleo-image-elements/tuskar/os-config-applier/etc/tuskar/tuskar.conf | 15:34 |
| *** denis_makogon has quit IRC | 15:34 | |
| anteaya | morning zaro | 15:34 |
| fungi | dhellmann: seems so | 15:34 |
| *** nati_ueno has joined #openstack-infra | 15:34 | |
| *** mfer has quit IRC | 15:35 | |
| *** jp_at_hp has quit IRC | 15:35 | |
| trinaths | Good Morning | 15:35 |
| *** starmer has joined #openstack-infra | 15:35 | |
| fungi | dhellmann: maybe setting packages = tripleo_image_elements and just adding that dir with an empty __init__.py would be a suitable workaround | 15:36 |
| dhellmann | fungi: ok, then I guess the next step is to decide whether to rename "elements" or make a new python package in the tie repo so the wheel build will work | 15:36 |
| *** bruff has left #openstack-infra | 15:36 | |
| dhellmann | fungi: yeah, that was the "make a mini package" alternative that I didn't clearly articulate | 15:36 |
| mtreinish | mestery: I don't know why it would be any different. It just moves the method up a level but they're basically identical between the patches | 15:36 |
| *** jp_at_hp has joined #openstack-infra | 15:36 | |
| *** gyee has joined #openstack-infra | 15:36 | |
| fungi | dhellmann: well, it sort of just clicked with me that the mini package you were suggesting could simply be an empty one | 15:37 |
| *** dangers_away is now known as dangers | 15:37 | |
| mestery | mtreinish: I'll look closer, maybe I'm missing something. Thanks! | 15:37 |
| dhellmann | fungi: we might even be able to get away with just a module | 15:37 |
| *** mfer has joined #openstack-infra | 15:38 | |
| openstackgerrit | Michael Krotscheck proposed a change to openstack-infra/storyboard-webclient: Added field restrictions and error messages to project forms https://review.openstack.org/95873 | 15:38 |
| mestery | mtreinish: Argh, you're right, sorry about that. :( | 15:38 |
| fungi | dhellmann: good point. echo 'print("These are not the droids you're looking for...")' > tripleo_image_elements.py | 15:38 |
| mestery | mtreinish: Now I'm left wondering why I don't see the console dump when I'm running with those two patches, back to looking at that now. | 15:38 |
| *** nati_ueno has quit IRC | 15:39 | |
| mat128 | What's missing for https://review.openstack.org/#/c/98263/ to be merged? Verified and workflow? What are those? | 15:40 |
| anteaya | bruff: so are you saying that you are behind a firewall and that pip is upset when you are behind a firewall? | 15:40 |
| anteaya | hello trianths | 15:40 |
| *** starmer has quit IRC | 15:40 | |
| *** ihrachyshka_ is now known as ihrachyshka | 15:41 | |
| fungi | mat128: a devstack core reviewer needs to approve it by applying a workflow +1 vote | 15:41 |
| *** arunkant has quit IRC | 15:41 | |
| fungi | mat128: the automated ci will leave a +2 verified vote when it merges it, after one final pass of testing | 15:41 |
| *** social__ has quit IRC | 15:42 | |
| *** nati_ueno has joined #openstack-infra | 15:44 | |
| asselin | Hi, I'm trying to use my 3rd paty ci account to stream gerrit events, but permission is denied: http://paste.openstack.org/show/83131/ | 15:45 |
| anteaya | asselin: it doesnt' like your ssh key | 15:45 |
| *** isviridov is now known as isviridov|away | 15:45 | |
| anteaya | is the the correct ssh key that you submitted when you set up the account? | 15:45 |
| anteaya | and is your matching private key on the same machine? | 15:46 |
| asselin | anteaya, the private key is there. they should match I will double-check | 15:46 |
| fungi | anteaya: asselin: also possible it's using the wrong username... try doing whatever-username@review.openstack.org in the ssh command | 15:46 |
| anteaya | the host key verification succeeded | 15:46 |
| *** jgallard has quit IRC | 15:47 | |
| fungi | asselin: by default ssh assumes your remote username is identical to your local username unless you tell it otherwise | 15:47 |
| anteaya | ah yes, there is no username specified in the original command | 15:47 |
| asselin | fungi, anteaya ok I think that was it! it works now! | 15:48 |
| anteaya | kk | 15:48 |
| mat128 | fungi: thanks! I see workflow has been approved and found the change request in zuul's status page. ty | 15:48 |
| *** yfried has quit IRC | 15:49 | |
| fungi | mat128: you're welcome. now you just get to kick back and drink beer for ~48 hours until it finds its way through the currently jammed integrated gate queue, and hope it doesn't get failed out on some unrelated race condition in one of the tests ;) | 15:49 |
| *** starmer has joined #openstack-infra | 15:50 | |
| *** nati_ueno has quit IRC | 15:51 | |
| *** doude has quit IRC | 15:51 | |
| *** doude has joined #openstack-infra | 15:51 | |
| *** hashar has quit IRC | 15:51 | |
| *** openstackgerrit has quit IRC | 15:53 | |
| anteaya | dang he is off again | 15:53 |
| Kiall | ttx / jeblair - Designate DevStack job running + voting - details @ https://review.openstack.org/#/c/97609/ :) | 15:53 |
| anteaya | whatever gender openstackgerrit is | 15:53 |
| *** jistr has quit IRC | 15:54 | |
| *** ihrachyshka has quit IRC | 15:54 | |
| *** jistr has joined #openstack-infra | 15:54 | |
| *** starmer has quit IRC | 15:55 | |
| fungi | dhellmann: yeah, simplest workaround is to touch tripleo_image_elements.py and then add files.modules = tripleo_image_elements in setup.cfg. seems to dtrt after that | 15:57 |
| fungi | dhellmann: mordred_phone: i'll propose that ^ as a patch to tie and see how many people vomit all over it | 15:57 |
| dhellmann | fungi: that doesn't seem too invasive | 15:57 |
| *** markmcclain has joined #openstack-infra | 15:58 | |
| mordred_phone | anteaya: might be genderless | 15:58 |
| *** fbo is now known as fbo_away | 15:58 | |
| anteaya | mordred_phone: might be | 15:59 |
| anteaya | but I don't like calling things it | 15:59 |
| anteaya | especially things with a name | 15:59 |
| anteaya | maybe I'm too picky? | 15:59 |
| anteaya | nah, that can't be it | 16:00 |
| *** zns has quit IRC | 16:00 | |
| *** gokrokv__ has quit IRC | 16:00 | |
| *** arunkant has joined #openstack-infra | 16:00 | |
| *** atiwari has quit IRC | 16:01 | |
| *** nati_ueno has joined #openstack-infra | 16:02 | |
| *** jistr has quit IRC | 16:02 | |
| *** reed has joined #openstack-infra | 16:02 | |
| *** zns has joined #openstack-infra | 16:02 | |
| *** jistr has joined #openstack-infra | 16:02 | |
| fungi | dstufft: if you have a sec to look at https://review.openstack.org/98456 let me know if that's something which needs fixing upstream too | 16:02 |
| *** IvanBerezovskiy has left #openstack-infra | 16:04 | |
| *** zns has quit IRC | 16:04 | |
| *** trinaths has quit IRC | 16:04 | |
| *** rpodolyaka has left #openstack-infra | 16:04 | |
| *** arnaud__ has joined #openstack-infra | 16:05 | |
| *** zns has joined #openstack-infra | 16:05 | |
| *** starmer_ has joined #openstack-infra | 16:05 | |
| *** mmaglana has joined #openstack-infra | 16:06 | |
| *** basha has joined #openstack-infra | 16:06 | |
| *** e0ne has quit IRC | 16:08 | |
| *** jlibosva has quit IRC | 16:08 | |
| *** gyee has quit IRC | 16:08 | |
| *** jlibosva has joined #openstack-infra | 16:08 | |
| *** jlibosva has quit IRC | 16:09 | |
| *** starmer_ has quit IRC | 16:10 | |
| *** lcheng has joined #openstack-infra | 16:11 | |
| *** talluri has joined #openstack-infra | 16:11 | |
| *** dhellman_ has joined #openstack-infra | 16:12 | |
| *** marcoemorais has joined #openstack-infra | 16:13 | |
| *** doude has quit IRC | 16:13 | |
| *** basha has quit IRC | 16:14 | |
| *** trinaths has joined #openstack-infra | 16:15 | |
| *** markwash has joined #openstack-infra | 16:16 | |
| *** arnaud__ has quit IRC | 16:17 | |
| *** _nadya_ has quit IRC | 16:18 | |
| *** zns has quit IRC | 16:18 | |
| *** starmer_ has joined #openstack-infra | 16:20 | |
| *** rwsu has joined #openstack-infra | 16:21 | |
| *** ramashri has joined #openstack-infra | 16:21 | |
| clarkb | o/ | 16:23 |
| *** trinaths has quit IRC | 16:24 | |
| *** marun has joined #openstack-infra | 16:24 | |
| *** zns has joined #openstack-infra | 16:24 | |
| *** starmer_ has quit IRC | 16:25 | |
| *** aconrad has joined #openstack-infra | 16:25 | |
| *** loquacities has quit IRC | 16:25 | |
| *** mikal has quit IRC | 16:26 | |
| anteaya | hey clarkb | 16:26 |
| anteaya | clarkb: can you bring openstackstatus back when you have a moment? | 16:26 |
| pleia2 | morning | 16:26 |
| anteaya | morning pleia2 | 16:27 |
| anteaya | sorry openstackgerrit | 16:27 |
| *** mikal has joined #openstack-infra | 16:27 | |
| *** nati_ueno has quit IRC | 16:28 | |
| *** dizquierdo has quit IRC | 16:29 | |
| fungi | anteaya: i can kick it again | 16:29 |
| anteaya | fungi: thanks | 16:29 |
| anteaya | fungi: thought you might have gotten bored with kicking it | 16:29 |
| clarkb | if 97757 is brought up for promotion one more time .... :P | 16:30 |
| *** _nadya_ has joined #openstack-infra | 16:30 | |
| * mordred does not understand why that upsets so many people, given that it's completely blocking one of our programs from working | 16:31 | |
| clarkb | mordred: because it hasn't passed in like 5 attempts | 16:31 |
| clarkb | and all of those people keep complaining here instead of fixing the actual problems | 16:31 |
| clarkb | there have been no offers of help to fix the gate that I have seen from that group of people | 16:31 |
| *** aconrad_ has joined #openstack-infra | 16:31 | |
| clarkb | just requests to babysit their special thing | 16:31 |
| *** loquacities has joined #openstack-infra | 16:31 | |
| mordred | is that different from anyone else in the project? | 16:32 |
| fungi | well, to be fair, they've declared a holiday approving any new changes to ironic until the gate fixes itself | 16:32 |
| clarkb | everyone else in the project hasn't shown up here over and over asking for a promotion on a change that cannot pass | 16:32 |
| * fungi stares intently at the zuul status screen. doesn't want to miss the gate fixing itself | 16:33 | |
| mordred | clarkb: the change looks green to me and it looks like it's been green for quite some time | 16:33 |
| mordred | other than the usual gate failures | 16:33 |
| clarkb | 97757 has been promoted a bunch of times and not merged | 16:33 |
| clarkb | because of the state of the gate | 16:34 |
| clarkb | we need to fix the gate before 97757 should be promoted again | 16:34 |
| mordred | sure. I'm just saying that I think it's not helpful if we're snarky at people in our own project | 16:34 |
| *** maxbit_ has joined #openstack-infra | 16:34 | |
| mordred | who are also experiencing pain | 16:34 |
| *** zns has quit IRC | 16:34 | |
| clarkb | mordred: I think the issue is the disconnect | 16:34 |
| zaro | gerritbot down again? | 16:34 |
| clarkb | zaro: yes | 16:35 |
| *** maxbit has quit IRC | 16:35 | |
| clarkb | mordred: I appreciate they are hurting too | 16:35 |
| *** aconrad has quit IRC | 16:35 | |
| fungi | zaro: it's on it's way back up again now | 16:35 |
| zaro | cool | 16:35 |
| clarkb | mordred: but that should be a prompt for participating in the larger fixing | 16:35 |
| *** mwagner_lap has joined #openstack-infra | 16:35 | |
| fungi | zaro: keeps timing out its connection to the irc server it's chosen | 16:35 |
| mordred | clarkb: possibly so - I mainly just worry that we develop a bofh attitude at times and get angry with people for not "fixing the gate" - when I imagine actually doing so is outside of the abilities or knowledge of a large percentage of our user base | 16:35 |
| *** starmer_ has joined #openstack-infra | 16:36 | |
| *** openstackgerrit has joined #openstack-infra | 16:36 | |
| mordred | I mean, I'm pretty sure that one of the biggest issues is currently a neutron bug that several of the neutron cores are heads-down on, having reproduced it locally and still don't know how to fit | 16:36 |
| mordred | fix | 16:36 |
| *** ildikov has joined #openstack-infra | 16:37 | |
| *** rcarrillocruz has joined #openstack-infra | 16:37 | |
| clarkb | sure, but there are lots of ways you can participate without being head down on neutron | 16:37 |
| clarkb | classifying failures is a good place to start | 16:37 |
| *** godp1301 has quit IRC | 16:37 | |
| *** ociuhandu has quit IRC | 16:38 | |
| *** tkelsey has quit IRC | 16:38 | |
| clarkb | http://status.openstack.org/elastic-recheck/data/uncategorized.html | 16:38 |
| *** zns has joined #openstack-infra | 16:38 | |
| *** rcarrill` has quit IRC | 16:38 | |
| jogo | mordred: actually one of the big ones is a !neutron network bug | 16:39 |
| mordred | of course. I'm merely pointing out that us being snarky or snippy with people probably will not actually motivate them to take positive action - even though I do it myself all the time | 16:39 |
| *** aconrad_ has quit IRC | 16:39 | |
| anteaya | if anyone wants to learn how to categorized bugs I will show them what I know | 16:40 |
| jogo | mordred: https://review.openstack.org/#/c/98450/ | 16:40 |
| anteaya | and mriedem is teaching me, so can refine them after I have done the introductions | 16:40 |
| *** starmer_ has quit IRC | 16:40 | |
| anteaya | my cat keeps crawling in my lap | 16:42 |
| *** _nadya_ has quit IRC | 16:42 | |
| anteaya | everytime I don't notice I look down and there she is again | 16:42 |
| *** penguinRaider has quit IRC | 16:42 | |
| *** zns has quit IRC | 16:42 | |
| *** zns has joined #openstack-infra | 16:45 | |
| pleia2 | anteaya: aw :) | 16:48 |
| pleia2 | mine are sleeping on my bed, taunting me with their pile of snuggly blankets | 16:48 |
| mordred | pleia2: mmm. snuggly cat blankets | 16:49 |
| pleia2 | it's like the work from home siren song | 16:49 |
| openstackgerrit | Valeriy Ponomaryov proposed a change to openstack-infra/config: Added 'docs' job for manila project https://review.openstack.org/98465 | 16:50 |
| *** penguinRaider has joined #openstack-infra | 16:50 | |
| *** starmer_ has joined #openstack-infra | 16:50 | |
| anteaya | yup | 16:52 |
| anteaya | mine sleeps in teh clean laundry | 16:53 |
| anteaya | and I can't kick her out | 16:53 |
| anteaya | so all my clean laundry has cat hair on it | 16:53 |
| pleia2 | lol | 16:53 |
| anteaya | if come to accept my life for what it is | 16:54 |
| anteaya | s/if/I've | 16:54 |
| *** starmer_ has quit IRC | 16:55 | |
| *** marcoemorais has quit IRC | 16:56 | |
| *** marcoemorais has joined #openstack-infra | 16:56 | |
| *** atiwari has joined #openstack-infra | 16:58 | |
| *** apevec has joined #openstack-infra | 16:58 | |
| openstackgerrit | Khai Do proposed a change to openstack-infra/jenkins-job-builder: re-arrange docs for clarity https://review.openstack.org/98467 | 16:58 |
| openstackgerrit | Michael Krotscheck proposed a change to openstack-infra/config: Modularized StoryBoard Module https://review.openstack.org/98007 | 16:59 |
| *** derekh_ is now known as derekh_afk | 17:00 | |
| *** comstud is now known as bearhands | 17:00 | |
| *** aconrad_ has joined #openstack-infra | 17:01 | |
| *** nati_ueno has joined #openstack-infra | 17:02 | |
| pleia2 | jogo: we should do lunch, you in town on monday? | 17:04 |
| *** dansmith is now known as superdan | 17:04 | |
| jogo | pleia2: surprisingly yes I am | 17:05 |
| jogo | lunch monday it is | 17:05 |
| pleia2 | cool | 17:05 |
| *** starmer_ has joined #openstack-infra | 17:05 | |
| jogo | what time and where? | 17:06 |
| clarkb | mordred: http://paste.openstack.org/show/83144/ | 17:06 |
| clarkb | jogo: sdague ^ not sure if you have looked at numbers like that yet | 17:06 |
| jroll | does anyone around here know how to merge two launchpad accounts? | 17:06 |
| pleia2 | jogo: could do the grove again, say 1pm? | 17:06 |
| jogo | sounds good | 17:07 |
| * pleia2 calendars it up | 17:07 | |
| *** dhellman_ has quit IRC | 17:08 | |
| dstufft | fungi: oh I'm an idiot, I looked at that CR and then forgot to respond | 17:08 |
| dstufft | fungi: I think that is something that the upstream should probably handle out of the box | 17:08 |
| *** vhoward has left #openstack-infra | 17:09 | |
| clarkb | jogo: sdague mordred I think hpcloud-b may have shaken some things loose but not everything | 17:09 |
| fungi | dstufft: should as in probably already does and i'm missing a documented option? or as in needs fixing (and if so, where? setuptools?) | 17:09 |
| clarkb | there is only one bug in the handful I checked that really looks like it is only a region b failure | 17:09 |
| jogo | clarkb: when did the hpcloud-b switch kick in? | 17:09 |
| clarkb | (granted sample sizes are small and all that) | 17:09 |
| clarkb | jogo: friday | 17:09 |
| jogo | last friday? | 17:10 |
| clarkb | (a week ago, is it already friday?) | 17:10 |
| dstufft | fungi: well you can manually specify the tags in the latest wheel, but I think wheel should be smart enough to handle something without python files at all | 17:10 |
| *** starmer_ has quit IRC | 17:10 | |
| jogo | ahh so this new network bug isn't related probably | 17:10 |
| dstufft | fungi: and it'd be in bitbucket.org/pypa/wheel | 17:10 |
| clarkb | jogo: no doesn't look like it | 17:10 |
| clarkb | jogo: the mirror bug is definitely more prevalent in rax | 17:10 |
| fungi | dstufft: got it--thanks! | 17:10 |
| clarkb | jogo: and the ceilomter bug is an hpcloud thing | 17:10 |
| clarkb | but nothing drastic like all failures are hpcloud | 17:11 |
| *** vponomaryov has left #openstack-infra | 17:11 | |
| *** arnaud__ has joined #openstack-infra | 17:12 | |
| anteaya | jroll: why do you ask? | 17:12 |
| *** jogo is now known as flashgordon | 17:12 | |
| *** gokrokve has joined #openstack-infra | 17:13 | |
| *** harlowja_away is now known as harlowja | 17:13 | |
| clarkb | flashgordon: sdague mordred and I didn't look at the check numbers to avoid potential noise. Though it may be interesting to do anyways | 17:14 |
| *** ociuhandu has joined #openstack-infra | 17:14 | |
| *** zns has quit IRC | 17:15 | |
| jroll | anteaya: so, I have an older account with my personal email. I have a newer account that was registered with my work email, but managed to get the email in my profile set to my personal email. The only thing using my work email is auth, and I want to change that to my personal email, but it won't let me | 17:15 |
| jroll | anteaya: (because the email is registered already) | 17:16 |
| *** talluri has quit IRC | 17:16 | |
| reed | what's the root cause for the bad status of the gate? | 17:16 |
| *** markmcclain has quit IRC | 17:17 | |
| clarkb | reed: there is about 100 | 17:17 |
| fungi | jroll: it's worth noting that the openid provider implementation in launchpad generates a new openid for the merged account. as such if you do get lp admins to merge your two accounts, you'll end up with a new gerrit account when authenticating from lp | 17:17 |
| clarkb | reed: http://status.openstack.org/elastic-recheck/ that gives a really high overview of the state of the world | 17:17 |
| clarkb | * high level | 17:17 |
| jroll | fungi: gah. maybe there's a way to disable the other account and be able to add it to the other account | 17:18 |
| reed | thanks clarkb... i'm discussing with developers complaining about the time it takes to get his patch in, want to give them an overview of where things fail and make the argument that it's everybody's responsibility to fix them | 17:18 |
| *** ociuhandu has quit IRC | 17:19 | |
| clarkb | reed: pointing them at that page is a great place to start | 17:19 |
| *** aconrad_ has quit IRC | 17:19 | |
| clarkb | reed: also the uncategorized tab shows failures we haven't classified http://status.openstack.org/elastic-recheck/data/uncategorized.html | 17:19 |
| *** basha has joined #openstack-infra | 17:19 | |
| clarkb | reed: devs that want to help make it better than categorize bugs so that we have a better understanding of the problems which leads to fixing them | 17:19 |
| *** gyee has joined #openstack-infra | 17:19 | |
| clarkb | s/than/can/ | 17:20 |
| reed | clarkb, tnx | 17:20 |
| *** godp1301 has joined #openstack-infra | 17:20 | |
| *** starmer_ has joined #openstack-infra | 17:20 | |
| openstackgerrit | Michael Krotscheck proposed a change to openstack-infra/config: [WIP] Added RabbitMQ to StoryBoard https://review.openstack.org/98474 | 17:20 |
| reed | clarkb, it would help to have a link to the documentation of elasticrecheck from that page | 17:21 |
| clarkb | reed: patches welcome | 17:21 |
| *** Ryan_Lane has joined #openstack-infra | 17:21 | |
| clarkb | reed: also anteaya has volunteered to help folks wanting to classify bugs | 17:22 |
| reed | clarkb, I don't like that answer, I feel diminished every time I hear it | 17:22 |
| *** russellb is now known as rustlebee | 17:22 | |
| reed | clarkb, I'll file a bug as a start :) | 17:22 |
| clarkb | reed: I understand but we have been under a ton of pressure fixing the gate problems and cloud problems and so on | 17:22 |
| fungi | reed: we would appreciate a bug report, and a patch even more! ;) | 17:22 |
| clarkb | ++ | 17:23 |
| reed | clarkb, I know, I'm not offended or anything, I just stated how I felt | 17:23 |
| clarkb | let me get you a link to the docs | 17:23 |
| reed | clarkb, don't worry, I'll find it | 17:23 |
| *** zns has joined #openstack-infra | 17:23 | |
| *** thedodd has quit IRC | 17:23 | |
| krotscheck | Any +2’s out there feel like reviewing some unit test typo’s? https://review.openstack.org/#/c/98428/ | 17:23 |
| clarkb | reed: http://docs.openstack.org/infra/elastic-recheck/ | 17:23 |
| clarkb | I was going to do my first storyboard review and fungi swooped in and beat me to it | 17:24 |
| *** jistr has quit IRC | 17:24 | |
| fungi | clarkb: sorry about that | 17:25 |
| krotscheck | clarkb: Well then, how about this one? https://review.openstack.org/#/c/98007/ :D | 17:25 |
| clarkb | is that one easy too? | 17:25 |
| clarkb | +-1100 lines not easy | 17:25 |
| *** ominakov has joined #openstack-infra | 17:25 | |
| krotscheck | Ok, here’s an easy one: https://review.openstack.org/#/c/95763/ | 17:26 |
| *** starmer_ has quit IRC | 17:26 | |
| *** markmc has quit IRC | 17:27 | |
| *** talluri has joined #openstack-infra | 17:28 | |
| *** medieval1 has quit IRC | 17:29 | |
| pleia2 | we probably should have a rechecks page on ci.openstack.org that links to docs, I'll toss up some patches so we can get links together | 17:30 |
| pleia2 | (and on the status page too) | 17:31 |
| *** mwagner_lap has quit IRC | 17:31 | |
| *** zzelle has joined #openstack-infra | 17:31 | |
| anteaya | so yeah, it appears we still need to debate the use of the word certified around testing: http://lists.openstack.org/pipermail/openstack-dev/2014-June/036933.html | 17:31 |
| anteaya | I had thought I was clear at summit, the cinder folks still want to use the word | 17:31 |
| anteaya | ttx ^^ | 17:31 |
| anteaya | mordred: ^^ | 17:32 |
| krotscheck | fungi: clarkb: Thanks guys! | 17:32 |
| clarkb | anteaya: you don't want to certify that we are pep8 clean!? (fwiw I agree with you "certification" isn't really what we do nor do we want to lead people to thinking it is what we do) | 17:33 |
| *** thedodd has joined #openstack-infra | 17:33 | |
| openstackgerrit | A change was merged to openstack-infra/storyboard-webclient: Tests updated to check StoryBoard spelling https://review.openstack.org/98428 | 17:33 |
| clarkb | reed: oh and there is a thread on the ml which may help explain things /me gets link | 17:33 |
| anteaya | clarkb: are we pep8 clean? | 17:33 |
| clarkb | reed: http://lists.openstack.org/pipermail/openstack-dev/2014-June/036810.html | 17:33 |
| *** ominakov has quit IRC | 17:34 | |
| clarkb | anteaya: for certain versions of pep8 excluding any exceptions we have made | 17:34 |
| anteaya | there are a lot of projects skipping various pep8 rules if I do recall | 17:34 |
| anteaya | right | 17:34 |
| anteaya | so we are except for when we arent' | 17:34 |
| anteaya | should go on a banner | 17:34 |
| *** aconrad_ has joined #openstack-infra | 17:34 | |
| *** starmer_ has joined #openstack-infra | 17:35 | |
| openstackgerrit | Christian Berendt proposed a change to openstack-infra/releasestatus: Prettified all HTML files https://review.openstack.org/98476 | 17:37 |
| asselin | has anyone gotten zuul to work behind an http proxy? | 17:37 |
| clarkb | asselin: yup | 17:37 |
| clarkb | but not to an external gerrit though | 17:37 |
| asselin | clarkb, names? I've not succeded yet | 17:38 |
| clarkb | asselin: me | 17:38 |
| clarkb | well AaronGr jesusaurus et al really | 17:38 |
| clarkb | but I have hit it with a hammer too | 17:38 |
| asselin | clarkb, what do you mean by "not an external gerrit" | 17:38 |
| clarkb | asselin: it is speaking to a gerrit on the same side of the proxy as zuul | 17:38 |
| clarkb | only the slaves are on the other side of the firewall | 17:38 |
| jesusaurus | asselin: are you trying to set it up too? | 17:38 |
| openstackgerrit | Elizabeth K. Joseph proposed a change to openstack-infra/elastic-recheck: Add link to elastic-recheck docs to index https://review.openstack.org/98477 | 17:39 |
| asselin | jesusaurus, yes I'm behind an http proxy and want zull to be able to get the openstack gerrit stream | 17:39 |
| clarkb | asselin: you are at hp right? | 17:39 |
| asselin | clarkb, yes | 17:39 |
| clarkb | I think tsocks would work | 17:39 |
| clarkb | er wait | 17:39 |
| clarkb | that depends on which tcp connections need proxying ... | 17:40 |
| anteaya | reed: you feel diminished everytime you hear that I am willing to help people learn how to classify bugs? | 17:40 |
| asselin | clarkb, here's my error: http://paste.openstack.org/show/83154/ | 17:40 |
| clarkb | asselin: fwiw review.openstack.org:29418 should be open from inside hp | 17:40 |
| *** starmer_ has quit IRC | 17:40 | |
| fungi | the logstash job queue is heading into a backlog again, but the cluster looks healthy | 17:40 |
| clarkb | but the recent move broke that I think | 17:40 |
| clarkb | fungi: cool. I think we just throw too much data at the front end of it | 17:40 |
| *** lcheng has quit IRC | 17:40 | |
| jesusaurus | clarkb: that ip change issue should be resolved now | 17:41 |
| clarkb | jesusaurus: oh cool | 17:41 |
| *** Shrews has quit IRC | 17:41 | |
| reed | anteaya, not at all :) | 17:41 |
| anteaya | reed oh okay good | 17:41 |
| *** Shrews has joined #openstack-infra | 17:41 | |
| asselin | clarkb, I can use plain old ssh as the zuul user to get the event stream. Just not the zuul process itself... | 17:41 |
| clarkb | it was my patches welcome statement | 17:41 |
| anteaya | reed let me know if anyone wants to learn | 17:41 |
| *** ominakov has joined #openstack-infra | 17:41 | |
| anteaya | clarkb: ah, hmmmm | 17:41 |
| clarkb | asselin: from the same host as the same user with the same keys? | 17:41 |
| reed | am I correct in assuming that in patchset 7 the developer didn't run the local tests before sending it up for review? https://review.openstack.org/#/c/84244/8 | 17:42 |
| asselin | clarkb, yes, as far as I can tell..... | 17:42 |
| *** marcoemorais has quit IRC | 17:42 | |
| reed | anteaya, will do, thank you for helping | 17:42 |
| anteaya | reed absolutely | 17:42 |
| *** marcoemorais has joined #openstack-infra | 17:42 | |
| fungi | reed: looks that way... "AttributeError: Mock object has no attribute 'hds_hnas_iscsi_config_file'" | 17:43 |
| anteaya | the more devs know how to grok status.o.o/zuul and know where to turn when they want to learn how to help, the better | 17:43 |
| reed | fungi, ok, that's what I thought | 17:43 |
| openstackgerrit | A change was merged to openstack-infra/storyboard: Name fields checked with regex https://review.openstack.org/95763 | 17:43 |
| asselin | clarkb, actually I found a possible difference: no port in my zuul.conf file.... | 17:44 |
| mestery | fungi: Can you temporarily remove voting rights for the OpenDaylight Jenkins? They just informed me there is an issue they are working to fix, but would like voting removed to stop from voting -1 for now. | 17:44 |
| clarkb | asselin: ah ya that may do it | 17:44 |
| fungi | mestery: sure, doing that now | 17:44 |
| *** viktors is now known as viktors|afk | 17:44 | |
| *** trinaths has joined #openstack-infra | 17:44 | |
| mestery | fungi: Thank you so much! They hope to sort it out by early next week, will let you know. | 17:45 |
| fungi | mestery: i have moved them into the non-voting group for now | 17:45 |
| mestery | fungi: Great, much appreciated! | 17:45 |
| anteaya | mestery: thanks for letting us know | 17:45 |
| mestery | anteaya: Thanks to dave_tucker from Red Hat for pinging me. Team work is #ftw | 17:46 |
| * krotscheck facepalms. | 17:46 | |
| anteaya | agreed | 17:47 |
| anteaya | thanks dave_tucker | 17:47 |
| krotscheck | I don’t suppose I can ask someone to look at the logs on storyboard.o.o to see why that most recent change is breaking things? | 17:47 |
| fungi | krotscheck: sure, just a sec | 17:47 |
| *** ominakov has quit IRC | 17:47 | |
| krotscheck | fungi: Thanks | 17:48 |
| *** palar has joined #openstack-infra | 17:48 | |
| clarkb | the one I approved? | 17:48 |
| clarkb | clearly I failed at reviewing code | 17:48 |
| krotscheck | clarkb: Apparently. | 17:48 |
| krotscheck | clarkb: I approved it too. | 17:48 |
| fungi | krotscheck: what log files in particular? | 17:48 |
| clarkb | I even tested the regex | 17:48 |
| clarkb | though maybe not well enough | 17:48 |
| fungi | looks like we have an empty /var/log/storyboard | 17:49 |
| krotscheck | fungi: Urm… wsgi logs, which I think live in apache? | 17:49 |
| fungi | krotscheck: okie dokie | 17:49 |
| *** otherwiseguy has quit IRC | 17:49 | |
| krotscheck | clarkb: I have a reasonably-up-to-date local mirror of the DB and it’s not failing with me. | 17:50 |
| *** starmer_ has joined #openstack-infra | 17:50 | |
| *** basha has quit IRC | 17:51 | |
| fungi | nothing recent in the apache error.log or storyboard-error.log... though puppet has trouble applying the manifest: | 17:51 |
| fungi | Jun 6 17:45:24 storyboard puppet-agent[21806]: (/Stage[main]/Openstack_project::Base/Package[virtualenv]/ensure) change from absent to present failed: Could not set 'present on ensure: Could not locate the pip command. at /opt/config/production/modules/openstack_project/manifests/base.pp:58 | 17:51 |
| krotscheck | In base? WAT | 17:52 |
| openstackgerrit | Elizabeth K. Joseph proposed a change to openstack-infra/config: Add basic elastic-recheck docs page https://review.openstack.org/98481 | 17:52 |
| *** _nadya_ has joined #openstack-infra | 17:52 | |
| fungi | yeah | 17:53 |
| clarkb | pleia2: is the doc link change tested? | 17:53 |
| clarkb | pleia2: because I am ready to approve if you ran it locally | 17:53 |
| fungi | krotscheck: looks like something has failed to get pip onto the server | 17:53 |
| clarkb | fungi: does which pip show you a pip? | 17:53 |
| fungi | digging deeper | 17:53 |
| krotscheck | fungi: I swear I haven’t touched it. Yet. | 17:53 |
| fungi | clarkb: yeah, /usr/local/bin/pip | 17:53 |
| clarkb | I haven oticed that when pip moves from /usr/bin/pip to /usr/local/bin/pip puppet doesn't notice until the next run | 17:53 |
| clarkb | because it caches the which output | 17:53 |
| asselin | clarkb, fyi the correct port is defaulted in zuul's GerritWatcher | 17:54 |
| fungi | seems that it's been doing this over and over on every run though | 17:54 |
| pleia2 | clarkb: I loaded it up locally as the index page (without graph data), it's in the header so it should be fine | 17:54 |
| clarkb | pleia2: great thanks | 17:54 |
| clarkb | fungi: oh different problem then | 17:54 |
| fungi | looking now to see how long ago it started | 17:54 |
| *** starmer_ has quit IRC | 17:55 | |
| fungi | it's been happening on every run since at least the end of may where the syslog rotation expires | 17:56 |
| fungi | so no clue how long really | 17:56 |
| fungi | but a while | 17:56 |
| *** marcoemorais has quit IRC | 17:56 | |
| *** marcoemorais has joined #openstack-infra | 17:57 | |
| *** pelix has quit IRC | 17:57 | |
| clarkb | krotscheck: so probably just broken and not due to that change | 17:57 |
| clarkb | krotscheck: if that makes you feel better | 17:57 |
| openstackgerrit | A change was merged to openstack-infra/elastic-recheck: Add link to elastic-recheck docs to index https://review.openstack.org/98477 | 17:57 |
| krotscheck | clarkb: Yeah, sounds like a red herring to me. | 17:57 |
| *** marcoemorais has quit IRC | 17:57 | |
| krotscheck | clarkb fungi: Though we did just have some normal patches land without an issue. | 17:58 |
| fungi | probably so, though needs fixing | 17:58 |
| *** marcoemorais has joined #openstack-infra | 17:58 | |
| *** signed8bit has quit IRC | 17:59 | |
| *** signed8bit has joined #openstack-infra | 17:59 | |
| krotscheck | fungi: Digging through base.pp - seems like it’s depending on pip/init.pp, which… only seems like it’s installing python-devel, but not python-pip. | 18:00 |
| *** e0ne has joined #openstack-infra | 18:00 | |
| *** godp13011 has joined #openstack-infra | 18:01 | |
| fungi | krotscheck: yeah, it's supposed to use the get-pip.py script to install pip without using the extremely ancient distro package which operates in very insecure ways | 18:01 |
| krotscheck | fungi: Righto | 18:02 |
| *** marcoemorais has quit IRC | 18:02 | |
| *** marcoemorais has joined #openstack-infra | 18:02 | |
| *** godp1301 has quit IRC | 18:03 | |
| * krotscheck read the intro to get-pip… | 18:03 | |
| *** marcoemorais has quit IRC | 18:03 | |
| *** marcoemorais has joined #openstack-infra | 18:03 | |
| fungi | basically ubuntu 12.04 carries pip 1.0 which doesn't use https, crawls external project websites, installs pre-release versions by default, et cetera | 18:03 |
| fungi | pip releases prior to 1.4 are basically unsafe | 18:04 |
| *** zz_gondoi has quit IRC | 18:04 | |
| krotscheck | fungi: Makes sense. | 18:04 |
| clarkb | krotscheck: careful it is a base64 encoded zip file with self extracting python footer | 18:04 |
| reed | do I remember correctly that abandoned patches can be 'resuscitated' on gerrit? | 18:04 |
| reed | how? | 18:04 |
| clarkb | reed: you do, the patch owner can click the restore button or a gerrit admin can do it | 18:04 |
| *** praneshp has joined #openstack-infra | 18:04 | |
| krotscheck | fungi: But OMG ascii-zip | 18:04 |
| fungi | reed: by either the change owner or by a core reviewer for that project | 18:04 |
| reed | that's what I remembered, thanks | 18:04 |
| fungi | clarkb: project cores now too. we added that acl provided by new gerrit | 18:05 |
| clarkb | woot | 18:05 |
| *** godp1301 has joined #openstack-infra | 18:05 | |
| *** changbl has joined #openstack-infra | 18:05 | |
| *** godp13011 has quit IRC | 18:05 | |
| krotscheck | And, you know, a module called pip that doesn’t actually do anything with pip. | 18:05 |
| *** starmer_ has joined #openstack-infra | 18:05 | |
| fabo | zaro: only one left to review ;) | 18:06 |
| fungi | krotscheck: yes, it's sort of fright-inducing. reminds me of the old edlin-based code you could use to uudecode a uuencoded copy of the uudecode utility | 18:06 |
| *** gondoi has joined #openstack-infra | 18:07 | |
| * krotscheck ponders if it’s wsgi that’s broken. He’s not getting any love on any API endpoints | 18:08 | |
| fungi | krotscheck: yeah, i'm still digging for some evidence of something relevant | 18:08 |
| *** _nadya_ has quit IRC | 18:08 | |
| openstackgerrit | Matt Riedemann proposed a change to openstack-infra/elastic-recheck: Add query for 1244457 https://review.openstack.org/98297 | 18:08 |
| mtreinish | clarkb, fungi: https://review.openstack.org/#/c/98406/ might be useful for debugging grenade fails | 18:08 |
| jesusaurus | where can i find the gerrit queries used to create the dashboards? | 18:09 |
| clarkb | jesusaurus: in the refs/meta/dashboards/* refs in All-Projects | 18:09 |
| clarkb | jesusaurus: you should have read access to those | 18:09 |
| clarkb | and if not we should fix the ACLs | 18:09 |
| fungi | jesusaurus: though sdague also has a utility you can use to build them from dashboard config syntax | 18:10 |
| *** starmer_ has quit IRC | 18:10 | |
| fungi | jesusaurus: http://git.openstack.org/cgit/stackforge/gerrit-dash-creator/tree/README.rst | 18:10 |
| *** adrian_otto has joined #openstack-infra | 18:11 | |
| zaro | fabo: we welcome reviews as well as contributions :) | 18:11 |
| mestery | With regards to the "ssh timeout" issue: armax just pushed this: https://review.openstack.org/#/c/98483/ | 18:11 |
| mestery | If this triggers the failure, can we get access to debug things further? | 18:11 |
| mestery | fungi clarkb: ^^^ | 18:12 |
| devananda | clarkb: re: one db per test, is that for testing migrations or something else? | 18:12 |
| clarkb | mtreinish: reviewing | 18:12 |
| openstackgerrit | A change was merged to openstack-infra/elastic-recheck: Add query for 1244457 https://review.openstack.org/98297 | 18:12 |
| clarkb | devananda: anything you want. Now that we are fully single use slaves for unittests we give the test db user access to create and drop arbitrary schemas | 18:12 |
| clarkb | devananda: on mysql I think the access is ALL | 18:13 |
| clarkb | postgres has finer grained ACLs so its restricted to creating, owning and deleting schemas | 18:13 |
| fungi | krotscheck: found a traceback | 18:13 |
| fungi | krotscheck: lemme pretty it up real quick | 18:13 |
| krotscheck | fungi: WOO | 18:13 |
| clarkb | mtreinish: lgtm | 18:17 |
| fungi | krotscheck: http://paste.openstack.org/show/83156 | 18:17 |
| mtreinish | clarkb: cool, thanks | 18:17 |
| clarkb | mtreinish: but http://logs.openstack.org/06/98406/1/check/check-grenade-dsvm/9932708/logs/old/ | 18:18 |
| krotscheck | fungi: Thanks | 18:18 |
| *** andreykurilin_ has joined #openstack-infra | 18:18 | |
| clarkb | has tempest files in it so we can probably just have fungi approve it if he is happy with review | 18:18 |
| clarkb | fungi: ^ | 18:18 |
| *** devkulkarni has joined #openstack-infra | 18:19 | |
| jesusaurus | clarkb: perfect thanks for the ref/ | 18:19 |
| fungi | mtreinish: clarkb: seems sort of duplicative, but okay i guess | 18:20 |
| clarkb | fungi: ya but meh | 18:20 |
| mtreinish | fungi: I tried to do it smarter, but this was easier :) | 18:20 |
| fungi | mtreinish: wfm then | 18:20 |
| *** basha has joined #openstack-infra | 18:20 | |
| jesusaurus | fungi: thats pretty cool, but i dont think theres a way for me to have a personal dash so im going to just bookmark some query results (i want to remove +2'd changes from my important changes because im not core) | 18:20 |
| *** starmer_ has joined #openstack-infra | 18:20 | |
| devkulkarni | hey there.. quick question. recently there was email by Sean about race condition issues on the gates and patches should not be approved until those get resolved. does anyone have status about when we can start approving patches again? | 18:20 |
| devananda | devkulkarni: you can see the queue here: http://status.openstack.org/zuul/ | 18:21 |
| clarkb | devkulkarni: and I expect sdague to send mail to the list with an all clear when things are happy again | 18:21 |
| devkulkarni | okay. thanks devananda and clarkb | 18:21 |
| clarkb | devkulkarni: you can also look at http://status.openstack.org/elastic-recheck/ for a state of things as related to bugs | 18:22 |
| *** Ryan_Lane has quit IRC | 18:22 | |
| fungi | mestery: yeah, we can catch a worker while it's running the job we expect to break, and then place it on hold so it won't be deleted when the job ends. after that we can add developer ssh keys as needed to get you access to debug it while it's still up and in a dirty state | 18:23 |
| mestery | fungi: That would be perfect, exactly what we need! | 18:24 |
| *** _nadya_ has joined #openstack-infra | 18:24 | |
| fungi | looking to see if any of the current run from 98483 are still up | 18:24 |
| mestery | fungi: cool | 18:24 |
| fungi | mestery: expecting it to probably break on check-tempest-dsvm-neutron ? | 18:25 |
| clarkb | I was just going to ask that | 18:25 |
| *** starmer_ has quit IRC | 18:25 | |
| clarkb | does this affect nova net too? | 18:25 |
| clarkb | fungi: 150490 and 150606 seem to be the nodes you want | 18:26 |
| fungi | yup | 18:26 |
| *** _nadya_ has quit IRC | 18:26 | |
| fungi | held | 18:26 |
| openstackgerrit | Michael Krotscheck proposed a change to openstack-infra/storyboard: Updated WSME version https://review.openstack.org/98487 | 18:26 |
| *** ramashri has quit IRC | 18:26 | |
| krotscheck | fungi clarkb: I _think_ this’ll do it ^ | 18:27 |
| krotscheck | I’m not that familiar with what pip install will do though when something got updated in requirements.txt | 18:27 |
| *** _nadya_ has joined #openstack-infra | 18:27 | |
| fungi | krotscheck: it should update it | 18:28 |
| mestery | fungi: I think that's the one, yes. | 18:28 |
| fungi | mestery: i held that one and the one running the non-voting neutron-full job too for good measure. i'm keeping a tab on the job progress for those and will give you a heads up once we have logs so you can determine if it tripped it | 18:29 |
| *** changbl has quit IRC | 18:29 | |
| mestery | fungi: Cool, thank you! | 18:29 |
| jerryz | clarkb: about yesterday's discussion, can i still use images.username and images.private_key in nodepool.yaml for jump host connected slave config? | 18:32 |
| *** ramashri has joined #openstack-infra | 18:32 | |
| clarkb | jerryz: I think you should. Both the jump host and destination should accept the same user and key | 18:33 |
| clarkb | jerryz: it is a lot simpler that way | 18:33 |
| *** trinaths has quit IRC | 18:34 | |
| jerryz | clarkb: yeah. if that is the case, i will just need a proxy host ip in provider section. | 18:35 |
| *** starmer_ has joined #openstack-infra | 18:35 | |
| *** otherwiseguy has joined #openstack-infra | 18:36 | |
| clarkb | fungi: wow logstash really doesn't like this | 18:39 |
| openstackgerrit | Michael Krotscheck proposed a change to openstack-infra/storyboard: Update and freeze requirements https://review.openstack.org/98491 | 18:39 |
| clarkb | fungi: oh thats the other thing. It will index cancelled jobs if they don't properly mark themselves as cancelled/aborted | 18:40 |
| *** changbl has joined #openstack-infra | 18:40 | |
| clarkb | which I think contributes to the problem when the gate is really flaky | 18:40 |
| *** starmer_ has quit IRC | 18:40 | |
| fungi | yeah... than you jenkins | 18:40 |
| *** crc32 has joined #openstack-infra | 18:41 | |
| *** chuckC has quit IRC | 18:43 | |
| clarkb | ok lunchish time. I am going to try and dive back into dib after lunch if there aren't more pressing gate things to review or debug (like those nodes we just held) | 18:43 |
| *** chuckC has joined #openstack-infra | 18:44 | |
| openstackgerrit | A change was merged to openstack-infra/storyboard: Updated WSME version https://review.openstack.org/98487 | 18:45 |
| openstackgerrit | Michael Krotscheck proposed a change to openstack-infra/storyboard: Update and freeze requirements https://review.openstack.org/98491 | 18:45 |
| harlowja | i have a new logo for u guys, for the CI group | 18:48 |
| harlowja | ci == http://d1xcqlxj49e9dd.cloudfront.net/wp-content/uploads/LETC-new001-1024x902.jpg?4a2207 :-P | 18:48 |
| harlowja | *infra group i mean | 18:48 |
| * krotscheck patiently watis for puppet to pick up the changes... | 18:48 | |
| clarkb | harlowja: :) | 18:49 |
| clarkb | reed: pleia2 docs link on the e-r status page is live | 18:49 |
| krotscheck | harlowja: I was thinking more this: http://www.trueventurestec.com/wp-content/uploads/2010/07/Rainbow-Unicorn.png | 18:49 |
| *** aconrad_ has quit IRC | 18:49 | |
| harlowja | krotscheck maybe a unicorn riding a blue locomotive | 18:50 |
| harlowja | that'd be a good combination | 18:50 |
| reed | thanks pleia2 | 18:50 |
| krotscheck | harlowja: TO THE GOOGLES! | 18:50 |
| harlowja | ha | 18:50 |
| *** starmer_ has joined #openstack-infra | 18:50 | |
| reed | did you guys burned the gates already? I have gasoline left over here | 18:50 |
| reed | god, my English is getting worse | 18:51 |
| harlowja | thats what happens when u play with gasoline | 18:51 |
| reed | harlowja, AH! It's not written on the box, I'm filing a lawsuit :) | 18:51 |
| harlowja | u must be in the US :-P | 18:52 |
| fungi | harlowja: nice! | 18:53 |
| harlowja | :) | 18:53 |
| harlowja | i'm pretty sure that book was made in ~1930 so u can probably use the logo, haha | 18:54 |
| *** Longgeek has quit IRC | 18:54 | |
| harlowja | although who knows anymore (copyrights might last thousands years with the current system, haha) | 18:54 |
| fungi | unless it was made by disney. they own all the copyright lawyers | 18:54 |
| harlowja | but krotscheck finds a unicorn version then that will be even better, ha | 18:55 |
| *** amcrn has joined #openstack-infra | 18:55 | |
| *** starmer_ has quit IRC | 18:55 | |
| *** markmcclain has joined #openstack-infra | 18:56 | |
| dhellmann | is gerritbot down? I expected a notification in #openstack-oslo when something merged in oslo-specs, but it didn't come through. | 18:58 |
| *** _nadya_ has joined #openstack-infra | 18:58 | |
| fungi | dhellmann: it's been bouncing like a rubber ball all week | 18:58 |
| dhellmann | fungi: ok, I won't worry about it if you're already aware | 18:58 |
| fungi | i'll give it another beating when it times out | 18:58 |
| fungi | though /ping is getting a response | 18:59 |
| *** markmcclain1 has joined #openstack-infra | 19:00 | |
| krotscheck | Ok, so puppet ran, the vcsrepo updated, but pip install didn’t get called. | 19:01 |
| *** apevec has quit IRC | 19:01 | |
| *** aconrad_ has joined #openstack-infra | 19:01 | |
| *** markmcclain has quit IRC | 19:02 | |
| dstufft | Hey quick question, it looks like the barbican-specs repo didn't seed correctly, is there a reason for this? | 19:04 |
| *** nati_ueno has quit IRC | 19:04 | |
| fungi | krotscheck: ... | 19:04 |
| fungi | fungi@storyboard:/opt/storyboard$ pip freeze|grep WSME | 19:04 |
| fungi | WSME==0.6.1 | 19:04 |
| fungi | dstufft: looking | 19:04 |
| dstufft | fungi: thanks! | 19:05 |
| *** praneshp_ has joined #openstack-infra | 19:05 | |
| fungi | dstufft: http://git.openstack.org/cgit/openstack/barbican-specs/ is it missing commits? | 19:05 |
| *** arnaud__ has quit IRC | 19:05 | |
| krotscheck | fungi: Now I’m confused. | 19:05 |
| *** starmer_ has joined #openstack-infra | 19:05 | |
| fungi | dstufft: oh, i see it had trouble creating the github repo again | 19:05 |
| *** basha has quit IRC | 19:06 | |
| fungi | dstufft: fixing | 19:06 |
| dstufft | fungi: thanks sir | 19:06 |
| fungi | github's api is beyong unreliable | 19:06 |
| fungi | beyond | 19:06 |
| *** praneshp has quit IRC | 19:06 | |
| *** praneshp_ is now known as praneshp | 19:06 | |
| krotscheck | fungi: Why would VCSRepo update, but the refreshonly exec subscribed to it not trigger? http://puppetboard.openstack.org/report/storyboard.openstack.org/f2d5f221c3b9ab9bd93e75c736194d2ad61be0d8 | 19:06 |
| *** andreaf has quit IRC | 19:07 | |
| clarkb | devananda: would it be useful for me to attend ironic meetup? | 19:08 |
| clarkb | I should be in the beaverton area those days | 19:08 |
| clarkb | and wondering if yall think infra core can help | 19:08 |
| devananda | clarkb: possibly, given that there will necessarily be discussions about how do we actually merge things | 19:08 |
| *** dprince has quit IRC | 19:08 | |
| devananda | clarkb: sdague and jeblair were helpful at the summit in formulating a plan. I would like to actually implement most of it by the sprint, then "flip the switch" while we're there | 19:09 |
| Shrews | clarkb: only if you save some of your homebrew for us | 19:09 |
| devananda | ...pending nova core actually reviewing the spec and code ahead of time | 19:10 |
| fungi | dstufft: https://github.com/openstack/barbican-specs | 19:10 |
| *** starmer_ has quit IRC | 19:10 | |
| fungi | krotscheck: great question... | 19:10 |
| clarkb | devananda rgr so sounds like a good idea | 19:10 |
| clarkb | where in beaverton? | 19:11 |
| clarkb | shrews I might be able to arrange that | 19:11 |
| devananda | clarkb: intel campus. which exact campus TBD | 19:11 |
| krotscheck | AFK. Need fud. | 19:11 |
| fungi | krotscheck: i think it did refresh... pip freeze claims storyboard==519a134 which matches your latest commit | 19:11 |
| clarkb | devananda ok thats easy then | 19:12 |
| dstufft | fungi: thanks | 19:12 |
| *** aburaschi has quit IRC | 19:12 | |
| fungi | mestery: http://logs.openstack.org/83/98483/1/check/check-tempest-dsvm-neutron/e7f8591/ succeeded but http://logs.openstack.org/83/98483/1/check/check-tempest-dsvm-neutron-full/e44a86c/ failed... did it tickle the bug you were trying for? | 19:14 |
| * mestery goes and looks. | 19:14 | |
| clarkb | also hawthorne farms is the one you want >_> | 19:15 |
| clarkb | there are pre security accomodations iirc which is nice for something like this | 19:15 |
| mestery | fungi: I think that's the one, yes! | 19:15 |
| mestery | fungi: What do you need from me now to get access to it? | 19:15 |
| clarkb | and its by MAX | 19:16 |
| *** starmer_ has joined #openstack-infra | 19:20 | |
| fungi | mestery: the ssh public key of someone who wants to log into it | 19:22 |
| *** aburaschi has joined #openstack-infra | 19:22 | |
| fungi | mestery: doesn't look like you have one on your lp profile... but was it armax who wanted to look it over anyway? | 19:22 |
| *** aburaschi has quit IRC | 19:23 | |
| mestery | fungi: Yes, let me ping him, he's somewhere in Europe ATM, if he can't, I'll add one to my LP page. One minute. | 19:23 |
| fungi | if so, just find out if he wants me to grant it to the one he has on his lp profile... https://launchpad.net/%7Earmando-migliaccio/+sshkeys | 19:24 |
| *** Longgeek has joined #openstack-infra | 19:24 | |
| *** starmer_ has quit IRC | 19:25 | |
| *** hashar has joined #openstack-infra | 19:26 | |
| *** nati_ueno has joined #openstack-infra | 19:26 | |
| mestery | fungi: armax isn't online now, I just added a key to my LP profile, can you grant access to that one? | 19:27 |
| fungi | sure. i'll stick his in there too in case he shows up | 19:28 |
| mestery | fungi: perfect, and thanks! | 19:28 |
| fungi | mestery: ssh jenkins@23.253.76.65 | 19:28 |
| mestery | fungi: thanks! | 19:28 |
| fungi | mestery: that should be the one which ran http://logs.openstack.org/83/98483/1/check/check-tempest-dsvm-neutron-full/e44a86c | 19:29 |
| mestery | fungi: perfect! | 19:29 |
| fungi | the ip address show in the console log appears to confirm that too | 19:29 |
| *** Longgeek has quit IRC | 19:29 | |
| fungi | mestery: let me know when you're done tearing it apart and i'll recycle it back into the aether | 19:30 |
| mestery | fungi: will do | 19:30 |
| *** starmer_ has joined #openstack-infra | 19:36 | |
| *** ramashri has quit IRC | 19:37 | |
| openstackgerrit | Matt Riedemann proposed a change to openstack-infra/elastic-recheck: Add query for nova test_enforce_http_true unit test bug 1288178 https://review.openstack.org/98502 | 19:38 |
| uvirtbot | Launchpad bug 1288178 in tuskar "Sync new policy from oslo" [Medium,In progress] https://launchpad.net/bugs/1288178 | 19:38 |
| *** nati_ueno has quit IRC | 19:38 | |
| clarkb | woo debugging | 19:39 |
| *** starmer_ has quit IRC | 19:40 | |
| fungi | clarkb: oh, also https://review.openstack.org/98422 is probably a somewhat high priority to review since i have puppet disabled and that applied on pypi.slave.o.o right now | 19:40 |
| fungi | but it did seem to solve all the remaining wheel upload issues aside from the one where wheel doesn't grok data-only packages | 19:40 |
| fungi | which ideally should be solved upstream in wheel anyway | 19:41 |
| *** palar has quit IRC | 19:41 | |
| clarkb | cool reviewing | 19:42 |
| clarkb | I just kicked off another dib run | 19:42 |
| fungi | excellent | 19:42 |
| *** markwash has quit IRC | 19:43 | |
| clarkb | I raelly have to say the caching is amazing | 19:43 |
| clarkb | the first run takes forever but subsequent runs are not terrible | 19:43 |
| *** changbl has quit IRC | 19:43 | |
| openstackgerrit | A change was merged to openstack-infra/elastic-recheck: Add query for nova test_enforce_http_true unit test bug 1288178 https://review.openstack.org/98502 | 19:44 |
| uvirtbot | Launchpad bug 1288178 in tuskar "Sync new policy from oslo" [Medium,In progress] https://launchpad.net/bugs/1288178 | 19:44 |
| clarkb | fungi: 'safer_name' is the best function name ever | 19:44 |
| fungi | it's more better than safe_name() | 19:44 |
| fungi | obviously | 19:44 |
| fungi | but i'll still hold out hope for a safest_name() | 19:45 |
| clarkb | approved | 19:45 |
| anteaya | dhellmann: I replied to your email and my reply is waiting for moderator approval | 19:45 |
| fungi | danke. once that's on the master i'll release the houds^Wpuppet agent | 19:46 |
| *** markwash has joined #openstack-infra | 19:47 | |
| *** dprince has joined #openstack-infra | 19:47 | |
| *** praneshp has quit IRC | 19:48 | |
| openstackgerrit | A change was merged to openstack-infra/config: Determine wheel names for PyPI uploading https://review.openstack.org/98422 | 19:49 |
| fungi | krotscheck: i think i found our automation issue. apache2 service needs kicking (reload is fine, don't have to restart) when things change under wsgi like that since it has to reimport things | 19:50 |
| fungi | and stuff | 19:50 |
| *** starmer_ has joined #openstack-infra | 19:50 | |
| mestery | fungi: I don't think this tripped the same bug, as armax's patch should have left the guest VM around, unless I read his patch wrong: https://review.openstack.org/#/c/98483/1 | 19:51 |
| *** jpeeler has quit IRC | 19:51 | |
| fungi | mestery: okay, i can recycle this and recheck the bug. maybe we'll get lucky on a retry? | 19:51 |
| mestery | fungi: Glass half full, I love it! :) | 19:52 |
| mestery | fungi: go for it! | 19:52 |
| *** maxbit has joined #openstack-infra | 19:52 | |
| fungi | mestery: done. once those jobs get new workers i'll put them on hold and hope one breaks the way we want this time | 19:53 |
| bknudson | it looks like there's a new release of oauthlib that breaks keystoneclient tests. | 19:54 |
| fungi | bknudson: that explains the unit test failures blocking 91216,2 | 19:54 |
| *** starmer_ has quit IRC | 19:55 | |
| *** maxbit_ has quit IRC | 19:55 | |
| *** arnaud has joined #openstack-infra | 19:55 | |
| clarkb | bknudson: if we can sort out whether or not it is just a bug or intentional backward compat breakage we can update requirements | 19:59 |
| clarkb | (or decide not to and possibly support both sides of the compat wall) | 19:59 |
| *** marcoemorais has quit IRC | 19:59 | |
| *** marcoemorais has joined #openstack-infra | 20:00 | |
| *** _nadya_ has quit IRC | 20:00 | |
| *** _nadya_ has joined #openstack-infra | 20:00 | |
| *** zns has quit IRC | 20:01 | |
| *** andreykurilin_ has quit IRC | 20:01 | |
| *** grantbow has quit IRC | 20:01 | |
| *** bcrochet has quit IRC | 20:01 | |
| *** juice has quit IRC | 20:01 | |
| *** adarazs has quit IRC | 20:01 | |
| *** juice has joined #openstack-infra | 20:01 | |
| *** bcrochet has joined #openstack-infra | 20:02 | |
| *** adarazs has joined #openstack-infra | 20:02 | |
| mestery | fungi: thanks! | 20:02 |
| *** jpeeler has joined #openstack-infra | 20:02 | |
| *** jpeeler has joined #openstack-infra | 20:02 | |
| *** _nadya_ has quit IRC | 20:02 | |
| *** gargola has quit IRC | 20:03 | |
| *** grantbow has joined #openstack-infra | 20:03 | |
| *** gargola has joined #openstack-infra | 20:04 | |
| *** starmer_ has joined #openstack-infra | 20:06 | |
| krotscheck | fungi: Got it. Let me put up a patch. | 20:07 |
| *** adalbas has quit IRC | 20:09 | |
| fungi | okay, puppet has been reenabled on pypi.slave now | 20:09 |
| *** starmer_ has quit IRC | 20:10 | |
| *** e0ne has quit IRC | 20:13 | |
| *** zns has joined #openstack-infra | 20:19 | |
| *** starmer_ has joined #openstack-infra | 20:20 | |
| *** nati_ueno has joined #openstack-infra | 20:22 | |
| *** adalbas has joined #openstack-infra | 20:23 | |
| *** starmer_ has quit IRC | 20:25 | |
| *** dosaboy_ has joined #openstack-infra | 20:25 | |
| *** plars has quit IRC | 20:25 | |
| *** plars has joined #openstack-infra | 20:25 | |
| *** dosaboy has quit IRC | 20:26 | |
| *** _nadya_ has joined #openstack-infra | 20:27 | |
| *** adalbas has quit IRC | 20:28 | |
| *** devkulkarni1 has joined #openstack-infra | 20:32 | |
| *** devkulkarni1 has quit IRC | 20:33 | |
| *** devkulkarni1 has joined #openstack-infra | 20:33 | |
| *** devkulkarni has quit IRC | 20:34 | |
| devananda | clarkb: any chance to bump 97757 up the queue? 21hrs since it last ran (and failed to merge due to a neutron bug) | 20:34 |
| devananda | clarkb: in the meantime, ironic patches cant even pass check | 20:35 |
| *** starmer_ has joined #openstack-infra | 20:35 | |
| clarkb | devananda: looking at the gate it looks pretty broken still. I don't expect promoting it again will help | 20:36 |
| devananda | clarkb: anything specific i can do to help? | 20:36 |
| *** mbacchi has quit IRC | 20:36 | |
| *** _nadya_ has quit IRC | 20:37 | |
| clarkb | devananda: classifying bugs is probably the easiest place to start | 20:37 |
| *** dkliban is now known as dkliban_afk | 20:37 | |
| *** praneshp has joined #openstack-infra | 20:38 | |
| * anteaya is willing to help people learn how to classify bugs | 20:38 | |
| *** maxbit_ has joined #openstack-infra | 20:38 | |
| *** starmer_ has quit IRC | 20:40 | |
| *** maxbit has quit IRC | 20:40 | |
| *** rcarrill` has joined #openstack-infra | 20:40 | |
| *** chuckC has quit IRC | 20:42 | |
| openstackgerrit | Michael Krotscheck proposed a change to openstack-infra/config: StoryBoard module bounces apache on update https://review.openstack.org/98512 | 20:42 |
| *** rcarrillocruz has quit IRC | 20:43 | |
| anteaya | I'm so out of steam, it is such a beautiful day | 20:44 |
| mestery | anteaya: +1000 to out of steam, though it's cloudy here. :) | 20:45 |
| fungi | krotscheck: technically just a service reload should work (and should avoid the downtime blip). though doing that via puppet is a little mor of a pain | 20:45 |
| krotscheck | fungi: I just added a notify for Service[‘httpd | 20:47 |
| krotscheck | Let me see if that’s defined with hasreload= true... | 20:47 |
| fungi | krotscheck: oh, good point... the version of the apache module we're using is oldish, but still might do reload on notify instead of restart | 20:48 |
| krotscheck | fungi: Well, hasreload isn’t set in apache’s declaration of Service[‘httpd’]. | 20:48 |
| krotscheck | fungi: And as I recall that’s default set to false. | 20:48 |
| fungi | yeah. as i said, more of a pain to implement in that case | 20:49 |
| fungi | you end up needing a separate exec and yadda yadda | 20:49 |
| fungi | so probably not worth the extra effort | 20:49 |
| *** james_li has quit IRC | 20:49 | |
| *** eharney has quit IRC | 20:50 | |
| *** starmer_ has joined #openstack-infra | 20:50 | |
| devananda | so not related to the gate issues we're having now -- but we've just realized that ironic hasn't been running db migration tests in our gate (we also haven't had any new db migrations in Juno yet) | 20:51 |
| krotscheck | fungi: Yeah, as much as the OCD in me really wants to go upgrade/fix all the puppet things, I have storyboard thigns to do :/ | 20:52 |
| fungi | totally | 20:52 |
| *** salv-orlando has joined #openstack-infra | 20:53 | |
| clarkb | devananda: do you know why it wasn't running the tests? | 20:53 |
| devananda | clarkb: i think we stopped at the same time we switched to oslo.db (just atheory, atm) | 20:53 |
| devananda | clarkb: looks like we need to have jenkins set tests/db/sqlalchemy/test_migrations.conf now | 20:54 |
| devananda | changing that locally gets the migrations to run | 20:54 |
| clarkb | devananda: no please no. The tests should just run | 20:55 |
| *** starmer_ has quit IRC | 20:55 | |
| *** adalbas has joined #openstack-infra | 20:55 | |
| clarkb | if the tests need to write a file thats fine (they shouldn't) but encoding that into jenkins makes it harder for people to reproduce at home | 20:55 |
| *** dprince has quit IRC | 20:55 | |
| clarkb | there is already enough overhead with the DB needing to be configured in a special way | 20:55 |
| devananda | clarkb: gotcha. so you're suggesting we encode the upstream DB connection params into that file? | 20:56 |
| clarkb | devananda: it would be better if the tests encoded it as appropriate | 20:56 |
| clarkb | that may mean writing it out as a fixture | 20:56 |
| *** krotscheck has quit IRC | 20:57 | |
| clarkb | but alembic is python you should be able to import alembic, run migration | 20:57 |
| clarkb | and do everything in memory | 20:57 |
| devananda | sure | 20:57 |
| *** salv-orlando has quit IRC | 20:57 | |
| devananda | i've forgotten how nova did its db migration tests -- are the cxn params hard-coded in the unit test file(s)? | 20:58 |
| *** marcoemorais has quit IRC | 20:58 | |
| *** godp1301 has quit IRC | 20:58 | |
| *** jgrimm has quit IRC | 20:58 | |
| clarkb | yes. it probes using the hardcoded values. if it can connect it runs the tests | 20:58 |
| * devananda looks | 20:58 | |
| devananda | heh | 20:58 |
| clarkb | but nodepool does it even better | 20:59 |
| devananda | so making it configurable is worse, you're saying :p | 20:59 |
| *** marcoemorais has joined #openstack-infra | 20:59 | |
| clarkb | no I think going the nodepool route is the way it should be done | 20:59 |
| devananda | docs/example for me to look at? | 20:59 |
| openstackgerrit | Nikhil Manchanda proposed a change to openstack-infra/config: Added new experimental job for trove functional tests https://review.openstack.org/98517 | 21:00 |
| clarkb | devananda: https://git.openstack.org/cgit/openstack-infra/nodepool/tree/nodepool/tests/__init__.py#n71 the username and password are hard coded | 21:00 |
| *** krotscheck has joined #openstack-infra | 21:00 | |
| clarkb | not sure why the db is set there. maybe sqlalchemy refuses to connect otherwise | 21:00 |
| clarkb | but then everything else is per test on the fly schemas so you can run tests in parallel | 21:01 |
| devananda | ack, thanks | 21:01 |
| fungi | mestery: no dice. both jobs succeeded this time around | 21:01 |
| mestery | fungi: OK. I'm likely to take off in an hour, though I'll be around all weekend off and on. If we run them again and armax or I can't get to a fail fast enough, is that a problem? | 21:02 |
| fungi | mestery: well, if it gets rerun, we'll want to hold prospective hosts before their jobs complete... otherwise nodepool removes them to make way for more nodes more or less immediately | 21:03 |
| *** basha has joined #openstack-infra | 21:03 | |
| *** nati_ueno has quit IRC | 21:03 | |
| sdague | clarkb: so logstash is sad panda again? | 21:03 |
| *** nati_ueno has joined #openstack-infra | 21:03 | |
| clarkb | sdague: yes | 21:03 |
| fungi | though it does seem to have turned the corner | 21:04 |
| clarkb | I think gate thrash has overloaded it | 21:04 |
| mestery | fungi: OK, got it. | 21:04 |
| openstackgerrit | A change was merged to openstack-infra/storyboard-webclient: Added simple logged-in dashboard https://review.openstack.org/92662 | 21:04 |
| bknudson | clarkb: fungi: looks like we can do version check and workaround for the keystoneclient / oauthlib issue. | 21:04 |
| fungi | mestery: i'll be around tonight and over the weekend too, so if i can get one to fail with ssh timeouts i'll give you a heads up and an ip address | 21:04 |
| sdague | clarkb: do you have a graph of processed events as well? | 21:05 |
| mestery | fungi: That would be awesome! I'll be checking IRC all weekend, ping with a PM if you would. Thanks! | 21:05 |
| sdague | I wonder if other cloud events take our processed rate down | 21:05 |
| *** ArxCruz has quit IRC | 21:05 | |
| fungi | mestery: can do | 21:05 |
| bodepd | I heard a rumor you guys do community stuff on Fridays | 21:05 |
| clarkb | sdague: its in the graph the scale is just much smaller so yo ucan't see it | 21:05 |
| bodepd | if that's true could I ask for a bump :) | 21:05 |
| clarkb | sdague: if you copy pasta the image link and remove the extra numbers you will see it | 21:05 |
| *** starmer_ has joined #openstack-infra | 21:06 | |
| mestery | fungi: So, here's something funky. In my own testing, with a fresh devstack, I can recreate the timeout. But subsequent runs on that same host never produce a failure. It's a data point I've collected today at least. | 21:06 |
| * mestery takes the small victory. | 21:06 | |
| sdague | clarkb: so what's our process rate per minute? | 21:06 |
| fungi | bodepd: we do community stuff all week (and weekend)... this week has just been worse than usual... not been getting much reviewing at all done with the brokenness everywhere | 21:06 |
| sdague | is it staying pretty constant? | 21:06 |
| fungi | mestery: interesting... no idea what that implies (primed the pump?) but maybe it'll help other factors make sense when we uncover them | 21:07 |
| bodepd | fungi: thanks. I'll ask if you guys are above water again next week | 21:07 |
| fungi | mestery: it would at least highlight why we see it more often, since we always test on fresh systems | 21:07 |
| *** hashar has quit IRC | 21:07 | |
| clarkb | sdague: http://goo.gl/SwqQFi | 21:08 |
| *** mriedem has quit IRC | 21:08 | |
| clarkb | sdague: you can see its pegged while we fall behind and now it isn't as we catch up | 21:08 |
| *** pdmars has quit IRC | 21:08 | |
| mestery | fungi: I think so too. | 21:08 |
| *** aysyd has quit IRC | 21:08 | |
| sdague | clarkb: gotcha | 21:09 |
| clarkb | also we have less than 48 workers for some reason | 21:09 |
| clarkb | I will figure that out now | 21:09 |
| mestery | fungi: So, I was able to just recreate it locally again, and with this patch and it's dependent dumping the console on error, there actually is no console data (https://review.openstack.org/#/c/95844/) | 21:09 |
| sdague | clarkb: cool | 21:09 |
| mestery | fungi: To me, that seems to impy the instance isn't there or never made it up. | 21:09 |
| fungi | clarkb: i've rechecked 98483 again but it's a ways from getting more workers and i need to go grab some chow. can you try to catch the -dsvm-neutron and -dsvm-neutron-full master job workers it gets and put them on hold in nodepool? i'll check back in on them when i get back | 21:10 |
| *** starmer_ has quit IRC | 21:10 | |
| fungi | mestery: seems likely | 21:10 |
| mestery | fungi: http://paste.openstack.org/show/83183/ <-- Can you eyeball and verify that for me? | 21:10 |
| clarkb | fungi: sure | 21:11 |
| fungi | mestery: to my untrained eye it looks like there's no console output, though i'm not well versed on nova api behaviors | 21:12 |
| mestery | fungi: +1 for me on both fronts. And this would obviously explain why we can't connect to it. :) | 21:12 |
| mestery | fungi: I'll dig into nova logs now and see what I find. | 21:13 |
| fungi | anyway, disappearing for a bit. wife is going to gnaw my arm off otherwise | 21:13 |
| clarkb | sdague: they oomed | 21:14 |
| *** smarcet has quit IRC | 21:14 | |
| clarkb | I think 4GB is probably not enough for 3 workers + an indexer :( | 21:14 |
| clarkb | the indexers can soak up memory because lol java | 21:14 |
| sdague | ah gotcha | 21:15 |
| sdague | so what about using those high memory HP cloud nodes? | 21:16 |
| clarkb | lolno | 21:16 |
| mestery | later fungi, thanks for the help today! | 21:16 |
| sdague | he | 21:16 |
| devananda | clarkb: fwiw - config file is coming from here: https://github.com/openstack/oslo-incubator/blob/master/openstack/common/db/sqlalchemy/test_migrations.py#L71 | 21:17 |
| lifeless | clarkb: btw how do we make sure new releases of sqlalchemy etc can still apply old migrations ? | 21:17 |
| *** mriedem has joined #openstack-infra | 21:17 | |
| lifeless | clarkb: with the cached dbs thing | 21:18 |
| clarkb | I don't think that is tested | 21:18 |
| *** devkulkarni1 has quit IRC | 21:19 | |
| *** starmer_ has joined #openstack-infra | 21:21 | |
| *** mmaglana has quit IRC | 21:21 | |
| devananda | wtf, unit test shells out to mysql in _reset_databases? | 21:21 |
| openstackgerrit | Nikhil Manchanda proposed a change to openstack-infra/config: Added new experimental job for trove functional tests https://review.openstack.org/98517 | 21:21 |
| devananda | huh, same for pgsql | 21:22 |
| *** devkulkarni has joined #openstack-infra | 21:23 | |
| clarkb | devananda: use the fixture from nodepool luke | 21:23 |
| *** james_li has joined #openstack-infra | 21:24 | |
| devananda | clarkb: we've already got all the migration code from oslo-incubator | 21:25 |
| devananda | clarkb: using what nodepool's got would mean ripping that out -- perhaps that's the right thing to do, but then it should be ripped out of oslo too | 21:25 |
| *** starmer_ has quit IRC | 21:25 | |
| devananda | or at least replace a fair chunk of it | 21:25 |
| devananda | and i think some of it is broken anyway | 21:26 |
| clarkb | :/ fwiw the way nodepool does it is probably how it should be done | 21:26 |
| clarkb | maybe we should talk to oslo db folks | 21:26 |
| devananda | yep | 21:26 |
| *** devkulkarni1 has joined #openstack-infra | 21:26 | |
| sdague | the olso stuff is I think just how I hackered it up to work in nova 2 years ago | 21:27 |
| sdague | it works, it's not pretty | 21:27 |
| sdague | agree it should be better | 21:27 |
| devananda | sdague: i suspect boris-42 reworked it before it went into oslo | 21:27 |
| sdague | well the shell out was in my stuff :) | 21:27 |
| devananda | bt based on that | 21:27 |
| devananda | heh | 21:27 |
| *** devkulkarni has quit IRC | 21:28 | |
| devananda | sdague: hah. | 21:28 |
| sdague | but, I had only been on the project 6 months, so old and dirty | 21:28 |
| *** mfer has quit IRC | 21:28 | |
| sdague | ok, bar time. After this week, need some beers. Good weekend all. | 21:29 |
| devananda | i've found two bugs in this already: it "passes" when it runs no tests. and _test_*_opportunistically fails when a configured backend is not present (exactly what it should be preventing) | 21:29 |
| devananda | sdague: enjoy! have a good weekend :) | 21:29 |
| clarkb | devananda: nice | 21:29 |
| sdague | devananda: actually, it's supposed to do that fail | 21:29 |
| sdague | that *not* fail | 21:29 |
| sdague | there was this whole convoluted logic that it was supposed to skip if the db wasn't there | 21:30 |
| sdague | so it was laptop friendly | 21:30 |
| devananda | right | 21:30 |
| devananda | my point is, that's now broken | 21:30 |
| sdague | then another env var to force it on | 21:30 |
| devananda | it now FAILS when the backend is missing | 21:30 |
| sdague | for gate | 21:30 |
| sdague | oh... barf | 21:30 |
| devananda | yea | 21:30 |
| devananda | it reads in test_migrations.conf | 21:30 |
| sdague | oh, ffs | 21:31 |
| devananda | if i uncomment the #mysql line, then it fails | 21:31 |
| sdague | yeh, so that's definitely wrong | 21:31 |
| devananda | because alembic is called in setUp() | 21:31 |
| devananda | before it runs _check_mysql_available | 21:31 |
| devananda | ** _have_mysql | 21:31 |
| sdague | yeh, honestly, a lot of oslo.db seems to have lost things that were working in nova | 21:31 |
| devananda | yep | 21:31 |
| sdague | I think this is the problem with not forward testing it | 21:32 |
| sdague | it goes off and evolves for 6 months without ever being used | 21:32 |
| sdague | happens to break a ton of nova, takes us a long time to back out why | 21:32 |
| devananda | yea. we switched to oslo-incubator/db code as soon as juno opened | 21:32 |
| devananda | most things worked ... only now we're trying to land a db migration | 21:33 |
| devananda | and realizing that it's not testing them at all | 21:33 |
| sdague | well, have fun with that :) | 21:33 |
| devananda | hah | 21:33 |
| sdague | ok, beer time for real | 21:33 |
| devananda | thanks :) | 21:33 |
| devananda | o/ | 21:33 |
| *** eharney has joined #openstack-infra | 21:34 | |
| *** markwash has quit IRC | 21:34 | |
| *** chuckC has joined #openstack-infra | 21:34 | |
| *** starmer_ has joined #openstack-infra | 21:35 | |
| clarkb | fungi: mestery I have held 158422 and 158128 | 21:37 |
| *** nati_ueno has quit IRC | 21:37 | |
| devananda | dhellmann: around? | 21:38 |
| *** mmaglana has joined #openstack-infra | 21:38 | |
| dhellmann | devananda, sdague, clarkb : it would be good for you to provide this feedback about the db lib and migrations to victors and rpodolyaka (they own oslo.db) | 21:39 |
| devananda | dhellmann: in looking at oslo-incubator/db stuff, i'm wondering how this could possibly work | 21:39 |
| devananda | dhellmann: https://github.com/openstack/oslo-incubator/blob/master/openstack/common/db/sqlalchemy/test_migrations.py#L90 | 21:39 |
| *** mmaglana has quit IRC | 21:40 | |
| devananda | that comment makes absolutely no sense | 21:40 |
| *** starmer_ has quit IRC | 21:40 | |
| dhellmann | devananda: yeah, I'm not sure what that means. setup() is definitely run for each test. | 21:41 |
| devananda | setUp is calling _reset_databases, but afaict, every test uses the same db connection parameters | 21:41 |
| devananda | and now that i got ironic to actually run the migration tests, if I run them in parallel, sure enough, i get threads blowing away each other's schema mid-test | 21:41 |
| devananda | dhellmann: fwiw, i just filed https://bugs.launchpad.net/oslo/+bug/1327397 | 21:41 |
| uvirtbot | Launchpad bug 1327397 in ironic "No notice given when db migrations are not run due to missing engine" [Low,In progress] | 21:41 |
| *** mriedem has quit IRC | 21:41 | |
| *** thedodd has quit IRC | 21:42 | |
| devananda | it's been "passing" in our unit tests for months now :( | 21:42 |
| dhellmann | devananda: I'm going to have to confess ignorance of these modules. victor and roman ought to be able to help, though. | 21:42 |
| devananda | ack. i'll drop something on the ML, but then need to context switch | 21:43 |
| dhellmann | devananda: yeah, sorry I can't be of more direct help :-/ | 21:43 |
| *** basha has quit IRC | 21:45 | |
| openstackgerrit | Anita Kuno proposed a change to openstack-infra/elastic-recheck: Add query for ceilometer periodic notifier ut bug 1321826 https://review.openstack.org/98149 | 21:45 |
| uvirtbot | Launchpad bug 1321826 in ceilometer "periodic notifier unit test failure" [Medium,In progress] https://launchpad.net/bugs/1321826 | 21:45 |
| *** r-daneel has joined #openstack-infra | 21:48 | |
| *** palar has joined #openstack-infra | 21:48 | |
| openstackgerrit | Anita Kuno proposed a change to openstack-infra/elastic-recheck: Add query for ceilometer test_notification_service bug 1327344 https://review.openstack.org/98535 | 21:50 |
| uvirtbot | Launchpad bug 1327344 in ceilometer "notification test failure in gate" [Medium,Triaged] https://launchpad.net/bugs/1327344 | 21:50 |
| *** starmer_ has joined #openstack-infra | 21:50 | |
| jesusaurus | so, it looks like prepare_node.sh expects a hostname to be passed in at $1 but that parameter is never actually being passed in | 21:51 |
| clarkb | jesusaurus: yes nodepool bug iirc | 21:51 |
| *** jamielennox is now known as jamielennox|away | 21:52 | |
| jesusaurus | i think it's more an -infra/config bug than a nodepool bug, personally | 21:53 |
| flashgordon | anyone know why grenade-dsvm is blowing up? | 21:53 |
| clarkb | jesusaurus: nodepool says it will pass that argument | 21:53 |
| clarkb | but ti doesn't | 21:53 |
| jesusaurus | oh, in that case, yeah | 21:53 |
| flashgordon | fungi: ^ | 21:53 |
| clarkb | flashgordon: I haven't looked at it | 21:54 |
| flashgordon | jogo.github.io/gate | 21:54 |
| flashgordon | its just going up and up | 21:54 |
| flashgordon | ahh its the ssh test | 21:55 |
| *** starmer_ has quit IRC | 21:55 | |
| flashgordon | looks like east coast is offline for the weekend | 21:55 |
| morganfainberg | fungi, clarb, The keystoneclient related bug, due to oauthlib argument changes is here https://bugs.launchpad.net/python-keystoneclient/+bug/1327430 fix is bing worked on | 21:56 |
| uvirtbot | Launchpad bug 1327430 in python-keystoneclient "get_oauth_params() missing 1 required positional argument: 'request'" [Undecided,New] | 21:56 |
| morganfainberg | clarkb, ^ | 21:56 |
| *** maxbit_ has quit IRC | 21:57 | |
| *** dtroyer has quit IRC | 21:57 | |
| *** maxbit has joined #openstack-infra | 21:57 | |
| *** nati_ueno has joined #openstack-infra | 21:59 | |
| *** nati_ueno has quit IRC | 22:00 | |
| *** signed8bit has quit IRC | 22:01 | |
| *** signed8bit has joined #openstack-infra | 22:01 | |
| *** lascii is now known as alaski | 22:01 | |
| *** nati_ueno has joined #openstack-infra | 22:02 | |
| *** dtroyer has joined #openstack-infra | 22:04 | |
| flashgordon | dtroyer sdague clarkb: anyone there? | 22:05 |
| *** starmer_ has joined #openstack-infra | 22:05 | |
| flashgordon | I am looking into the spike of grenade failures | 22:05 |
| *** signed8bit has quit IRC | 22:06 | |
| flashgordon | got a cirros question | 22:06 |
| flashgordon | devstack old uses cirros 0.3.1 and trunk uses cirros 0.3.2 | 22:06 |
| flashgordon | and we run trunk tempest on old | 22:06 |
| flashgordon | is that an issue | 22:06 |
| *** jp_at_hp has quit IRC | 22:07 | |
| clarkb | you mean grenade uses 0.3.1? | 22:07 |
| *** atiwari has quit IRC | 22:07 | |
| clarkb | its worth pushing a patch for | 22:07 |
| flashgordon | yes | 22:07 |
| clarkb | just to be consistent | 22:08 |
| flashgordon | ttp://logs.openstack.org/71/96571/3/gate/gate-grenade-dsvm/15fe9af/console.html#_2014-06-06_21_15_23_108 | 22:08 |
| *** maxbit_ has joined #openstack-infra | 22:08 | |
| jesusaurus | clarkb: prepare_node.sh is running during image creation. it doesn't actually make any sense to do anything with the hostname there | 22:09 |
| *** starmer_ has quit IRC | 22:10 | |
| *** maxbit has quit IRC | 22:12 | |
| clarkb | jesusaurus: it still has a hostname | 22:12 |
| flashgordon | lets see if that helps: https://review.openstack.org/98545 | 22:13 |
| jesusaurus | clarkb: but that hostname is not the hostname of the node being created from the image. | 22:13 |
| jesusaurus | clarkb: in the case of single_use_slave, each slave is going to have its own hostname, which is different from the hostname of the instance used to create the base image | 22:14 |
| jesusaurus | clarkb: we should be passing the hostname in to the ready-script not to the setup script | 22:17 |
| *** devkulkarni1 has quit IRC | 22:17 | |
| clarkb | jesusaurus: we should do both | 22:17 |
| clarkb | because we use sudo in the image builds too | 22:18 |
| clarkb | we want to solve the same problem in both places | 22:18 |
| jesusaurus | ah, true | 22:18 |
| *** vipuls has joined #openstack-infra | 22:19 | |
| *** masayukig has joined #openstack-infra | 22:19 | |
| *** sarob has joined #openstack-infra | 22:20 | |
| *** starmer_ has joined #openstack-infra | 22:20 | |
| *** salv-orlando has joined #openstack-infra | 22:25 | |
| jesusaurus | clarkb: where does nodepool say that it passes in hostname to the script? i dont see that documented anywhere | 22:25 |
| *** starmer_ has quit IRC | 22:25 | |
| *** atiwari has joined #openstack-infra | 22:26 | |
| openstackgerrit | A change was merged to openstack-dev/cookiecutter: Add .gitreview generation https://review.openstack.org/94848 | 22:27 |
| clarkb | I think it is one of those undocumented features | 22:27 |
| clarkb | I think the issue is that it does it for normal nodes but not image build nodes | 22:28 |
| *** james_li has quit IRC | 22:28 | |
| openstackgerrit | K Jonathan Harker proposed a change to openstack-infra/nodepool: Pass in hostname as a script parameter https://review.openstack.org/98549 | 22:29 |
| jesusaurus | clarkb: ^^ thats me poking blindly at the data structure | 22:30 |
| *** signed8bit has joined #openstack-infra | 22:32 | |
| *** vipul has quit IRC | 22:34 | |
| *** vipuls is now known as vipul | 22:34 | |
| *** starmer_ has joined #openstack-infra | 22:36 | |
| *** devkulkarni has joined #openstack-infra | 22:36 | |
| *** maxbit has joined #openstack-infra | 22:36 | |
| openstackgerrit | A change was merged to openstack-dev/hacking: Make import checks properly detect blank lines and comments https://review.openstack.org/97988 | 22:36 |
| *** arnaud has quit IRC | 22:37 | |
| *** arnaud has joined #openstack-infra | 22:38 | |
| *** devkulkarni has quit IRC | 22:38 | |
| ianw | dtroyer / sdague : https://review.openstack.org/#/c/95500/ would be helpful for rhel6 testing, if you get a chance (Parse openstackclient output to get EC2 and S3 urls.) | 22:38 |
| clarkb | according to logs both neutron tests with that tempest change succeeded | 22:39 |
| clarkb | not sure if that helps | 22:39 |
| *** otherwiseguy has quit IRC | 22:39 | |
| *** starmer_ has quit IRC | 22:40 | |
| *** maxbit_ has quit IRC | 22:40 | |
| openstackgerrit | A change was merged to openstack-dev/hacking: Clarify where the "following template" is https://review.openstack.org/97698 | 22:41 |
| *** signed8bit has quit IRC | 22:41 | |
| fungi | flashgordon: east coast is off drinking for the weekend. the whole coast. all of us | 22:42 |
| flashgordon | fungi: ^_^ | 22:43 |
| *** adalbas has quit IRC | 22:43 | |
| *** gondoi is now known as zz_gondoi | 22:44 | |
| flashgordon | is there a bug open for tempest.scenario.test_snapshot_pattern.TestSnapshotPattern.test_snapshot_pattern FAILURE | 22:44 |
| fungi | flashgordon: it's one of those traditions you wouldn't know about unless you lived here | 22:44 |
| flashgordon | fungi: I grew up in MD | 22:44 |
| *** chuckC has quit IRC | 22:44 | |
| fungi | oh, then you should have already been aware. did you live next door to john waters, by any chance? | 22:45 |
| flashgordon | fungi: I wish | 22:45 |
| * anteaya is off drinking | 22:45 | |
| * anteaya except she doesn't drink | 22:46 | |
| fungi | flashgordon: so, yes, i suspect the grenade bug rise is more of bug 1323658 | 22:46 |
| uvirtbot | Launchpad bug 1323658 in neutron "SSH EOFError - Public network connectivity check failed" [Undecided,New] https://launchpad.net/bugs/1323658 | 22:46 |
| fungi | hard to say for sure | 22:46 |
| flashgordon | its always the same test that fails | 22:46 |
| fungi | but the handful i've peeked at looked like that one | 22:46 |
| SlickNik | anteaya: you don't need to drink to be off drinking :) | 22:46 |
| anteaya | SlickNik: I like your definition | 22:46 |
| anteaya | so yeah, I'm off drinking | 22:46 |
| fungi | gah | 22:47 |
| flashgordon | fungi: I think this is subtly a different bug | 22:47 |
| fungi | flashgordon: possibly | 22:47 |
| fungi | one of my patches just hit an ssh timeout, but not on a neutron job :/ | 22:48 |
| fungi | this one struck in a novanet based job, on test_volume_boot_pattern | 22:48 |
| anteaya | fungi: https://bugs.launchpad.net/tempest/+bug/1298472 | 22:49 |
| uvirtbot | Launchpad bug 1298472 in tempest "tempest.scenario.test_volume_boot_pattern.TestVolumeBootPattern.test_volume_boot_pattern fails with TimeoutException on SSH connection" [Undecided,In progress] | 22:49 |
| *** zns has quit IRC | 22:49 | |
| fungi | anteaya: yep, that--thanks! | 22:49 |
| anteaya | np | 22:49 |
| *** sarob has quit IRC | 22:50 | |
| *** sarob has joined #openstack-infra | 22:50 | |
| *** starmer_ has joined #openstack-infra | 22:50 | |
| fungi | clarkb: thanks for holding those nodes. i'm recycling them now since the jobs on them succeeded | 22:50 |
| clarkb | ok | 22:51 |
| fungi | can't remember the last time i ran jobs hoping to catch them fail | 22:51 |
| flashgordon | so I am looking at test snapshot_pattern failing | 22:51 |
| flashgordon | message:"tempest.scenario.test_snapshot_pattern.TestSnapshotPattern.test_snapshot_pattern" AND message:"FAILED" | 22:52 |
| fungi | flashgordon: in grenade jobs? | 22:52 |
| *** palar has quit IRC | 22:52 | |
| SlickNik | clarkb / fungi: Can you take a quick look at https://review.openstack.org/#/c/97608/ when you get a chance? Just a couple of fixes to the experimental trove-image-build job to correct the location from where the image is copied and the name of the working directory to avoid confusion. | 22:52 |
| SlickNik | Thanks! | 22:52 |
| flashgordon | fungi: yes | 22:52 |
| *** palar has joined #openstack-infra | 22:53 | |
| *** marcoemorais has quit IRC | 22:53 | |
| *** rfolco has quit IRC | 22:53 | |
| *** sarob_ has joined #openstack-infra | 22:53 | |
| *** marcoemorais has joined #openstack-infra | 22:54 | |
| *** sarob has quit IRC | 22:54 | |
| *** starmer_ has quit IRC | 22:55 | |
| flashgordon | preliminary bug report https://bugs.launchpad.net/nova/+bug/1327445 | 22:55 |
| *** Sukhdev has joined #openstack-infra | 22:55 | |
| uvirtbot | Launchpad bug 1327445 in tempest "tempest.scenario.test_snapshot_pattern.TestSnapshotPattern.test_snapshot_pattern is failing on icehouse in devstack" [Undecided,New] | 22:55 |
| *** UtahDave has joined #openstack-infra | 22:56 | |
| fungi | SlickNik: do you have an example failure from the image build job trying to upload to the wrong path? | 22:56 |
| *** palar has quit IRC | 22:57 | |
| *** lcostantino has joined #openstack-infra | 22:57 | |
| *** lcostantino has quit IRC | 22:58 | |
| *** r-daneel has quit IRC | 22:58 | |
| *** doug-fish has left #openstack-infra | 23:01 | |
| *** Sukhdev_ has joined #openstack-infra | 23:03 | |
| flashgordon | fungi: in grenade where is the full tempest logs from the 'old' run stored? | 23:03 |
| *** W00die has quit IRC | 23:03 | |
| *** viktors|afk has quit IRC | 23:04 | |
| *** homeless has quit IRC | 23:05 | |
| *** Sukhdev has quit IRC | 23:05 | |
| *** viktors has joined #openstack-infra | 23:06 | |
| clarkb | they werent | 23:07 |
| clarkb | mtreinish has a change to fix that | 23:07 |
| flashgordon | ahh never mind I think | 23:07 |
| flashgordon | so the failure case I am seeing is: boot instance, attach floating ip ssh in. snapshot instance boot second copy, attach floating ip. ssh fail | 23:08 |
| *** lcostantino has joined #openstack-infra | 23:08 | |
| fungi | flashgordon: yeah, if it turns out to be urgent to help debugging gate failures, we can enqueue and promote 98406 | 23:11 |
| *** dims_ has joined #openstack-infra | 23:11 | |
| *** nati_ueno has quit IRC | 23:11 | |
| fungi | flashgordon: the irony is, it didn't make it to the gate because of a test_snapshot_pattern failure with SSHTimeout | 23:12 |
| fungi | (in a grenade job, of course) | 23:12 |
| fungi | flashgordon: in fact, maybe the pre-upgrade tempest logs from that failure will help? ;) http://logs.openstack.org/06/98406/1/check/check-grenade-dsvm-partial-ncpu/4ac0551/logs/old/ | 23:13 |
| fungi | er, not tempest logs, but devtsack/service logs from the pre-upgrade tempest run | 23:14 |
| *** dims has quit IRC | 23:14 | |
| *** dangers is now known as dangers_away | 23:14 | |
| *** lcostantino has quit IRC | 23:15 | |
| flashgordon | strange the EC2 is being used | 23:15 |
| fungi | flashgordon: http://logs.openstack.org/06/98406/1/check/check-grenade-dsvm-partial-ncpu/4ac0551/logs/old/screen-n-api.txt.gz?level=INFO#_2014-06-06_20_52_00_216 | 23:15 |
| flashgordon | fungi: that is a common error | 23:16 |
| flashgordon | it doesn't maen what you think it does | 23:16 |
| flashgordon | lol | 23:16 |
| fungi | got it. i definitely don't spend much time staring at these logs (i have plenty of non-openstack-related system logs to stare at already) | 23:16 |
| flashgordon | but I think its worth prioritizing 98406 | 23:17 |
| flashgordon | as I don't think that can hurt | 23:17 |
| *** chuckC has joined #openstack-infra | 23:17 | |
| fungi | there's been quite a few nova changes bouncing out of the gate now on unit test jobs with "PolicyNotAuthorized: Policy doesn't allow example:get_http to be performed." | 23:20 |
| fungi | is that something which needs some more eyeballs? | 23:20 |
| flashgordon | fungi: yeah for https://bugs.launchpad.net/tempest/+bug/1298472 | 23:21 |
| uvirtbot | Launchpad bug 1298472 in tempest "tempest.scenario.test_volume_boot_pattern.TestVolumeBootPattern.test_volume_boot_pattern fails with TimeoutException on SSH connection" [Undecided,In progress] | 23:21 |
| flashgordon | so I am seeing ec2 API calls | 23:21 |
| flashgordon | not sure why | 23:21 |
| flashgordon | err https://bugs.launchpad.net/nova/+bug/1327445 | 23:21 |
| uvirtbot | Launchpad bug 1327445 in tempest "tempest.scenario.test_snapshot_pattern.TestSnapshotPattern.test_snapshot_pattern is failing on icehouse in devstack" [Undecided,New] | 23:22 |
| anteaya | I'm running away for the weekend, see you Monday | 23:22 |
| anteaya | or if I get wifi | 23:22 |
| fungi | have a good weekend anteaya | 23:22 |
| flashgordon | fungi: http://logs.openstack.org/45/84945/13/gate/gate-grenade-dsvm/3c7dd97/logs/old/screen-n-api.txt.gz#_2014-06-06_16_51_57_851 | 23:22 |
| flashgordon | anteaya: o/ | 23:22 |
| fungi | nova.api.ec2.cloud | 23:23 |
| fungi | yep | 23:23 |
| *** nati_ueno has joined #openstack-infra | 23:24 | |
| flashgordon | so something is dissaciosating the addr that the tempest test is trying to use | 23:24 |
| openstackgerrit | K Jonathan Harker proposed a change to openstack-infra/nodepool: Pass in hostname as a script parameter https://review.openstack.org/98549 | 23:24 |
| flashgordon | not sure if this is red herring due to timings | 23:25 |
| fungi | flashgordon: http://logs.openstack.org/45/84945/13/gate/gate-grenade-dsvm/3c7dd97/logs/old/screen-n-net.txt.gz#_2014-06-06_16_51_57_998 | 23:26 |
| openstackgerrit | A change was merged to openstack-infra/config: Fix experimental trove-image-build gate job https://review.openstack.org/97608 | 23:26 |
| fungi | sudo nova-rootwrap /etc/nova/rootwrap.conf ip addr del 172.24.4.1/32 dev br100 | 23:26 |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: Add support for disk-image-builder in nodepool https://review.openstack.org/88479 | 23:26 |
| fungi | flashgordon: oh, i'm looking at the downstream result of the api call | 23:26 |
| fungi | nevermind | 23:26 |
| *** Longgeek has joined #openstack-infra | 23:28 | |
| fungi | flashgordon: could be seeing ec2 calls there because tempest.thirdparty.boto.test_s3_ec2_images.S3ImagesTest.test_register_get_deregister_ami_image started 10 seconds before | 23:28 |
| fungi | oh, wait, that's the end time not the start time | 23:29 |
| fungi | though tempest.thirdparty.boto.test_ec2_instance_run.InstanceRunTest.test_compute_with_volumes ends a few seconds later and takes nearly a minute | 23:29 |
| *** marcoemorais has quit IRC | 23:29 | |
| flashgordon | that could be in | 23:30 |
| flashgordon | could be it | 23:30 |
| *** marcoemorais has joined #openstack-infra | 23:30 | |
| flashgordon | two servers fighting for same floating IP? | 23:30 |
| *** Sukhdev_ has quit IRC | 23:30 | |
| fungi | hrm... maybe? | 23:30 |
| *** Longgeek has quit IRC | 23:32 | |
| flashgordon | hmm at least in nova-api there is no overlap | 23:35 |
| flashgordon | checking in n-net | 23:35 |
| *** starmer_ has joined #openstack-infra | 23:35 | |
| flashgordon | nothing | 23:36 |
| *** _nadya_ has joined #openstack-infra | 23:37 | |
| *** starmer_ has quit IRC | 23:38 | |
| *** starmer_ has joined #openstack-infra | 23:39 | |
| flashgordon | so this has been failing for a long time I am thinking just turn off this test | 23:41 |
| clarkb | mordred: ok 88479 runs puppet cleanly and allows you to injcet a dev puppet modules repo. I am happy with it for precise | 23:42 |
| *** _nadya_ has quit IRC | 23:42 | |
| clarkb | going t okick off a trusty build soonish (qemu is still converting my precise chroot into an image) and see how badly it blows up | 23:42 |
| fungi | right on | 23:43 |
| clarkb | though this hasn't tackled the DB problem | 23:43 |
| clarkb | still focusing on devtack-* image because its easier :) | 23:43 |
| fungi | oh, as far as working it to start the dbs in the chroot on fifos | 23:44 |
| fungi | and then preconfiguring them thataway | 23:44 |
| clarkb | ya | 23:44 |
| *** amcrn has quit IRC | 23:45 | |
| *** e0ne has joined #openstack-infra | 23:45 | |
| clarkb | will need to refactor the puppet DB stuff so that it installs them but does not manage services or configure anything when in chroot | 23:45 |
| clarkb | then as a post install step we do the config and managemetn for postres and mysql | 23:45 |
| clarkb | and trusty build started. let us see how badly this dies | 23:47 |
| *** e0ne has quit IRC | 23:47 | |
| *** crc32 has quit IRC | 23:47 | |
| asselin | clarkb, I still haven't been able to get parimiko to ssh out of http proxy. I haven't found much on the web except this: http://stackoverflow.com/questions/10488832/how-to-ssh-over-http-proxy-in-python | 23:50 |
| flashgordon | fungi: so here is a workaround: https://review.openstack.org/98558 | 23:50 |
| flashgordon | skip the test | 23:50 |
| flashgordon | in the worst case | 23:50 |
| clarkb | asselin: you can't | 23:50 |
| asselin | clarkb, seems there's a possible solution by chaning zuul to have paramiko use a predefined socket. | 23:50 |
| clarkb | asselin: I mean you probably can but I wouldn't want to | 23:50 |
| fungi | flashgordon: add that one to the etherpad in case we want to promote it | 23:51 |
| clarkb | asselin: but if yo uare at HP you don't need to proxy to review.openstack.org | 23:51 |
| asselin | clarkb, why is that? | 23:51 |
| clarkb | asselin: because ssh isn't an http protocol | 23:51 |
| *** sarob_ has quit IRC | 23:51 | |
| flashgordon | fungi: done | 23:51 |
| flashgordon | fungi: what was the other bug you mentioned popping up? | 23:51 |
| *** sarob has joined #openstack-infra | 23:52 | |
| asselin | clarkb, hp's corporate firewall is blocking external ssh. | 23:52 |
| fungi | clarkb: lies. all network protocols descend from the one true protocol... telnet^H^H^H^H^H^Hhttp | 23:52 |
| clarkb | asselin: it shouldn't | 23:52 |
| clarkb | asselin: not to review.openstack.org | 23:52 |
| clarkb | jesusaurus: ^ | 23:52 |
| asselin | clarkb, for git-review, etc. we use corkscrew to tunnel ssh via http and out of the firewall | 23:52 |
| clarkb | asselin: and you shouldn't need to | 23:52 |
| clarkb | asselin: we finally fixed that | 23:52 |
| asselin | clarkb, how so? | 23:52 |
| clarkb | asselin: connections from hp to review.openstack.org over port 29418 are allowed by the firewall | 23:53 |
| fungi | flashgordon: ssh timeouts. i think last i saw that was nullified by disabling a test as well | 23:53 |
| clarkb | asselin: it was broken for a short period of time but jesusaurus says it is fixed now | 23:53 |
| flashgordon | fungi: sonething about 'PolicyNotAuthorized:' | 23:53 |
| asselin | clarkb, jesusaurus any contacts on this? it doesn't work from my location | 23:53 |
| asselin | clarkb, jesusaurus which proxy are you using? | 23:53 |
| clarkb | asselin: I refuse to use a proxy >_> and vpn | 23:54 |
| fungi | flashgordon: oh, the thing killing nova unit tests in the gate now | 23:54 |
| clarkb | asselin: I would test it right now if it wasn't complicated due to vpn routing | 23:54 |
| clarkb | asselin: an easy way to test is ssh -p 29418 user@review.openstack.org gerrit ls-projects | 23:54 |
| fungi | flashgordon: "PolicyNotAuthorized: Policy doesn't allow example:get_http to be performed." | 23:54 |
| * fungi hits up logstash | 23:55 | |
| flashgordon | message:"PolicyNotAuthorized: Policy doesn\'t allow example:get_http to be performed." AND ( build_name:"gate-nova-python27" OR build_name:"gate-nova-python26") AND project:"openstack/nova" | 23:55 |
| asselin | clarkb, I've spent a lot of time in the past few months working around hp's http proxies..... | 23:55 |
| asselin | clarkb, that works perfectly with corkscrew enabled. | 23:55 |
| clarkb | asselin: for many things you do need to talk through the proxy | 23:55 |
| clarkb | asselin: right so try it without corkscrew | 23:55 |
| flashgordon | fungi: ah hqueries/1288178.yaml: | 23:55 |
| *** sarob has quit IRC | 23:56 | |
| flashgordon | ohh its the new policy stuff | 23:56 |
| flashgordon | derp | 23:56 |
| clarkb | fungi: lots of red puppet warnings but nothing fatal as far as I can tell. Course now that i have said that... | 23:56 |
| flashgordon | dhellmann: ^ new oslo policy pain | 23:56 |
| flashgordon | fungi: that one is easy | 23:57 |
| flashgordon | gonna push up a revert | 23:57 |
| fungi | flashgordon: the logstash frequency graph on that one looks like a clear warning | 23:57 |
| fungi | four hits on change 87002 spaced over a while, then a slew of hits from numerous changes | 23:58 |
| fungi | looks like they really started in full bore today around 1600 utc | 23:59 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!
