| *** zz_dimtruck is now known as dimtruck | 00:00 | |
| *** xarses_ has quit IRC | 00:02 | |
| pleia2 | Kiall: btw, not sure if someone actively did anything, but I had looked at the designate-tempest-plugin merge about an hour before you asked for wznoinsk and it was still in the post queue, it's up on git.o.o now | 00:02 |
|---|---|---|
| *** thorst has quit IRC | 00:02 | |
| *** Goneri has quit IRC | 00:02 | |
| *** akshai has joined #openstack-infra | 00:03 | |
| clarkb | pleia2: ya ansible stopped running whihc kept puppet form updating the things | 00:03 |
| clarkb | pabelanger sorted that out while I nodepoold | 00:03 |
| pleia2 | clarkb: ah, gotcha | 00:03 |
| pleia2 | thanks pabelanger \o/ | 00:03 |
| openstackgerrit | Merged openstack-infra/project-config: Add non-voting pylint job to os-brick experimental https://review.openstack.org/301938 | 00:04 |
| pabelanger | Yup, things should be back to normal for puppet now | 00:05 |
| greghaynes | clarkb: and uploads are going fine? | 00:05 |
| pabelanger | clarkb: do we have any docs to enqueue a periodic job in zuul? | 00:06 |
| *** arxcruz has quit IRC | 00:08 | |
| clarkb | greghaynes: mostly, occasionally have to restart ovh uploads | 00:09 |
| clarkb | pabelanger: uh zuul enqueue? | 00:09 |
| clarkb | pabelanger: you can also likely just trigger it in jenkins directly | 00:10 |
| *** akshai_ has joined #openstack-infra | 00:10 | |
| clarkb | greghaynes: I am saving rax for last since those are fun | 00:10 |
| pabelanger | clarkb: tried to enqueue but didn't take. Wonder if I need to use a --ref | 00:11 |
| clarkb | periodic isn't ref'd though | 00:11 |
| clarkb | did you set the trigger to timer? | 00:11 |
| pabelanger | clarkb: seems the job requires zuul params, so web UI won't work | 00:11 |
| clarkb | instead of gerrit? | 00:11 |
| pabelanger | Oh | 00:11 |
| pabelanger | Ya, I used gerrit | 00:11 |
| clarkb | I think it may need to be timer | 00:11 |
| pabelanger | Ah | 00:11 |
| pabelanger | there we go | 00:11 |
| pabelanger | danke | 00:12 |
| *** mriedem has left #openstack-infra | 00:12 | |
| *** akshai has quit IRC | 00:13 | |
| *** dimtruck is now known as zz_dimtruck | 00:13 | |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/system-config: Add election alias to governance.openstack.org https://review.openstack.org/292675 | 00:14 |
| *** tqtran has quit IRC | 00:15 | |
| pabelanger | ianw: okay, not sure I did it right. But will wait and see | 00:15 |
| *** sflanigan has quit IRC | 00:17 | |
| ianw | ok | 00:17 |
| *** sputnik13 has quit IRC | 00:18 | |
| ianw | jhesketh / pabelanger : so everything seems to have built correctly, so presumably the newly uploaded devstack-centos7 images will unblock puppet CI | 00:18 |
| *** sputnik13 has joined #openstack-infra | 00:19 | |
| clarkb | ianw: note they probably won't be put into use until after the release at 1400UTC today (UTC) | 00:19 |
| *** mriedem has joined #openstack-infra | 00:19 | |
| pabelanger | Looks like kolla won't be tagging mitaka until the 15th now... so devstack-centos7 might live a little longer | 00:20 |
| clarkb | pabelanger: meh | 00:20 |
| jhesketh | ianw: cool, thanks :-) | 00:20 |
| pabelanger | clarkb: indeed | 00:20 |
| clarkb | at this point we have all the disk we need :) | 00:20 |
| pabelanger | clarkb: going to get an experiment job and test their gates still | 00:21 |
| pabelanger | indeed | 00:21 |
| *** amrith is now known as _amrith_ | 00:21 | |
| pabelanger | rally should be off bare-precise by tomorrow | 00:21 |
| *** sflanigan has joined #openstack-infra | 00:21 | |
| clarkb | oh good, I am slightly worriedabout bare-precise because it is a snapshot image so I can't build it with nodepoold offline :/ | 00:22 |
| clarkb | I can proably edit the db directly to add the existing ones in | 00:23 |
| *** sputnik13 has quit IRC | 00:23 | |
| clarkb | or we just wait an hour ofr one to build | 00:23 |
| clarkb | the release doesn't need it so not too worried | 00:23 |
| *** thorst has joined #openstack-infra | 00:25 | |
| pabelanger | think we should be safe to let nodepoold build it | 00:25 |
| pabelanger | just means system-config / project-config are blocked | 00:25 |
| *** ddieterly[away] has quit IRC | 00:26 | |
| clarkb | ya | 00:26 |
| *** mtanino__ has joined #openstack-infra | 00:27 | |
| *** Qiming has quit IRC | 00:28 | |
| *** mtanino has quit IRC | 00:28 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Experimental gate-kolla-dsvm-build-centos-7-binary https://review.openstack.org/302481 | 00:31 |
| pabelanger | Ya, so just kolla left (and us for devstack-centos7) | 00:32 |
| *** asettle has quit IRC | 00:32 | |
| pabelanger | ^ will give us an experimental job to test drive in kolla | 00:32 |
| *** mtanino__ has quit IRC | 00:33 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/nodepool: Add some tracking for image sizes https://review.openstack.org/289678 | 00:34 |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Migrate rally to centos-7 https://review.openstack.org/301334 | 00:36 |
| ianw | distro: oraclelinuxnode: 'devstack-centos7' | 00:36 |
| pabelanger | Ya see that | 00:36 |
| pabelanger | seems odd | 00:36 |
| ianw | not sure i want to even know what's going on | 00:36 |
| pabelanger | :) | 00:36 |
| *** erlon has quit IRC | 00:37 | |
| *** akshai_ has quit IRC | 00:37 | |
| *** achanda has quit IRC | 00:40 | |
| *** ddieterly has joined #openstack-infra | 00:40 | |
| *** _amrith_ is now known as amrith | 00:40 | |
| *** tiswanso has quit IRC | 00:40 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Experimental gate-kolla-dsvm-build-centos-7-binary https://review.openstack.org/302481 | 00:40 |
| jhesketh | So has anybody discussed mirroring npm in infra before? | 00:41 |
| *** sc68cal has quit IRC | 00:42 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Set min-ready for devstack-centos7 to 0 https://review.openstack.org/302484 | 00:42 |
| pabelanger | jhesketh: I think krotscheck is working on it | 00:43 |
| clarkb | jhesketh: I think krotscheck was working on it and there is an afs volume and everything | 00:43 |
| clarkb | unfortunately I don't think it will fix all of our issues if mirroring external link things works the same way as with pypi | 00:43 |
| clarkb | would be good to figur ethat out | 00:43 |
| jhesketh | cool | 00:44 |
| jhesketh | I'll see if I can find the changes | 00:44 |
| jhesketh | yeah but it's something we should be doing for the same reasons as pypi | 00:45 |
| clarkb | except that I htink it won't fix any of the recent fails we have had | 00:45 |
| pabelanger | I am surprised we have xenial-backports enabled: http://logs.openstack.org/26/300526/1/experimental/gate-project-config-bindep-fallback-ubuntu-xenial/6025d22/console.html | 00:45 |
| *** thorst has quit IRC | 00:45 | |
| pabelanger | do we do that for trusty too? | 00:45 |
| clarkb | the first was bitbucket rejecting downloads on external links, the other was json encoded in a way that npm couldn't understand | 00:45 |
| clarkb | having a mirror won't help either case aiui | 00:45 |
| *** thorst has joined #openstack-infra | 00:46 | |
| clarkb | pabelanger: I think there was a thing that spamaps fixed with that? | 00:46 |
| *** keedya has quit IRC | 00:46 | |
| jhesketh | clarkb: oh yeah, agreed.. just thinking about it in general | 00:46 |
| *** Jeffrey4l has joined #openstack-infra | 00:48 | |
| pabelanger | clarkb: Ya, looks like we mirror it with reprepro too | 00:48 |
| pabelanger | so, some projects must us it | 00:48 |
| *** annegentle has joined #openstack-infra | 00:49 | |
| krotscheck | jhesketh: Yes. I have an approach, it requires some modification of an existing registry mirror. | 00:50 |
| jhesketh | krotscheck: cool, do you have a review topic or a list of required work? | 00:50 |
| krotscheck | jhesketh: It's pretty far down my todo list what with the summit coming up. I have nothing written down. | 00:52 |
| *** r1chardj0n3s has joined #openstack-infra | 00:52 | |
| jhesketh | okay, no worries | 00:52 |
| krotscheck | jhesketh: the project is here https://github.com/krotscheck/js-registry-static-afs | 00:52 |
| Shrews | jhesketh: it wasn't the merge speed so much as there were things that were missed. i +A'd that one | 00:53 |
| jhesketh | krotscheck: oh how annoyign to have to work around that limit | 00:53 |
| krotscheck | r1chardj0n3s, jhesketh: The project right now is a bunch of noop hooks. I managed to get so far as to have the tarballs be rewritten into the appropriate folder structure, but the mirror doesn't permit doing the same with the json files. | 00:54 |
| Shrews | jhesketh: i'm going to be writing up a reviewer guide so mordred and myself aren't the only ones holding the context you were missing :) | 00:54 |
| jhesketh | Shrews: noted. It was certainly my mistake, sorry. | 00:54 |
| clarkb | all image builds complete with no issues | 00:54 |
| clarkb | and I have started the rax uploads | 00:54 |
| Shrews | jhesketh: no worries | 00:54 |
| *** zz_dimtruck is now known as dimtruck | 00:54 | |
| *** thorst has quit IRC | 00:54 | |
| clarkb | once !rax is done I am just going to queue up all the rax uploads | 00:54 |
| clarkb | then stop computering for a bit | 00:54 |
| *** annegentle has quit IRC | 00:54 | |
| krotscheck | r1chardj0n3s, jhesketh: So the next step is to go to registry-static and the yahoo team and check with them how to modify the existing script to permit that. | 00:54 |
| r1chardj0n3s | krotscheck: is there some background on that porject and why existing npm mirroring solutions aren't being used? | 00:55 |
| pabelanger | clarkb: are you doing the cut over tonight, or going to wait until tomorrow morning? | 00:55 |
| krotscheck | r1chardj0n3s: Existing solutions like what, mirroring the CouchDB? | 00:55 |
| clarkb | pabelanger: I think tomorrow morning after release team releases | 00:56 |
| clarkb | that way I don't have to work until 10pm tonight | 00:56 |
| krotscheck | r1chardj0n3s: NPM explicitly disallows that one now. Their fullfat db has been deprecated and is no longer guaranteed to exist in the future. | 00:56 |
| pabelanger | clarkb: sounds good. I'll sign off now then | 00:57 |
| pabelanger | later all | 00:57 |
| *** zhurong has joined #openstack-infra | 00:58 | |
| krotscheck | I just got "E: Unable to locate package apt-transport-https" on a rax-dfw node, any ideas? | 00:59 |
| r1chardj0n3s | krotscheck: I'm not sure what you're referring to with the couchdb stuff; I've only just started looking into the mirroring and started at https://docs.npmjs.com/enterprise/mirroring | 01:00 |
| *** harlowja has quit IRC | 01:00 | |
| krotscheck | r1chardj0n3s NPM on-site is their $$$ solution | 01:01 |
| krotscheck | r1chardj0n3s https://www.npmjs.com/enterprise/license | 01:01 |
| r1chardj0n3s | ah, right | 01:02 |
| krotscheck | r1chardj0n3s So we're doing this instead: https://github.com/davglass/registry-static | 01:02 |
| clarkb | krotscheck: where? | 01:02 |
| clarkb | krotscheck: always always provide job log links :) | 01:02 |
| krotscheck | clarkb http://logs.openstack.org/79/301679/1/check/ironic-webclient-npm-publish-draft/024f53b/console.html | 01:02 |
| *** bpokorny has quit IRC | 01:02 | |
| r1chardj0n3s | krotscheck: ok, cool. | 01:02 |
| *** pvaneck has quit IRC | 01:03 | |
| *** harlowja has joined #openstack-infra | 01:03 | |
| r1chardj0n3s | krotscheck: I think I'll step back and stop interfering. If I can help though, please ask. | 01:03 |
| krotscheck | r1chardj0n3s: Would _love_ the help. But I have dinner with the fam now, so can't do much right now to loop you in :) | 01:04 |
| r1chardj0n3s | krotscheck: okie; also, I've thrown jhesketh under the "help if possible from the infra side" bus :-) | 01:04 |
| clarkb | krotscheck: best guess based on that is you are lacking an apt-get update before apt-get install and it doens't know where the package is as a result | 01:04 |
| openstackgerrit | Fei Long Wang proposed openstack-infra/project-config: Add Tempest gate job for Zaqar https://review.openstack.org/302487 | 01:04 |
| *** jcooley_ has joined #openstack-infra | 01:07 | |
| *** r1chardj0n3s has left #openstack-infra | 01:08 | |
| *** tqtran has joined #openstack-infra | 01:11 | |
| *** tqtran has quit IRC | 01:16 | |
| *** tiswanso has joined #openstack-infra | 01:18 | |
| *** achanda has joined #openstack-infra | 01:18 | |
| *** sridhar_ram1 has quit IRC | 01:18 | |
| *** asettle has joined #openstack-infra | 01:22 | |
| *** dprince has joined #openstack-infra | 01:24 | |
| *** keedya has joined #openstack-infra | 01:24 | |
| openstackgerrit | Fei Long Wang proposed openstack-infra/project-config: Add Tempest gate job for Zaqar https://review.openstack.org/302487 | 01:25 |
| *** Qiming has joined #openstack-infra | 01:26 | |
| *** shashank_hegde has joined #openstack-infra | 01:26 | |
| *** asettle has quit IRC | 01:28 | |
| openstackgerrit | Matt Riedemann proposed openstack/requirements: Require fixtures 2.0.0 https://review.openstack.org/302488 | 01:28 |
| *** sdake has quit IRC | 01:28 | |
| *** aysyd has quit IRC | 01:30 | |
| *** sdake has joined #openstack-infra | 01:32 | |
| *** ajmiller has joined #openstack-infra | 01:38 | |
| *** ajmiller has quit IRC | 01:39 | |
| *** julim has joined #openstack-infra | 01:40 | |
| *** baoli has joined #openstack-infra | 01:44 | |
| *** baoli has quit IRC | 01:47 | |
| *** shashank_hegde has quit IRC | 01:47 | |
| *** ddieterly is now known as ddieterly[away] | 01:47 | |
| *** baoli has joined #openstack-infra | 01:48 | |
| *** thorst has joined #openstack-infra | 01:51 | |
| *** nwkarsten has joined #openstack-infra | 01:52 | |
| *** cody-somerville has joined #openstack-infra | 01:53 | |
| *** Thelo has joined #openstack-infra | 01:55 | |
| Thelo | Hi all, I got this error message in zuul "This change was unable to be automatically merged with the current state of the repository. Please rebase your change and upload a new patchset" (http://paste.openstack.org/show/493257/), however I dont understand why the merge would fail since the patchset has no conflict (https://review.openstack.org/#/c/301264/1), Does anyone know why? and what I can do to fix that? | 01:55 |
| *** esikachev has joined #openstack-infra | 01:55 | |
| fungi | Thelo: it's saying it conflicts with the state of the target branch, not with other proposed changes | 01:57 |
| *** nwkarsten has quit IRC | 01:57 | |
| fungi | also the merge resolver in the jgit library gerrit uses is not as smart as the variety of conflict resolution algorithms cgit employs, so something that rebases cleanly locally may still be something gerrit is unable to figure out how to resolve itself | 01:58 |
| *** thorst has quit IRC | 01:59 | |
| *** amrith is now known as _amrith_ | 01:59 | |
| fungi | and now i'm caught up with scrollback... missed a very exciting day. sorry all! :/ | 01:59 |
| fungi | clarkb: thanks for building us a new nodepool | 02:00 |
| *** esikachev has quit IRC | 02:00 | |
| clarkb | fungi: np | 02:00 |
| clarkb | image uploads are progressing well should be ready to switch servers tomorrow | 02:00 |
| fungi | there's one other server that can _probably_ be deleted to free up quota now: the old openstackid.org running precise. but since we just replaced it yesterday i'm hesitant to say for certain we won't need to switch back | 02:01 |
| fungi | when that goes we'll get back another 15gb memory quota | 02:01 |
| clarkb | afstest was sufficient for my needs | 02:01 |
| fungi | i saw in skimming | 02:01 |
| fungi | i didn't mean to leave us that tight on quota | 02:02 |
| Thelo | fungi: So if I understand it has nothing to do with this patchset but with the other party of the merge. If so, how can I rebase locally? Do you know where zuul maintain it? | 02:02 |
| *** apoorvad has quit IRC | 02:02 | |
| fungi | also the old groups.o.o may also be able to be deleted if mrmartin and yolanda have finished with that | 02:02 |
| *** kengo_sakai has joined #openstack-infra | 02:02 | |
| *** salv-orlando has joined #openstack-infra | 02:03 | |
| fungi | Thelo: download the change, rebase it on the current branch tip, then push it back up for review | 02:03 |
| fungi | if i understand what you're asking | 02:03 |
| *** rajinir has quit IRC | 02:05 | |
| *** shashank_hegde has joined #openstack-infra | 02:05 | |
| fungi | Thelo: oh, _your_ zuul is reporting that it can't merge a change? | 02:06 |
| fungi | i thought you were saying our zuul was leaving a comment on a change of yours that it couldn't merge | 02:06 |
| *** baoli has quit IRC | 02:06 | |
| Thelo | fungi: Yes | 02:06 |
| fungi | Thelo: you may need to look at the log for your merger worker. one cause could be a corrupt openstack/cinder repo on disk | 02:09 |
| fungi | Thelo: basically the merger worker reported to zuul that it was unable to create a merge commit in response to a merge work request | 02:09 |
| Thelo | fungi: ok, thanks I ll check that | 02:13 |
| *** mriedem has quit IRC | 02:16 | |
| *** piet has joined #openstack-infra | 02:17 | |
| *** dprince has quit IRC | 02:19 | |
| *** sdake_ has joined #openstack-infra | 02:20 | |
| *** ddieterly[away] is now known as ddieterly | 02:21 | |
| *** sdake has quit IRC | 02:22 | |
| *** salv-orlando has quit IRC | 02:23 | |
| *** shashank_hegde has quit IRC | 02:24 | |
| *** ddieterly is now known as ddieterly[away] | 02:26 | |
| *** salv-orlando has joined #openstack-infra | 02:28 | |
| *** asettle has joined #openstack-infra | 02:29 | |
| *** salv-orlando has quit IRC | 02:33 | |
| *** edmondsw has quit IRC | 02:36 | |
| *** piet has quit IRC | 02:37 | |
| *** yamamoto_ has joined #openstack-infra | 02:37 | |
| *** gildub has quit IRC | 02:42 | |
| *** shashank_hegde has joined #openstack-infra | 02:45 | |
| *** jamesmcarthur has joined #openstack-infra | 02:47 | |
| clarkb | this ovh bhs one upload is going forever | 02:48 |
| clarkb | its funny that the cross atlantic uploads are quicker | 02:48 |
| *** jamesmcarthur has quit IRC | 02:50 | |
| *** jamesmcarthur has joined #openstack-infra | 02:51 | |
| *** piet has joined #openstack-infra | 02:55 | |
| krotscheck | clarkb: So why did it work in other nodes and not this one? | 02:55 |
| clarkb | krotscheck: index was up to date at the time | 02:55 |
| clarkb | ? | 02:56 |
| clarkb | ou should always apt-get update propr to doing an install if you want the install to succeed | 02:56 |
| *** thorst has joined #openstack-infra | 02:56 | |
| *** eliqiao has quit IRC | 03:01 | |
| *** openstackstatus has quit IRC | 03:01 | |
| *** zhurong_ has joined #openstack-infra | 03:02 | |
| *** zhurong has quit IRC | 03:02 | |
| *** xarses has joined #openstack-infra | 03:02 | |
| *** eliqiao has joined #openstack-infra | 03:02 | |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/project-config: Reordered builders in npm builds https://review.openstack.org/302501 | 03:03 |
| krotscheck | clarkb: You were right, order of operations error. ^^ | 03:03 |
| *** thorst has quit IRC | 03:04 | |
| *** tphummel has quit IRC | 03:04 | |
| *** jamesmcarthur has quit IRC | 03:05 | |
| fungi | oops, approved when i just meant to +2 | 03:05 |
| krotscheck | fungi: Wow, fast response :) | 03:05 |
| fungi | still needs another reviewer, but i agree | 03:05 |
| fungi | that should solve it | 03:05 |
| krotscheck | fungi Most of the horizon builds should now be running though, they're using a different job. | 03:06 |
| *** sdake_ has quit IRC | 03:06 | |
| krotscheck | At the moment, I believe only the ironic webclient and eslint-config-openstack are still impacted | 03:06 |
| fungi | finally caught up on irc and e-mail for the day, so about to drop off again | 03:06 |
| fungi | clarkb: sorry again for disappearing right before you discovered dib bug of doom :( | 03:07 |
| *** sdake has joined #openstack-infra | 03:07 | |
| clarkb | just 7 more image uploads to go! | 03:07 |
| fungi | i too did not leave myself ssh'd into nodepool.o.o, so couldn't have helped on that recovery option | 03:08 |
| clarkb | they are all queued I will probably drop off myself and get back to it tomorrow | 03:08 |
| krotscheck | Fastest code review ever. I should submit more patches around 8PM :) | 03:08 |
| fungi | i'll be around and solid at a computer again at or before 1800 utc, so can help with the nodepool switchover if it starts sometime after that | 03:09 |
| *** achanda has quit IRC | 03:09 | |
| fungi | driving home a good chunk of tomorrow so will not likely be online much prior to that though | 03:10 |
| fungi | night all! | 03:10 |
| clarkb | I have an erand I have to run prior to 1030 local time tomorrow so nodepool things may start later than initially anticipated | 03:10 |
| fungi | works fine for me in that case | 03:12 |
| fungi | happy to pitch in when the time comes | 03:13 |
| *** sdake has quit IRC | 03:17 | |
| *** sdake has joined #openstack-infra | 03:18 | |
| *** asselin has quit IRC | 03:19 | |
| *** yamamoto_ has quit IRC | 03:20 | |
| *** asselin has joined #openstack-infra | 03:20 | |
| *** prometheanfire has quit IRC | 03:22 | |
| *** prometheanfire has joined #openstack-infra | 03:23 | |
| openstackgerrit | Merged openstack-infra/project-config: Reordered builders in npm builds https://review.openstack.org/302501 | 03:25 |
| *** jmccrory has quit IRC | 03:26 | |
| *** ddieterly[away] has quit IRC | 03:26 | |
| *** jmccrory has joined #openstack-infra | 03:26 | |
| *** kengo_sa_ has joined #openstack-infra | 03:28 | |
| *** kengo_sa_ has quit IRC | 03:28 | |
| *** sdake has quit IRC | 03:28 | |
| *** kengo_sakai_mba has joined #openstack-infra | 03:28 | |
| *** piet has quit IRC | 03:29 | |
| *** kengo_sakai_mba has quit IRC | 03:30 | |
| *** kengo_sakai_mba has joined #openstack-infra | 03:30 | |
| *** gildub has joined #openstack-infra | 03:32 | |
| *** tiswanso has quit IRC | 03:32 | |
| *** dimtruck is now known as zz_dimtruck | 03:32 | |
| *** kengo_sakai_mba has quit IRC | 03:34 | |
| *** piet has joined #openstack-infra | 03:34 | |
| *** ramishra has joined #openstack-infra | 03:37 | |
| *** fitoduarte has joined #openstack-infra | 03:41 | |
| *** FallenPegasus has joined #openstack-infra | 03:42 | |
| *** shashank_hegde has quit IRC | 03:42 | |
| *** mylu has quit IRC | 03:45 | |
| *** keedya has quit IRC | 03:47 | |
| *** salv-orlando has joined #openstack-infra | 03:50 | |
| *** Daviey has quit IRC | 03:50 | |
| *** Daviey has joined #openstack-infra | 03:50 | |
| *** flepied1 has quit IRC | 03:50 | |
| *** flepied has joined #openstack-infra | 03:51 | |
| *** salv-orlando has quit IRC | 03:52 | |
| *** grassass has quit IRC | 03:52 | |
| *** salv-orlando has joined #openstack-infra | 03:53 | |
| *** piet has quit IRC | 03:56 | |
| *** julim has quit IRC | 03:57 | |
| *** salv-orlando has quit IRC | 03:58 | |
| *** jamespage has quit IRC | 04:00 | |
| *** gnuoy has quit IRC | 04:00 | |
| *** Nikolay_St has quit IRC | 04:00 | |
| *** gnuoy has joined #openstack-infra | 04:01 | |
| *** jamespage has joined #openstack-infra | 04:01 | |
| *** FallenPegasus has quit IRC | 04:01 | |
| *** thorst has joined #openstack-infra | 04:01 | |
| *** mylu has joined #openstack-infra | 04:02 | |
| *** fedexo has joined #openstack-infra | 04:02 | |
| *** yaume has joined #openstack-infra | 04:03 | |
| *** thorst has quit IRC | 04:09 | |
| *** yamamoto_ has joined #openstack-infra | 04:10 | |
| *** Sukhdev has joined #openstack-infra | 04:11 | |
| *** shu-mutou has joined #openstack-infra | 04:11 | |
| *** links has joined #openstack-infra | 04:12 | |
| *** jcooley_ has quit IRC | 04:15 | |
| *** asettle has quit IRC | 04:17 | |
| *** asettle has joined #openstack-infra | 04:18 | |
| *** yfried has joined #openstack-infra | 04:20 | |
| *** amotoki has joined #openstack-infra | 04:20 | |
| *** shashank_hegde has joined #openstack-infra | 04:22 | |
| *** achanda has joined #openstack-infra | 04:22 | |
| *** maishsk has quit IRC | 04:26 | |
| *** persia has quit IRC | 04:31 | |
| *** persia has joined #openstack-infra | 04:33 | |
| shu-mutou | Hi everyone, Error occurred at python jobs (docs, pep8 and py27) in Magnum-UI. According to logs, pbr seems to fail to parse upper-constraints.txt. https://review.openstack.org/#/c/302389/ | 04:36 |
| shu-mutou | everyone: Does following patch cause the error? https://review.openstack.org/#/c/300935/ | 04:37 |
| *** achanda has quit IRC | 04:37 | |
| *** yaume has quit IRC | 04:40 | |
| *** eliqiao_ has joined #openstack-infra | 04:43 | |
| *** eliqiao has quit IRC | 04:44 | |
| *** mylu has quit IRC | 04:45 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder: Fix disk usage report https://review.openstack.org/302509 | 04:47 |
| *** xyang1 has quit IRC | 04:48 | |
| *** salv-orlando has joined #openstack-infra | 04:57 | |
| *** armax has quit IRC | 04:59 | |
| *** abregman has joined #openstack-infra | 05:00 | |
| *** yfried has quit IRC | 05:01 | |
| *** salv-orlando has quit IRC | 05:01 | |
| *** kengo_sakai_mba has joined #openstack-infra | 05:02 | |
| *** asettle has quit IRC | 05:02 | |
| *** oanson has joined #openstack-infra | 05:03 | |
| *** asettle has joined #openstack-infra | 05:03 | |
| *** rossella_s has quit IRC | 05:03 | |
| *** achanda has joined #openstack-infra | 05:03 | |
| *** rossella_s has joined #openstack-infra | 05:03 | |
| *** kushal has quit IRC | 05:04 | |
| *** thorst has joined #openstack-infra | 05:06 | |
| *** salv-orlando has joined #openstack-infra | 05:11 | |
| *** jaosorior has joined #openstack-infra | 05:13 | |
| openstackgerrit | xiangxinyong proposed openstack/requirements: Add python-smaugclient into requirements https://review.openstack.org/302512 | 05:13 |
| *** thorst has quit IRC | 05:13 | |
| *** ramishra has quit IRC | 05:16 | |
| *** harlowja_at_home has joined #openstack-infra | 05:17 | |
| *** kushal has joined #openstack-infra | 05:17 | |
| *** nwkarsten has joined #openstack-infra | 05:19 | |
| openstackgerrit | Masayuki Igawa proposed openstack-infra/subunit2sql: Fix lock range, again https://review.openstack.org/302515 | 05:23 |
| *** nwkarsten has quit IRC | 05:24 | |
| sc` | pabelanger: worked around the pip uninstall in my bootstrap script. my first successful integration build post devstack-centos7 migration just completed. i intend to spend a few cycles looking at chef's infra bits and seeing how they can be improved | 05:24 |
| *** shu-mutou is now known as shu-mutou-AFK | 05:26 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder: Fix disk usage report https://review.openstack.org/302509 | 05:27 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder: Skip gentoo test https://review.openstack.org/302516 | 05:27 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder: Add qcow2 generation for better test coverage https://review.openstack.org/302517 | 05:27 |
| *** flepied has quit IRC | 05:33 | |
| *** achanda has quit IRC | 05:33 | |
| openstackgerrit | Joshua Harlow proposed openstack/requirements: Bump voluptuous to 0.8.9 https://review.openstack.org/291383 | 05:34 |
| *** Daisy has joined #openstack-infra | 05:39 | |
| *** fedexo has quit IRC | 05:39 | |
| *** Sukhdev has quit IRC | 05:40 | |
| *** fedexo has joined #openstack-infra | 05:42 | |
| *** rcernin has joined #openstack-infra | 05:45 | |
| openstackgerrit | Doug Hellmann proposed openstack-infra/project-config: clean up sahara-extra release jobs https://review.openstack.org/302520 | 05:45 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder: Fix disk usage report https://review.openstack.org/302509 | 05:50 |
| *** ybathia has joined #openstack-infra | 05:50 | |
| *** yfried has joined #openstack-infra | 05:53 | |
| *** jcooley_ has joined #openstack-infra | 05:54 | |
| AJaeger | shu-mutou-AFK: error is "git history requires a target version of pbr.version.SemanticVersion(1.0.1), but target version is pbr.version.SemanticVersion(0.0.1)" | 05:55 |
| openstackgerrit | Merged openstack-infra/irc-meetings: Update the Glare team meeting chair https://review.openstack.org/301263 | 05:55 |
| AJaeger | shu-mutou-AFK: https://review.openstack.org/302524 | 05:56 |
| *** achanda has joined #openstack-infra | 05:59 | |
| *** aeng has quit IRC | 05:59 | |
| *** cody-somerville has quit IRC | 06:02 | |
| AJaeger | wow, new zuul queue length record AFAIK: "Queue lengths: 80242 events, 51 results. " | 06:05 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder: Fix disk usage report https://review.openstack.org/302509 | 06:05 |
| *** Nikolay_St has joined #openstack-infra | 06:07 | |
| *** Hal has joined #openstack-infra | 06:07 | |
| *** Hal is now known as Guest71611 | 06:08 | |
| *** mikelk has joined #openstack-infra | 06:10 | |
| *** thorst has joined #openstack-infra | 06:11 | |
| *** harlowja_at_home has quit IRC | 06:12 | |
| *** kushal has quit IRC | 06:12 | |
| *** cody-somerville has joined #openstack-infra | 06:17 | |
| AJaeger | infra-root, I cannot login https://translate.openstack.org/ . Do we have a problem with openstackid again? | 06:18 |
| *** thorst has quit IRC | 06:18 | |
| *** kengo_sakai_mba has quit IRC | 06:19 | |
| AJaeger | infra-root: Didn't fungi move openstackid to a new server? Is there perhaps an old IP address or something like that in use? | 06:19 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack-infra/project-config: Normalize projects.yaml https://review.openstack.org/302529 | 06:20 |
| *** asettle has quit IRC | 06:20 | |
| *** achanda has quit IRC | 06:22 | |
| *** scheuran has joined #openstack-infra | 06:22 | |
| yolanda | AJaeger, will take a quick look. I need to leave for doctors the whole morning, but I'm still here | 06:23 |
| yolanda | AJaeger, what's the problem you are seeing? | 06:24 |
| *** xiangxinyong has joined #openstack-infra | 06:25 | |
| *** shashank_hegde has quit IRC | 06:25 | |
| *** pcaruana has joined #openstack-infra | 06:26 | |
| yolanda | i just logged successfully | 06:26 |
| *** kushal has joined #openstack-infra | 06:26 | |
| AJaeger | yolanda: no login possible at all, clicking on login button does not do anything | 06:26 |
| *** akaszuba has joined #openstack-infra | 06:26 | |
| yolanda | i am logged at translate now | 06:26 |
| AJaeger | "This site can’t be reached | 06:26 |
| AJaeger | openstackid.org took too long to respond. | 06:26 |
| AJaeger | Ok, then we have a connectivity problem ;( | 06:26 |
| yolanda | can you ping to the server? which ip is giving you? | 06:27 |
| * AJaeger will try later then. | 06:27 | |
| openstackgerrit | Doug Hellmann proposed openstack-infra/project-config: clean up sahara-extra release jobs https://review.openstack.org/302520 | 06:27 |
| *** fedexo has quit IRC | 06:27 | |
| AJaeger | IP is 104.130.154.113 | 06:27 |
| AJaeger | ping doesn't work for me... | 06:27 |
| yolanda | when i ping i have 23.253.243.97 | 06:28 |
| yolanda | let me double check, it may be related to server mov | 06:28 |
| * AJaeger checks other system | 06:28 | |
| AJaeger | yes, 23... is correct. | 06:28 |
| *** eliqiao_ is now known as eliqiao | 06:28 | |
| * AJaeger has an old IP address in /etc/hosts ;( | 06:29 | |
| AJaeger | yolanda: user error on my side -thanks | 06:29 |
| yolanda | ouch | 06:29 |
| * AJaeger removed the IP address now | 06:29 | |
| yolanda | cool, if all problems were like those :) | 06:29 |
| AJaeger | ;) | 06:30 |
| AJaeger | thanks for the help, the IP hint was essential | 06:30 |
| AJaeger | everything fine now, I'm in - yolanda all the best for your day! | 06:30 |
| *** flepied has joined #openstack-infra | 06:31 | |
| *** Thelo has quit IRC | 06:31 | |
| *** asettle has joined #openstack-infra | 06:33 | |
| *** mrmartin has joined #openstack-infra | 06:34 | |
| yolanda | thanks AJaeger, i'll be back on early afternoon | 06:34 |
| * AJaeger will try not to break anything ;) | 06:35 | |
| *** asettle has quit IRC | 06:37 | |
| *** ramishra has joined #openstack-infra | 06:39 | |
| *** ybathia has quit IRC | 06:41 | |
| openstackgerrit | xiangxinyong proposed openstack/requirements: Add python-smaugclient into requirements https://review.openstack.org/302512 | 06:41 |
| *** tqtran has joined #openstack-infra | 06:43 | |
| *** tesseract has joined #openstack-infra | 06:45 | |
| *** tesseract is now known as Guest12876 | 06:45 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/requirements: Updated from generate-constraints https://review.openstack.org/302539 | 06:46 |
| *** vgridnev has joined #openstack-infra | 06:46 | |
| *** tqtran has quit IRC | 06:47 | |
| *** camunoz has quit IRC | 06:48 | |
| *** yaume has joined #openstack-infra | 06:51 | |
| *** salv-orlando has quit IRC | 06:52 | |
| *** jcooley_ has quit IRC | 06:56 | |
| *** woodster_ has quit IRC | 06:57 | |
| *** ramineni_ has joined #openstack-infra | 06:57 | |
| ramineni_ | AJaeger: hi | 06:59 |
| openstackgerrit | Merged openstack-infra/project-config: Normalize projects.yaml https://review.openstack.org/302529 | 06:59 |
| ramineni_ | i have added a job to publish the docs to docs.openstack , I can see the docs also http://docs.openstack.org/developer/congress/ , but it is not added here http://docs.openstack.org/developer/openstack-projects.html , under openstack-services | 07:01 |
| AJaeger | ramineni_: That page needs to be edited manually. | 07:01 |
| AJaeger | It lives in openstack-manuals/www/developer/openstack-projects.html. Please send a change request... | 07:02 |
| ramineni_ | AJaeger: oohk , thanks a lot for clarifying :) | 07:02 |
| *** maishsk has joined #openstack-infra | 07:04 | |
| *** placko_ has joined #openstack-infra | 07:05 | |
| *** Nikolay_St has quit IRC | 07:07 | |
| *** mrmartin has quit IRC | 07:08 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder: [WIP] Fix disk usage report https://review.openstack.org/302509 | 07:09 |
| *** nmagnezi has joined #openstack-infra | 07:11 | |
| *** ramishra has quit IRC | 07:11 | |
| *** ihrachys has joined #openstack-infra | 07:11 | |
| *** jlanoux has joined #openstack-infra | 07:13 | |
| *** yaume has quit IRC | 07:15 | |
| *** tosky has joined #openstack-infra | 07:16 | |
| *** thorst has joined #openstack-infra | 07:16 | |
| *** arxcruz has joined #openstack-infra | 07:17 | |
| openstackgerrit | Thierry Carrez proposed openstack-infra/project-config: Give project teams access to approve stable/mitaka changes https://review.openstack.org/300667 | 07:19 |
| *** ramineni_ has quit IRC | 07:19 | |
| *** kengo_sakai_mba has joined #openstack-infra | 07:19 | |
| openstackgerrit | vikram.choudhary proposed openstack-infra/project-config: Adding neutron-dynamic-routing as neutron's stadium project. https://review.openstack.org/268726 | 07:19 |
| *** vikram has joined #openstack-infra | 07:20 | |
| vikram | AJaeger: Can you please bless https://review.openstack.org/#/c/302120/ | 07:21 |
| *** rlandy has quit IRC | 07:23 | |
| *** thorst has quit IRC | 07:24 | |
| *** vincentll has joined #openstack-infra | 07:24 | |
| *** Daisy has quit IRC | 07:24 | |
| *** Daisy has joined #openstack-infra | 07:25 | |
| AJaeger | vikram: I'll review it in the next days for sure. With release day today I'm only pushing critical stuff through. | 07:25 |
| vikram | AJaeger: Thanks.. np | 07:25 |
| vikram | AJaeger: BTW, thanks for your pointer on the repo split.. It helped | 07:25 |
| AJaeger | great! | 07:26 |
| *** ramishra has joined #openstack-infra | 07:27 | |
| *** kengo_sakai_mba has quit IRC | 07:28 | |
| *** matrohon has joined #openstack-infra | 07:29 | |
| *** Daisy has quit IRC | 07:29 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder: Fix disk usage report https://review.openstack.org/302509 | 07:29 |
| jaosorior | Hey guys, I just saw a commit to barbican with the title: "Merge tag '2.0.0'" ... it says it shouldn't contain any changes, however there is a change that removes the default branch from the .gitreview file. Is this alright? https://review.openstack.org/#/c/302555/1 | 07:29 |
| AJaeger | jaosorior: ask on #openstack-release - and this change is perfectly fine | 07:31 |
| *** dizquierdo has joined #openstack-infra | 07:31 | |
| *** daemontool has joined #openstack-infra | 07:32 | |
| AJaeger | jaosorior: gerrit just doesn't get it ;) | 07:32 |
| *** yfried_ has joined #openstack-infra | 07:32 | |
| *** yfried has quit IRC | 07:32 | |
| jaosorior | AJaeger: alright, thanks! though I guess the commit message should be changed in the bot, since it explicitly tells people to ask people in #openstack-infra | 07:33 |
| *** ramishra has quit IRC | 07:36 | |
| *** dtantsur|afk is now known as dtantsur | 07:36 | |
| *** mrmartin has joined #openstack-infra | 07:36 | |
| *** ramishra has joined #openstack-infra | 07:39 | |
| *** flepied1 has joined #openstack-infra | 07:41 | |
| *** asettle has joined #openstack-infra | 07:42 | |
| AJaeger | dhellmann, ttx ^ | 07:43 |
| *** flepied has quit IRC | 07:43 | |
| *** cody-somerville has quit IRC | 07:45 | |
| *** jamielennox is now known as jamielennox|away | 07:45 | |
| ttx | jaosorior: yes, that's a commit that Gerrit has trouble representing | 07:45 |
| *** Daisy has joined #openstack-infra | 07:45 | |
| ttx | It's about merging tags from release branches back into master | 07:45 |
| *** asettle has quit IRC | 07:45 | |
| ttx | It should be approved. I think that's documented somewhere. | 07:46 |
| * ttx looks | 07:46 | |
| *** shu-mutou-AFK is now known as shu-mutou | 07:46 | |
| shu-mutou | AJaeger: Thank you for your help!!! | 07:47 |
| *** jaosorior has quit IRC | 07:47 | |
| *** shardy has joined #openstack-infra | 07:47 | |
| ttx | hmm, can't find doc | 07:47 |
| *** jaosorior has joined #openstack-infra | 07:47 | |
| *** yaume has joined #openstack-infra | 07:47 | |
| jaosorior | ttx: I see | 07:48 |
| *** zeih_ has joined #openstack-infra | 07:51 | |
| *** tosky has quit IRC | 07:51 | |
| *** matrohon_ has joined #openstack-infra | 07:51 | |
| *** tosky has joined #openstack-infra | 07:51 | |
| *** matrohon_ has quit IRC | 07:51 | |
| ttx | jaosorior: found ref, let me link | 07:54 |
| ttx | http://lists.openstack.org/pipermail/openstack-dev/2015-June/066294.html | 07:54 |
| *** salv-orlando has joined #openstack-infra | 07:54 | |
| *** ramishra has quit IRC | 07:55 | |
| jaosorior | I see | 07:55 |
| jaosorior | I'll approve it now then | 07:55 |
| jaosorior | thanks for the info ttx | 07:55 |
| robcresswell | Glad I got caught the tail end of that, I was also a little confused. | 07:57 |
| *** yfried_ has quit IRC | 07:58 | |
| *** HeOS has joined #openstack-infra | 08:00 | |
| *** zeih_ has quit IRC | 08:01 | |
| *** vgridnev has quit IRC | 08:02 | |
| *** yolanda has quit IRC | 08:03 | |
| *** sflanigan has quit IRC | 08:04 | |
| *** jordanP has joined #openstack-infra | 08:06 | |
| *** Nikolay_St has joined #openstack-infra | 08:08 | |
| *** zeih_ has joined #openstack-infra | 08:09 | |
| *** salv-orlando has quit IRC | 08:10 | |
| *** zeih_ has quit IRC | 08:14 | |
| *** ramineni_ has joined #openstack-infra | 08:15 | |
| *** ramishra has joined #openstack-infra | 08:15 | |
| AJaeger | infra-root, the topic can be cleared now I guess - could anybody issue a #status ok, please? | 08:15 |
| *** Jeffrey4l has quit IRC | 08:16 | |
| *** ramineni_ has quit IRC | 08:16 | |
| *** ramineni_ has joined #openstack-infra | 08:17 | |
| jhesketh | AJaeger: what was the resolution? | 08:18 |
| AJaeger | jhesketh: update npm locally, let me grab you the change... | 08:20 |
| AJaeger | https://review.openstack.org/302501 | 08:20 |
| AJaeger | and https://review.openstack.org/302501 | 08:20 |
| AJaeger | https://review.openstack.org/302501 | 08:21 |
| * AJaeger can't copy and paste today ;( | 08:21 | |
| AJaeger | https://review.openstack.org/302292 is the one I want | 08:21 |
| AJaeger | That's the real change, the other is a fix | 08:21 |
| *** thorst has joined #openstack-infra | 08:21 | |
| jhesketh | AJaeger: thanks | 08:21 |
| jhesketh | AJaeger: I assume those jobs are passing? | 08:22 |
| *** Jeffrey4l has joined #openstack-infra | 08:22 | |
| AJaeger | jhesketh: reading backscoll and checking api-site and searchlight-ui: Yes | 08:23 |
| jhesketh | cool, thanks | 08:23 |
| jhesketh | (sorry thought you may have known off the top of your head, otherwise I Could have checked) | 08:23 |
| jhesketh | #status ok jobs depending on npm are now working again | 08:24 |
| jhesketh | hmm, where's the status bot | 08:24 |
| *** salv-orlando has joined #openstack-infra | 08:26 | |
| *** ramishra has quit IRC | 08:26 | |
| * jhesketh prods the bot | 08:28 | |
| *** openstackstatus has joined #openstack-infra | 08:29 | |
| *** ChanServ sets mode: +v openstackstatus | 08:29 | |
| *** thorst has quit IRC | 08:30 | |
| *** jistr has joined #openstack-infra | 08:30 | |
| jhesketh | #status ok jobs depending on npm are now working again | 08:30 |
| openstackstatus | jhesketh: sending ok | 08:30 |
| AJaeger | thanks, jhesketh ! | 08:32 |
| *** jcoufal has joined #openstack-infra | 08:32 | |
| -openstackstatus- NOTICE: jobs depending on npm are now working again | 08:33 | |
| jhesketh | AJaeger: hmm, but I think restarting the bot was a mistake... it's lost its memory of the channel topics | 08:33 |
| *** esikachev has joined #openstack-infra | 08:33 | |
| jhesketh | I don't think I had a choice though as it wasn't logging anything for the last 6 or so hours and appeared to be hung | 08:33 |
| jhesketh | (I should have straced and poked a little deeper, but didn't think restarting would be harmful) | 08:34 |
| AJaeger | OOps;( No idea on the bot | 08:34 |
| jhesketh | I'll send a notice after this for channel ops to reset their own topics | 08:34 |
| *** ramineni_ has quit IRC | 08:34 | |
| AJaeger | can you reset ours as well, please? | 08:35 |
| tosky | and the -meeting- ones | 08:36 |
| openstackstatus | jhesketh: finished sending ok | 08:36 |
| zigo | AJaeger: Hi there! Can I ask you for an advice? | 08:36 |
| zigo | AJaeger: We have the issue that "gbp buildpackage" fails because the $WORKSPACE git repository is detached from HEAD, so git-buildpackage doesn't know in which branch it's running. | 08:37 |
| openstackgerrit | Davanum Srinivas (dims) proposed openstack/requirements: Updated from generate-constraints https://review.openstack.org/302539 | 08:37 |
| zigo | AJaeger: Can we checkout a branch or something? How can we fix? | 08:37 |
| *** dtantsur is now known as dtantsur|brb | 08:38 | |
| tobiash | jhesketh: I just observed a strange behaviour in our zuul | 08:39 |
| AJaeger | zigo, there are environment variables you can use like ZUUL_REFNAME | 08:39 |
| zigo | AJaeger: I should use it to do what? How? I'm not sure I get it here ... | 08:40 |
| AJaeger | zigo: check your job in jenkins and see the environment variable settings for it. | 08:40 |
| zigo | AJaeger: I do know the branch name, it's referenced in debian/gbp.conf, that's not what I need. | 08:40 |
| zigo | AJaeger: I need the git repo to not be detached from HEAD. | 08:40 |
| zigo | AJaeger: Can I do a "git checkout debian/mitaka"? Will that work? | 08:41 |
| *** daemontool has quit IRC | 08:41 | |
| tobiash | I have pipelines check, test, etc. where test is manually triggered by comment. My project only defines builds for the test pipeline, but every patch enters the check pipeline triggering the zuul-merger and no builds and no voring | 08:41 |
| AJaeger | zigo, you need the git repo with the change you're testing applied to it. | 08:41 |
| AJaeger | And that's the detached state you get... | 08:41 |
| zigo | AJaeger: But I need the git repo to be on a branch name, otherwise git-buildpackage gets very much confused... | 08:42 |
| AJaeger | zigo: Now I'm confused - and don't have time to dig into this with you. | 08:42 |
| zigo | AJaeger: The change isn't commited to git, is it? | 08:42 |
| zigo | nv mind | 08:43 |
| *** yolanda has joined #openstack-infra | 08:43 | |
| *** ramishra has joined #openstack-infra | 08:45 | |
| *** yolanda has quit IRC | 08:49 | |
| *** ramineni_ has joined #openstack-infra | 08:50 | |
| *** cdent has joined #openstack-infra | 08:50 | |
| *** yolanda has joined #openstack-infra | 08:50 | |
| *** e0ne has joined #openstack-infra | 08:50 | |
| *** shu-mutou is now known as shu-mutou-AFK | 08:51 | |
| *** salv-orl_ has joined #openstack-infra | 08:52 | |
| *** salv-orlando has quit IRC | 08:55 | |
| *** jaosorior has quit IRC | 08:59 | |
| *** jaosorior has joined #openstack-infra | 09:00 | |
| zigo | AJaeger: FYI, looks like I fixed the issues, though now I need to log off and re-login to get sbuild working... :/ | 09:02 |
| openstackgerrit | Kiall Mac Innes proposed openstack-infra/project-config: Add new jobs to openstack/designate-tempest-plugin https://review.openstack.org/302606 | 09:02 |
| *** rossella_s has quit IRC | 09:03 | |
| *** rossella_s has joined #openstack-infra | 09:03 | |
| dims | Anyone here know the pypi mirror setup? need to tickle it to pickup latest reno release | 09:04 |
| *** nijaba has quit IRC | 09:05 | |
| *** sambetts|afk is now known as sambetts | 09:05 | |
| *** nijaba has joined #openstack-infra | 09:07 | |
| *** ifarkas has joined #openstack-infra | 09:07 | |
| *** derekh has joined #openstack-infra | 09:09 | |
| *** zhurong_ has quit IRC | 09:09 | |
| openstackgerrit | Thomas Goirand proposed openstack-infra/project-config: Execute sbuild after relogin https://review.openstack.org/302610 | 09:10 |
| *** ramineni_ has quit IRC | 09:10 | |
| zigo | AJaeger: https://review.openstack.org/#/c/302610/ <--- This should fix the last issue I found. | 09:11 |
| zigo | (no worries if you're busy right now...) | 09:12 |
| openstackgerrit | Andreas Jaeger proposed openstack/requirements: Require reno 1.6.2 or newer https://review.openstack.org/302612 | 09:12 |
| *** ChanServ changes topic to "Discussion of OpenStack Developer and Community Infrastructure | docs http://docs.openstack.org/infra/ | bugs https://storyboard.openstack.org/ | source https://git.openstack.org/cgit/openstack-infra/ | channel logs http://eavesdrop.openstack.org/irclogs/%23openstack-infra/" | 09:12 | |
| AJaeger | zigo: please do not add me to reviews, I'll review via gerrit/gertty and not via emails | 09:14 |
| *** vgridnev has joined #openstack-infra | 09:17 | |
| openstackgerrit | Kiall Mac Innes proposed openstack-infra/project-config: Add new jobs to openstack/designate-tempest-plugin https://review.openstack.org/302606 | 09:20 |
| *** hichihara has quit IRC | 09:23 | |
| *** kzaitsev_mb has joined #openstack-infra | 09:23 | |
| *** thorst has joined #openstack-infra | 09:27 | |
| *** ramishra has quit IRC | 09:28 | |
| *** ramineni_ has joined #openstack-infra | 09:29 | |
| *** ramishra has joined #openstack-infra | 09:30 | |
| openstackgerrit | Dong Ma proposed openstack-infra/subunit2sql: Add more db api tests https://review.openstack.org/302619 | 09:31 |
| vkramskikh | hey folks, does anybody know if it's decided to stay with node v4 on jenkins workers or node will be reverted back to 0.10 after fix of https://github.com/npm/npm/issues/12196 ? | 09:31 |
| *** links has quit IRC | 09:31 | |
| *** alex_xu has quit IRC | 09:32 | |
| AJaeger | vkramskikh: once we go to xenial, we will have newer version for sure. Best discuss with krotscheck later | 09:34 |
| *** thorst has quit IRC | 09:34 | |
| *** alex_xu has joined #openstack-infra | 09:36 | |
| *** nadya has joined #openstack-infra | 09:36 | |
| *** yamamoto_ has quit IRC | 09:40 | |
| *** Daisy has quit IRC | 09:40 | |
| vkramskikh | thanks | 09:41 |
| vkramskikh | and when is switch to xenial planned? | 09:42 |
| AJaeger | vkramskikh: in a couple of months | 09:42 |
| AJaeger | it'S not released yet but we've started testing already | 09:42 |
| *** links has joined #openstack-infra | 09:45 | |
| eliqiao | hi there, I have trouble while passing Jenkins gate. the thing is: I can pass Jenkins check, but faied to pas Jenkins gate with same job. Is there any sepcify word like 'reverify'/'recheck' I can use to kick infra to only run Jenkins gate? | 09:48 |
| openstackgerrit | Colleen Murphy proposed openstack-infra/puppet-pip: Add new pip provider to use pypi mirrors https://review.openstack.org/302396 | 09:48 |
| eliqiao | I tried 'reverify', infra will trigger Jenkins check too | 09:48 |
| AJaeger | eliqiao: no, it always goes back to check queue | 09:49 |
| AJaeger | eliqiao: so, there's some nondeterminism in the job/change/repo - let's investigate first and then decide what to do | 09:50 |
| eliqiao | AJaeger: okay, but it's really time costing. | 09:50 |
| AJaeger | eliqiao: if you enqueue again the gate and it fails again, it's much more time costing | 09:51 |
| AJaeger | IT could fail in the gate due to changes that merged before or newer packages - so we first run in check to ensure that everything is fine. | 09:51 |
| openstackgerrit | Marton Kiss proposed openstack-infra/groups: Fix strict warning on user group page https://review.openstack.org/302634 | 09:51 |
| eliqiao | AJaeger: I get it now. Make sense. | 09:52 |
| *** kzaitsev_mb has quit IRC | 09:52 | |
| *** gildub has quit IRC | 09:52 | |
| eliqiao | AJaeger: can I ask a question. I see some patches (only has doc changes) only run specify jobs but other run all jobs, do you konw how to configure it? for example https://review.openstack.org/#/c/302589/ | 09:52 |
| AJaeger | eliqiao: all done in zuul/layout.yaml - but that is complicated and has some pitfalls. | 09:54 |
| AJaeger | eliqiao: lunch time now, will be back later... | 09:54 |
| eliqiao | AJaeger: thanks, have good lunch. | 09:54 |
| *** dtantsur|brb is now known as dtantsur | 09:55 | |
| *** achanda has joined #openstack-infra | 09:56 | |
| *** k4n0 has joined #openstack-infra | 09:57 | |
| *** kzaitsev_mb has joined #openstack-infra | 09:59 | |
| frickler | there's still lots of channels with broken status lines | 09:59 |
| *** k4n0 has quit IRC | 10:00 | |
| *** obondarev has quit IRC | 10:00 | |
| frickler | somehow statusbot did not reset them properly? | 10:01 |
| *** ramineni_1 has joined #openstack-infra | 10:01 | |
| frickler | ah, just found it in the backlog, sorry for disturbing ;) | 10:02 |
| *** achanda has quit IRC | 10:02 | |
| *** ramineni_ has quit IRC | 10:04 | |
| *** ociuhandu has quit IRC | 10:05 | |
| *** obondarev has joined #openstack-infra | 10:06 | |
| *** obondarev has quit IRC | 10:12 | |
| *** kzaitsev_mb has quit IRC | 10:12 | |
| *** Qiming has quit IRC | 10:13 | |
| mrmartin | hi | 10:13 |
| mrmartin | what is wrong here: http://logs.openstack.org/34/302634/1/check/gate-groups-unittests/715cfb9/console.html ? bindep error as: "ERROR: These requested packages were not installed" | 10:13 |
| *** Kiall has quit IRC | 10:14 | |
| *** Kiall has joined #openstack-infra | 10:14 | |
| *** kzaitsev_mb has joined #openstack-infra | 10:15 | |
| *** cdent has quit IRC | 10:16 | |
| vkramskikh | could anybody please help with review https://review.openstack.org/#/c/301548/ ? there is a stable/mitaka branch created for fuel-ui repo, but fuel-ui-core group can't merge into that branch | 10:17 |
| *** esikachev has quit IRC | 10:17 | |
| *** mrmartin has quit IRC | 10:18 | |
| *** vtapia_ has quit IRC | 10:19 | |
| openstackgerrit | Vitaly Gridnev proposed openstack-infra/project-config: [sahara] setup additional jobs for sahara-dashboard https://review.openstack.org/302647 | 10:20 |
| *** sdague has joined #openstack-infra | 10:20 | |
| *** vtapia_ has joined #openstack-infra | 10:21 | |
| dims | fungi : good morning, please join us when you get in :) #openstack-release | 10:22 |
| *** daemontool has joined #openstack-infra | 10:22 | |
| *** yaume_ has joined #openstack-infra | 10:24 | |
| *** redixin has joined #openstack-infra | 10:24 | |
| *** yaume has quit IRC | 10:27 | |
| *** kmartin has quit IRC | 10:28 | |
| redixin | Hi all. *.css files are text/css right? So http servers should set Content-Type header to "text/css" right? | 10:29 |
| redixin | Or I missed something? Like log.openstack.org should serve .css files as "text/x-asm" for some reason? | 10:31 |
| *** obondarev has joined #openstack-infra | 10:31 | |
| *** thorst has joined #openstack-infra | 10:32 | |
| *** _degorenko|afk is now known as degorenko | 10:32 | |
| *** thorst has quit IRC | 10:39 | |
| *** links has quit IRC | 10:39 | |
| openstackgerrit | Vitaly Gridnev proposed openstack-infra/project-config: [sahara] setup additional jobs for sahara-dashboard https://review.openstack.org/302647 | 10:41 |
| *** yamamoto has joined #openstack-infra | 10:42 | |
| openstackgerrit | Merged openstack/requirements: Bump gnocchiclient version to 2.2.0 https://review.openstack.org/299770 | 10:42 |
| *** nwkarsten has joined #openstack-infra | 10:43 | |
| *** abregman has quit IRC | 10:44 | |
| *** abregman has joined #openstack-infra | 10:44 | |
| *** xwizard has joined #openstack-infra | 10:44 | |
| *** Guest12876 is now known as tesseract | 10:47 | |
| *** tesseract is now known as Guest74036 | 10:48 | |
| *** BobBall-1WOL is now known as BobBall | 10:49 | |
| *** andymaier has joined #openstack-infra | 10:53 | |
| *** alex_xu has quit IRC | 10:55 | |
| *** links has joined #openstack-infra | 10:56 | |
| *** kengo_sakai_mba has joined #openstack-infra | 10:56 | |
| *** alex_xu has joined #openstack-infra | 10:57 | |
| *** flongo has joined #openstack-infra | 10:58 | |
| *** jlanoux has quit IRC | 10:58 | |
| *** achanda has joined #openstack-infra | 10:58 | |
| *** nadya has quit IRC | 10:59 | |
| *** maishsk has quit IRC | 11:01 | |
| *** jlanoux has joined #openstack-infra | 11:03 | |
| *** pblaho has joined #openstack-infra | 11:03 | |
| *** achanda has quit IRC | 11:03 | |
| *** maishsk has joined #openstack-infra | 11:03 | |
| *** yamamoto has quit IRC | 11:04 | |
| *** jlanoux_ has joined #openstack-infra | 11:05 | |
| *** yamamoto has joined #openstack-infra | 11:06 | |
| *** jlanoux has quit IRC | 11:07 | |
| *** thorst has joined #openstack-infra | 11:11 | |
| *** thorst_ has joined #openstack-infra | 11:11 | |
| *** cdent has joined #openstack-infra | 11:14 | |
| *** thorst has quit IRC | 11:15 | |
| *** kengo_sakai_mba has quit IRC | 11:15 | |
| *** gildub has joined #openstack-infra | 11:17 | |
| *** tpsilva has joined #openstack-infra | 11:22 | |
| *** Qiming has joined #openstack-infra | 11:28 | |
| *** ramineni_1 has left #openstack-infra | 11:29 | |
| openstackgerrit | Merged openstack-infra/tripleo-ci: Add execute permissions to promote.sh https://review.openstack.org/296363 | 11:31 |
| *** fhubik has joined #openstack-infra | 11:33 | |
| *** ldnunes has joined #openstack-infra | 11:35 | |
| *** vgridnev has quit IRC | 11:39 | |
| *** alex_xu has quit IRC | 11:41 | |
| *** nadya has joined #openstack-infra | 11:43 | |
| *** alex_xu has joined #openstack-infra | 11:43 | |
| openstackgerrit | Kiall Mac Innes proposed openstack-infra/project-config: Add designate-tempest-plugin to Designate DSVM jobs https://review.openstack.org/302689 | 11:44 |
| *** _amrith_ is now known as amrith | 11:44 | |
| Kiall | any project-config cores around? Hoping to land ^ quickly, as I'm gone on holidays soon and want to get everything ready + passing for core's to +A :) | 11:44 |
| AJaeger | Kiall: I think it's not sufficient, see my comment for a quick review | 11:46 |
| Kiall | AJaeger: our DSVM plugin is already enabled + used etc - Is that what you mean? | 11:47 |
| Kiall | https://github.com/openstack/designate/blob/master/devstack/gate/gate_hook.sh etc manages to suck it in | 11:47 |
| AJaeger | I miss a line like: | 11:47 |
| Kiall | (It's old and crufty and in need of some TLC to cleanup) | 11:47 |
| AJaeger | export DEVSTACK_LOCAL_CONFIG="enable_plugin zaqar git://git.openstack.org/openstack/zaqar" | 11:48 |
| AJaeger | in designate.yaml | 11:48 |
| AJaeger | right now you enable the designate one - not sure whether that works with your new plugin as well. | 11:48 |
| AJaeger | export DEVSTACK_LOCAL_CONFIG="enable_plugin designate git://git.openstack.org/openstack/designate" | 11:48 |
| AJaeger | That'S what you have - better ask sdague or somebody else about this. | 11:49 |
| Kiall | Oh, right - it's not needed for this. | 11:49 |
| * AJaeger doesn't feel confident to +2 that quickly | 11:49 | |
| Kiall | https://review.openstack.org/#/c/302677/ handles adding the tempest plugin | 11:49 |
| Kiall | enable_plugin is for devstack plugins, rather than tempest ones :) | 11:49 |
| AJaeger | Ah... | 11:50 |
| sdague | Kiall: seems fine | 11:50 |
| Kiall | thanks sdague :) Once we move our existing tests over, I plan to spend some time cleaning the organic mess that is the designate DSVM jobs up ;) | 11:51 |
| sdague | cool | 11:51 |
| *** mrmartin has joined #openstack-infra | 11:51 | |
| Kiall | All the plugin supports things have been gaining means less hacks needed to wedge designate in ;) | 11:52 |
| *** maishsk has quit IRC | 11:53 | |
| *** moravec has quit IRC | 11:53 | |
| *** keedya has joined #openstack-infra | 11:55 | |
| *** cdent has quit IRC | 11:55 | |
| *** gordc has joined #openstack-infra | 11:55 | |
| *** weshay has joined #openstack-infra | 11:56 | |
| *** maishsk has joined #openstack-infra | 11:57 | |
| *** cdent has joined #openstack-infra | 11:59 | |
| *** andymaier has quit IRC | 11:59 | |
| *** BobBall_ has joined #openstack-infra | 12:01 | |
| *** BobBall_ is now known as Guest30758 | 12:01 | |
| openstackgerrit | Merged openstack-infra/project-config: Add designate-tempest-plugin to Designate DSVM jobs https://review.openstack.org/302689 | 12:02 |
| *** dingyichen has quit IRC | 12:03 | |
| jhesketh | FYI I've reset all of the channels that statusbot manages by pulling out the status's from the logs | 12:03 |
| jhesketh | AJaeger: ^ | 12:03 |
| *** vgridnev has joined #openstack-infra | 12:03 | |
| *** aysyd has joined #openstack-infra | 12:04 | |
| *** vincentll has quit IRC | 12:04 | |
| *** trown|outtypewww is now known as trown | 12:04 | |
| *** BobBall has quit IRC | 12:04 | |
| *** vincentll has joined #openstack-infra | 12:05 | |
| *** wfoster is now known as wfoster_afk | 12:05 | |
| *** Guest30758 has quit IRC | 12:05 | |
| *** BobBall has joined #openstack-infra | 12:06 | |
| *** dingyichen has joined #openstack-infra | 12:06 | |
| *** BobBall is now known as Guest34177 | 12:06 | |
| *** jhesketh has quit IRC | 12:07 | |
| *** achanda has joined #openstack-infra | 12:07 | |
| *** jhesketh has joined #openstack-infra | 12:07 | |
| openstackgerrit | Nate Johnston proposed openstack-infra/irc-meetings: Add Nate Johnston as neutron-qos co-chair https://review.openstack.org/302706 | 12:10 |
| krotscheck | vkramskikh: I have no plans to move back at this point, unless there's a significant reason to do so. Node 4 is LTS, under support, and will remain so for at least another year, which node 012 is not. | 12:11 |
| *** kengo_sakai_mba has joined #openstack-infra | 12:12 | |
| *** baoli has joined #openstack-infra | 12:13 | |
| *** moravec has joined #openstack-infra | 12:13 | |
| *** baoli_ has joined #openstack-infra | 12:13 | |
| *** achanda has quit IRC | 12:15 | |
| *** moravec has quit IRC | 12:16 | |
| *** kengo_sakai_mba has quit IRC | 12:17 | |
| *** baoli has quit IRC | 12:18 | |
| *** dprince has joined #openstack-infra | 12:19 | |
| krotscheck | jtomasek: Yeah, let's chat here. Probably more relevant. | 12:20 |
| jtomasek | krotscheck: ok | 12:20 |
| krotscheck | First time running npm install it got Killed. Odd that | 12:21 |
| krotscheck | And again. | 12:21 |
| jtomasek | krotscheck: you mean npm install errors? | 12:21 |
| krotscheck | HrmYeah, need more memory on the instance, sec. | 12:22 |
| jtomasek | krotscheck: ah | 12:22 |
| *** edmondsw has joined #openstack-infra | 12:23 | |
| *** doug-fish has joined #openstack-infra | 12:26 | |
| *** akshai has joined #openstack-infra | 12:26 | |
| *** salv-orl_ has quit IRC | 12:28 | |
| *** |-paul-| has joined #openstack-infra | 12:31 | |
| krotscheck | jtomasek: Def have a repro. Found a couple of related github issues, may need to ditch shrinkwrap altogether. | 12:31 |
| *** placko_ has quit IRC | 12:32 | |
| krotscheck | jtomasek: As an unrelated comment, patternfly looks pretty slick | 12:33 |
| *** cbader has quit IRC | 12:33 | |
| *** pradk has joined #openstack-infra | 12:34 | |
| *** dizquierdo has quit IRC | 12:34 | |
| *** kgiusti has joined #openstack-infra | 12:34 | |
| jtomasek | krotscheck: nice, thank you! yeah, it is built on top of bootstrap | 12:34 |
| jtomasek | krotscheck: so it seems the problem is again related to node + npm versions? | 12:35 |
| AJaeger | jhesketh: Wow, thanks a lot! | 12:36 |
| *** nwkarsten has quit IRC | 12:36 | |
| *** nwkarsten has joined #openstack-infra | 12:36 | |
| jhesketh | no worries... if it happens again I'll write a tool for it | 12:36 |
| mescanef_ | 8 | 12:37 |
| *** lucasagomes has quit IRC | 12:37 | |
| mescanef_ | hello. please review project config change: https://review.openstack.org/#/c/301080/ | 12:38 |
| krotscheck | jtomasek: Ok, so it appears that the issue is in patternfly - their npm-shrinkwrap file contains all of the dev-dependencies, which shrinkwrap run in the parent module will pick up and yell about. | 12:38 |
| mescanef_ | AJaeger: :+) | 12:38 |
| *** lucasagomes has joined #openstack-infra | 12:38 | |
| *** kzaitsev_mb has quit IRC | 12:38 | |
| jhesketh | tobiash: sorry I was preoccupied when you pinged earlier. Do you have any more detail or a copy of the configs? | 12:38 |
| krotscheck | jtomasek I'm guessing that patternfly generates that shrinkwrap file using --dev? | 12:39 |
| jtomasek | krotscheck: no idea. I can try to ask at #patternfly | 12:39 |
| *** salv-orlando has joined #openstack-infra | 12:40 | |
| *** nwkarsten has quit IRC | 12:40 | |
| krotscheck | jtomasek: Yeah, lemme check something real quick. We may get away with using the _actual_ npm lts version. | 12:40 |
| tobiash | jhesketh: just bisected to the root cause: https://review.openstack.org/#/c/207094/ | 12:41 |
| jtomasek | krotscheck: nice | 12:41 |
| tobiash | jhesketh: but I don't have a clue yet, how this breaks | 12:41 |
| *** erlon has joined #openstack-infra | 12:42 | |
| *** cbader has joined #openstack-infra | 12:42 | |
| *** doug-fish has quit IRC | 12:42 | |
| jhesketh | tobiash: do you have a paste of your layout? | 12:42 |
| pabelanger | sc`: thanks for the update | 12:42 |
| *** doug-fish has joined #openstack-infra | 12:42 | |
| tobiash | jhesketh: http://paste.openstack.org/show/493314/ | 12:44 |
| *** ramishra has quit IRC | 12:44 | |
| *** aysyd has quit IRC | 12:44 | |
| openstackgerrit | Nisha Agarwal proposed openstack/diskimage-builder: Install proliantutils in IPA's virtualenv https://review.openstack.org/299116 | 12:45 |
| *** rfolco has joined #openstack-infra | 12:45 | |
| *** ramishra has joined #openstack-infra | 12:45 | |
| krotscheck | jtomasek: NO dice, asking over in #patternfly and submitting an issue against the project. | 12:45 |
| tobiash | when pushing a change to testing/foo into review, the check pipeline is entered (without doing real builds, but with invoking zuul-merger and posting the start message to gerrit) | 12:45 |
| *** lucasagomes has quit IRC | 12:45 | |
| pabelanger | also, morning | 12:45 |
| tobiash | jhesketh: before that change, all is ok | 12:46 |
| tobiash | but I didn't understand this patch yet | 12:46 |
| jhesketh | tobiash: hmm .. what happens if you have a real job rather than 'noop' (which is a builtin exception) | 12:46 |
| *** lucasagomes has joined #openstack-infra | 12:46 | |
| tobiash | jhesketh: the noop was just for test case simplification | 12:47 |
| *** doug-fish has quit IRC | 12:47 | |
| *** sigmavirus24_awa is now known as sigmavirus24 | 12:47 | |
| *** achanda has joined #openstack-infra | 12:47 | |
| tobiash | with a real build it posts the start message, invokes zuul-cloner and does nothing after that | 12:47 |
| tobiash | also voting isn't done in this case | 12:48 |
| *** piet has joined #openstack-infra | 12:50 | |
| *** aysyd has joined #openstack-infra | 12:50 | |
| jtomasek | krotscheck: ok, thanks a lot | 12:50 |
| *** julim has joined #openstack-infra | 12:51 | |
| *** annegentle has joined #openstack-infra | 12:51 | |
| jhesketh | tobiash: and it works before 207094? | 12:51 |
| *** rlandy has joined #openstack-infra | 12:52 | |
| *** burgerk has joined #openstack-infra | 12:52 | |
| krotscheck | jtomasek I think I have a fix though | 12:53 |
| *** mrmartin has quit IRC | 12:53 | |
| *** kengo_sakai has quit IRC | 12:54 | |
| tobiash | jhesketh: yes, did a git bisect | 12:54 |
| *** doug-fish has joined #openstack-infra | 12:54 | |
| jtomasek | krotscheck: that would be great, because updating patternfly to a version which would eventually include the fix is a bit problematic | 12:55 |
| *** kzaitsev_mb has joined #openstack-infra | 12:55 | |
| tobiash | jhesketh: this patch definitely does something related to this (stuff with project names), but I was unable to find the issue with this yet | 12:56 |
| *** doug-fis_ has joined #openstack-infra | 12:56 | |
| *** kengo_sakai_mba has joined #openstack-infra | 12:56 | |
| jhesketh | tobiash: you aren't trying to enqueue something with a depends-on? | 12:56 |
| tobiash | nope | 12:56 |
| *** akshai has quit IRC | 12:56 | |
| jtomasek | krotscheck: I tried to install latest version as part of the debugging and now it has a dependency on globally installed bower, so that would probably be another problem at the gate | 12:57 |
| *** akshai has joined #openstack-infra | 12:57 | |
| krotscheck | jtomasek: Urm, what? Globally installed bower? | 12:57 |
| tobiash | jhesketh: happens to any patch regardless of the depends-on in the commit message | 12:57 |
| krotscheck | jtomasek: That seems like a thing you can easily patch for them, no? | 12:57 |
| jhesketh | tobiash: any notable differences between the logs before/after the commit? | 12:57 |
| *** achanda has quit IRC | 12:58 | |
| jtomasek | krotscheck: this is the issue https://github.com/jonmiles/bootstrap-treeview/issues/196 treeview is a patternfly dependency | 12:59 |
| *** doug-fish has quit IRC | 12:59 | |
| jtomasek | krotscheck: it should not be a problem to fix I think | 12:59 |
| *** keedya has quit IRC | 12:59 | |
| *** mrmartin has joined #openstack-infra | 13:00 | |
| *** keedya has joined #openstack-infra | 13:00 | |
| *** kengo_sakai_mba has quit IRC | 13:00 | |
| *** rodrigods has quit IRC | 13:01 | |
| *** rodrigods has joined #openstack-infra | 13:01 | |
| *** |-paul-| has quit IRC | 13:02 | |
| *** rossella_s has quit IRC | 13:03 | |
| *** nwkarsten has joined #openstack-infra | 13:03 | |
| *** rossella_s has joined #openstack-infra | 13:03 | |
| *** nwkarste_ has joined #openstack-infra | 13:04 | |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/project-config: Call npm prune before npm shrinkwrap https://review.openstack.org/302743 | 13:04 |
| krotscheck | jtomasek: ^^ | 13:05 |
| *** Guest34177 is now known as BobBall | 13:05 | |
| tobiash | jhesketh: found a difference in scheduler.addChange | 13:07 |
| tobiash | during debug | 13:07 |
| *** nwkarsten has quit IRC | 13:07 | |
| jtomasek | krotscheck: perfect! thanks again! | 13:07 |
| *** achanda has joined #openstack-infra | 13:08 | |
| *** dingyichen has quit IRC | 13:08 | |
| tobiash | jhesketh: http://paste.openstack.org/show/493328/ | 13:09 |
| tobiash | hope that makes the runtime difference clear | 13:10 |
| krotscheck | jtomasek: No prob, that's why I'm here :) | 13:10 |
| *** xarses has quit IRC | 13:10 | |
| *** eric_lopez has quit IRC | 13:10 | |
| *** Goneri has joined #openstack-infra | 13:10 | |
| *** flongo has quit IRC | 13:10 | |
| *** amitgandhinz has joined #openstack-infra | 13:11 | |
| *** eric_lopez has joined #openstack-infra | 13:11 | |
| *** flongo has joined #openstack-infra | 13:11 | |
| *** achanda has quit IRC | 13:13 | |
| tobiash | jhesketh: seems like getChangeQueue causes the issue | 13:13 |
| tobiash | it no longer returns an empty change queue for the project not part of the pipeline | 13:13 |
| *** akshai has quit IRC | 13:14 | |
| *** xarses has joined #openstack-infra | 13:15 | |
| tobiash | this causes the change to enter any change queue and processing until zuul notices that there are no builds to run | 13:15 |
| *** xarses has quit IRC | 13:15 | |
| *** xarses has joined #openstack-infra | 13:15 | |
| openstackgerrit | Merged openstack-infra/project-config: Ironic grenade jobs do not work until after Mitaka https://review.openstack.org/300082 | 13:16 |
| *** esikachev has joined #openstack-infra | 13:16 | |
| tobiash | jhesketh: I suspect https://review.openstack.org/#/c/207094/9/zuul/scheduler.py line 1804 causes the issue | 13:17 |
| *** esker has quit IRC | 13:17 | |
| openstackgerrit | Merged openstack-infra/project-config: Add translation-jobs to horizon-cisco-ui https://review.openstack.org/300065 | 13:17 |
| *** daemontool has quit IRC | 13:17 | |
| *** mrmartin has quit IRC | 13:19 | |
| *** _ari_ has joined #openstack-infra | 13:19 | |
| tobiash | jhesketh: yes, confirmed, if I revert this line then this issue is gone, but I don't know if this would break the foreign project functionality | 13:19 |
| openstackgerrit | Merged openstack-infra/project-config: Add requirements job to horizon-cisco-ui https://review.openstack.org/300061 | 13:20 |
| openstackgerrit | Merged openstack-infra/project-config: Keystone Experimental patch_cover gate job https://review.openstack.org/296808 | 13:20 |
| *** psilvad has joined #openstack-infra | 13:20 | |
| *** tiswanso has joined #openstack-infra | 13:20 | |
| *** moravec has joined #openstack-infra | 13:20 | |
| jhesketh | tobiash: hmm so I'm still confused what the actual bug is in terms of functionality.. are you testing this against review.o.o or another gerrit? | 13:21 |
| *** rcarrillocruz has quit IRC | 13:22 | |
| *** oanson has quit IRC | 13:22 | |
| *** doug-fis_ has quit IRC | 13:23 | |
| *** akshai has joined #openstack-infra | 13:23 | |
| tobiash | jhesketh: my own gerrit | 13:23 |
| jhesketh | tobiash: what version of gerrit? | 13:23 |
| tobiash | 2.11.3 | 13:24 |
| *** rcarrillocruz has joined #openstack-infra | 13:25 | |
| jhesketh | tobiash: and the layout example you gave.. are you trying to enqueue into the check or test pipeline? | 13:25 |
| jhesketh | or is it different | 13:25 |
| *** ociuhandu has joined #openstack-infra | 13:25 | |
| tobiash | jhesketh: I just upload the patch and expect that nothing is enqueued directly as foo is only in test pipeline | 13:25 |
| jhesketh | tobiash: as in that's expected right? | 13:26 |
| tobiash | but when I upload a patch it partly enters the check pipeline which is not configured for the project | 13:26 |
| *** xyang1 has joined #openstack-infra | 13:26 | |
| *** esker has joined #openstack-infra | 13:26 | |
| *** mriedem has joined #openstack-infra | 13:26 | |
| openstackgerrit | Merged openstack-infra/project-config: Experimental gate-kolla-dsvm-build-centos-7-binary https://review.openstack.org/302481 | 13:26 |
| tobiash | jhesketh: yes, expected is that no pipeline gets entered | 13:26 |
| jhesketh | tobiash: oh, I see what you're saying.. is it reporting back to gerrit too | 13:27 |
| jhesketh | ie it shouldn't be | 13:27 |
| tobiash | exactly | 13:27 |
| jhesketh | righto, lets see if we can replicate that as a test | 13:28 |
| *** daemontool has joined #openstack-infra | 13:28 | |
| tobiash | the real behaviour is that it enters the check pipeline, posts the start message on gerrit, calls zuul merger for every pipeline matching the requirements (this is pain for us due to limited bandwith) and after that stops processing | 13:28 |
| *** matt-borland has joined #openstack-infra | 13:28 | |
| *** psilvad_ has joined #openstack-infra | 13:29 | |
| jhesketh | tobiash: okay | 13:30 |
| *** cdent has quit IRC | 13:31 | |
| *** rcarrillocruz has quit IRC | 13:31 | |
| *** rcarrillocruz has joined #openstack-infra | 13:32 | |
| *** jamesmcarthur has joined #openstack-infra | 13:33 | |
| *** esker has quit IRC | 13:36 | |
| *** jsavak has joined #openstack-infra | 13:36 | |
| *** esker has joined #openstack-infra | 13:36 | |
| *** roxanaghe has joined #openstack-infra | 13:37 | |
| *** vincentll has quit IRC | 13:37 | |
| openstackgerrit | Rob Cresswell proposed openstack-infra/project-config: Make the Django 1.9 tests voting on OpenStack Auth https://review.openstack.org/302765 | 13:38 |
| *** annegentle has quit IRC | 13:38 | |
| *** moravec1 has joined #openstack-infra | 13:39 | |
| *** esikachev has quit IRC | 13:39 | |
| *** bdemers has quit IRC | 13:40 | |
| *** yamamoto has quit IRC | 13:40 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/shade: Update func tests for latest devstack flavors https://review.openstack.org/302767 | 13:40 |
| *** moravec has quit IRC | 13:41 | |
| *** roxanaghe has quit IRC | 13:41 | |
| Shrews | jhesketh: can you take a peek at ^^^ ? Shade gate is broken with the latest devstack | 13:41 |
| *** sc68cal has joined #openstack-infra | 13:42 | |
| jhesketh | Shrews: sure, looking | 13:42 |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/shade: Add support for Designate zones https://review.openstack.org/302770 | 13:44 |
| *** nwkarsten has joined #openstack-infra | 13:44 | |
| rcarrillocruz | Shrews: second try for designate ^ | 13:44 |
| rcarrillocruz | :D | 13:44 |
| rcarrillocruz | mordred: | 13:44 |
| rcarrillocruz | let me know if we want to gate func tests on this | 13:44 |
| rcarrillocruz | if so, we'll need to fiddle with tests to enable devstack designate plugin | 13:44 |
| *** kaisers has joined #openstack-infra | 13:44 | |
| *** bdemers has joined #openstack-infra | 13:45 | |
| jhesketh | Shrews: change looks fine, do you have a link handy to the updated devstack for checking ? | 13:45 |
| *** vincentll has joined #openstack-infra | 13:45 | |
| Shrews | rcarrillocruz: yes. we will not accept new APIs w/o func tests to know they work. | 13:45 |
| *** nwkarst__ has joined #openstack-infra | 13:45 | |
| Shrews | jhesketh: i do not. i just remember seeing a ML email about it, and verified with my local devstack | 13:45 |
| *** EricGonczer_ has joined #openstack-infra | 13:45 | |
| rcarrillocruz | so new test with devstack, or piggy back in some and just add the plugin | 13:45 |
| jhesketh | Shrews: no worires, gimme a bit to poke at devstack then | 13:46 |
| rcarrillocruz | i lean towards a new one | 13:46 |
| rcarrillocruz | new test with devstack designate i mean | 13:46 |
| *** links has quit IRC | 13:46 | |
| Shrews | rcarrillocruz: can you rephrase the question? not sure i understand the two options (i've never used designate) | 13:47 |
| *** nwkarste_ has quit IRC | 13:47 | |
| *** nwkars___ has joined #openstack-infra | 13:48 | |
| jhesketh | Shrews: lgtm :-) | 13:48 |
| rcarrillocruz | to run designate in devstack, you install a plugin | 13:48 |
| jhesketh | (so long as it passes) | 13:48 |
| *** bdemers_ has joined #openstack-infra | 13:48 | |
| *** nwkars___ has quit IRC | 13:48 | |
| rcarrillocruz | my question is, should I add the plugin to the existing tests (nova, neutron , whatever) | 13:48 |
| Shrews | jhesketh: thx | 13:48 |
| rcarrillocruz | or create a new job that will enable that plugin | 13:48 |
| Shrews | rcarrillocruz: ah, now i see | 13:48 |
| *** mrmartin has joined #openstack-infra | 13:48 | |
| rcarrillocruz | i lean towards a new one to not clutter existing ones | 13:48 |
| rcarrillocruz | although that adds more jobs to the gate | 13:49 |
| rcarrillocruz | not sure | 13:49 |
| *** nwkarste_ has joined #openstack-infra | 13:49 | |
| *** piet has quit IRC | 13:49 | |
| *** bdemers has quit IRC | 13:49 | |
| *** nwkarsten has quit IRC | 13:49 | |
| *** nwkarst__ has quit IRC | 13:50 | |
| Shrews | rcarrillocruz: i just enabled heat across all of the tests. i think we should just do the same for designate | 13:50 |
| rcarrillocruz | oh | 13:50 |
| rcarrillocruz | ok | 13:50 |
| rcarrillocruz | thanks | 13:50 |
| * rcarrillocruz goes check how designate folks set the plugin on their jobs | 13:50 | |
| Shrews | rcarrillocruz: otherwise we'll end up with 1 job per service, which is yuck | 13:50 |
| rcarrillocruz | yeah | 13:50 |
| rcarrillocruz | pros and cons | 13:51 |
| *** nwkarsten has joined #openstack-infra | 13:51 | |
| *** mrmartin has quit IRC | 13:52 | |
| *** nwkarst__ has joined #openstack-infra | 13:53 | |
| *** tonytan4ever has joined #openstack-infra | 13:53 | |
| *** nwkarste_ has quit IRC | 13:53 | |
| Shrews | rcarrillocruz: also, fyi, shade gate is broken until the change i just submitted merges | 13:54 |
| *** zeih has joined #openstack-infra | 13:54 | |
| *** ddieterly has joined #openstack-infra | 13:54 | |
| *** EricGonczer_ has quit IRC | 13:55 | |
| *** mrmartin has joined #openstack-infra | 13:55 | |
| *** nwkarsten has quit IRC | 13:55 | |
| *** nwkarst__ has quit IRC | 13:55 | |
| *** piet has joined #openstack-infra | 13:56 | |
| *** ddieterly is now known as ddieterly[away] | 13:56 | |
| *** kzaitsev_mb has quit IRC | 13:56 | |
| rcarrillocruz | ah | 13:56 |
| rcarrillocruz | cool thx | 13:56 |
| *** nwkarsten has joined #openstack-infra | 13:56 | |
| fungi | just a reminder, i'm not really around until probably 1800 utc today | 13:56 |
| fungi | briefly on from the passenger seat of a car | 13:57 |
| fungi | but about to drop back off soon | 13:57 |
| *** yamamoto has joined #openstack-infra | 13:57 | |
| Shrews | fungi: unacceptable | 13:58 |
| *** EricGonczer_ has joined #openstack-infra | 13:58 | |
| *** dizquierdo has joined #openstack-infra | 13:58 | |
| *** cdent has joined #openstack-infra | 13:59 | |
| krotscheck | Another patch working through some of the fallout from our npm upgrade, only needs one more core: https://review.openstack.org/#/c/302743/ | 13:59 |
| mrmartin | What's wrong with bindep here: http://logs.openstack.org/34/302634/1/check/gate-groups-unittests/03b44fa/console.html? groups.o.o project | 13:59 |
| pabelanger | mrmartin: http://logs.openstack.org/34/302634/1/check/gate-groups-unittests/03b44fa/console.html?#_2016-04-07_11_54_58_499 seems wrong | 14:00 |
| mrmartin | but what is wrong? | 14:00 |
| mrmartin | the sudo? | 14:01 |
| *** nwkarsten has quit IRC | 14:01 | |
| pabelanger | at first glance, I would say sudo is wrong some how. | 14:01 |
| mrmartin | it was working before :) | 14:01 |
| pabelanger | I know we recently upgrade the JJB to use bindep | 14:01 |
| *** yamahata has joined #openstack-infra | 14:01 | |
| fungi | mrmartin: i think our attempts to make the package installation quieter are overzealous and making that hard to debug | 14:02 |
| fungi | i have a feeling apt-get install is erroring somehow but being silenced | 14:02 |
| *** esikachev has joined #openstack-infra | 14:02 | |
| pabelanger | mrmartin: fungi I see the issue, we revoke-sudo before running bindep | 14:02 |
| pabelanger | let me get a patch | 14:03 |
| mrmartin | who can help to debug this? | 14:03 |
| mrmartin | thanks pabelanger | 14:03 |
| fungi | pabelanger: oh, yep install-distro-packages is supposed to always be before revoke-sudo, if present | 14:03 |
| *** yamamoto has quit IRC | 14:03 | |
| fungi | that explains sudo looking for an askpass client | 14:03 |
| fungi | didn't even dawn on me | 14:03 |
| fungi | so apt-get is never being called | 14:03 |
| fungi | that's why it was silent :/ | 14:04 |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: revoke-sudo after bindep for groups jobs https://review.openstack.org/302784 | 14:04 |
| fungi | good eye | 14:04 |
| pabelanger | mrmartin: fungi: ^ should fix it | 14:04 |
| openstackgerrit | Sean McGinnis proposed openstack/requirements: Update to os-brick 1.3.0 https://review.openstack.org/302786 | 14:05 |
| AJaeger | pabelanger: LGTM, yeah... | 14:05 |
| *** cloudtrainme has joined #openstack-infra | 14:05 | |
| mrmartin | ah | 14:05 |
| *** wfoster_afk is now known as wfoster | 14:05 | |
| fungi | hopefully that | 14:05 |
| fungi | 'll land quickly | 14:05 |
| mrmartin | thanks | 14:06 |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Migrate openstackid jobs to ubuntu-trusty https://review.openstack.org/301677 | 14:06 |
| *** esker has quit IRC | 14:06 | |
| pabelanger | Also fixed for openstackid jobs now ^ | 14:06 |
| pabelanger | which hasn't migrated to ubuntu-trusty yet | 14:06 |
| *** sdake has joined #openstack-infra | 14:07 | |
| *** thomasem_ is now known as thomasem | 14:08 | |
| openstackgerrit | Zhipeng Huang proposed openstack-infra/project-config: add new project Coupler to openstack repo https://review.openstack.org/302791 | 14:09 |
| *** knikolla has joined #openstack-infra | 14:10 | |
| *** akaszuba has quit IRC | 14:10 | |
| openstackgerrit | Merged openstack-infra/project-config: Give project teams access to approve stable/mitaka changes https://review.openstack.org/300667 | 14:11 |
| *** nwkarsten has joined #openstack-infra | 14:11 | |
| *** thiagop has joined #openstack-infra | 14:11 | |
| AJaeger | bbl | 14:11 |
| *** sdake has quit IRC | 14:12 | |
| *** rbrndt_ has joined #openstack-infra | 14:13 | |
| *** mylu has joined #openstack-infra | 14:17 | |
| *** julim has quit IRC | 14:19 | |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul: Projects without jobs still enter pipeline https://review.openstack.org/302800 | 14:20 |
| jhesketh | tobiash: I think ^ proves your bug.. The next part (and probably tomorrows job) is f | 14:20 |
| jhesketh | inding the fix | 14:20 |
| jhesketh | tobiash: when I remove the check pipeline from that layout the tests pass | 14:20 |
| *** thorst_ has quit IRC | 14:21 | |
| *** sdake has joined #openstack-infra | 14:21 | |
| *** jamesmcarthur has quit IRC | 14:21 | |
| *** julim has joined #openstack-infra | 14:22 | |
| *** gildub has quit IRC | 14:22 | |
| *** thorst_ has joined #openstack-infra | 14:22 | |
| *** baoli_ has quit IRC | 14:26 | |
| *** thorst__ has joined #openstack-infra | 14:26 | |
| vponomaryov | what happened to CI? queues kind of freezed | 14:27 |
| *** thorst_ has quit IRC | 14:27 | |
| pabelanger | zuul recalculating the world | 14:27 |
| openstackgerrit | Thomas Morin proposed openstack-infra/project-config: Add tempest-dsvm-networking-bgpvpn-bagpipe jobs https://review.openstack.org/258522 | 14:28 |
| vponomaryov | pabelanger: is it long operation? | 14:29 |
| *** yamamoto has joined #openstack-infra | 14:29 | |
| *** nwkarsten has quit IRC | 14:30 | |
| openstackgerrit | Merged openstack/requirements: Require reno 1.6.2 or newer https://review.openstack.org/302612 | 14:30 |
| *** baoli has joined #openstack-infra | 14:30 | |
| *** zz_dimtruck is now known as dimtruck | 14:30 | |
| *** matrohon has quit IRC | 14:31 | |
| *** matrohon has joined #openstack-infra | 14:31 | |
| pabelanger | vponomaryov: can take a few minutes to complete. | 14:31 |
| *** Qiming_ has joined #openstack-infra | 14:32 | |
| vponomaryov | pabelanger: ok, ty | 14:32 |
| *** Qiming has quit IRC | 14:32 | |
| *** psanchez has quit IRC | 14:33 | |
| *** eharney has joined #openstack-infra | 14:34 | |
| *** jamesmcarthur has joined #openstack-infra | 14:35 | |
| *** thorst__ has quit IRC | 14:37 | |
| *** mylu has quit IRC | 14:37 | |
| *** thorst_ has joined #openstack-infra | 14:41 | |
| *** mylu has joined #openstack-infra | 14:42 | |
| *** ddieterly has joined #openstack-infra | 14:42 | |
| *** rkukura has quit IRC | 14:42 | |
| *** psilvad_ has quit IRC | 14:44 | |
| *** moravec1 has quit IRC | 14:46 | |
| *** psilvad_ has joined #openstack-infra | 14:47 | |
| *** psilvad_ has quit IRC | 14:48 | |
| *** nwkarsten has joined #openstack-infra | 14:48 | |
| *** nmagnezi has quit IRC | 14:49 | |
| *** esikachev has quit IRC | 14:49 | |
| openstackgerrit | Derek Higgins proposed openstack-infra/tripleo-ci: Create a test flavor for the pingtest VM https://review.openstack.org/293533 | 14:50 |
| *** jcooley_ has joined #openstack-infra | 14:51 | |
| *** salv-orl_ has joined #openstack-infra | 14:52 | |
| *** jcooley_ has quit IRC | 14:52 | |
| *** kzaitsev_mb has joined #openstack-infra | 14:52 | |
| *** maishsk has quit IRC | 14:53 | |
| *** salv-orl_ has quit IRC | 14:53 | |
| *** nwkarsten has quit IRC | 14:53 | |
| *** esikachev has joined #openstack-infra | 14:54 | |
| *** salv-orlando has quit IRC | 14:54 | |
| *** thorst__ has joined #openstack-infra | 14:54 | |
| *** psilvad_ has joined #openstack-infra | 14:55 | |
| *** sigmavirus24 is now known as sigmavirus24_awa | 14:55 | |
| *** sigmavirus24_awa is now known as sigmavirus24 | 14:55 | |
| *** ramishra has quit IRC | 14:55 | |
| *** tphummel has joined #openstack-infra | 14:56 | |
| *** nwkarsten has joined #openstack-infra | 14:57 | |
| *** ramishra has joined #openstack-infra | 14:57 | |
| *** Qiming_ has quit IRC | 14:57 | |
| *** thorst_ has quit IRC | 14:58 | |
| *** kzaitsev_mb has quit IRC | 14:58 | |
| *** regXboi has joined #openstack-infra | 14:58 | |
| *** Qiming has joined #openstack-infra | 14:59 | |
| *** woodster_ has joined #openstack-infra | 15:00 | |
| openstackgerrit | Matthew Treinish proposed openstack-infra/subunit2sql: Fix lock range, again https://review.openstack.org/302515 | 15:00 |
| *** thorst__ is now known as thorst________ | 15:00 | |
| *** tiswanso has quit IRC | 15:01 | |
| *** EricGonczer_ has quit IRC | 15:02 | |
| *** tiswanso has joined #openstack-infra | 15:03 | |
| *** nwkarsten has quit IRC | 15:04 | |
| *** EricGonczer_ has joined #openstack-infra | 15:04 | |
| *** amitgandhinz has quit IRC | 15:04 | |
| vkramskikh | it seems node.js upgrade made gate-fuel-ui-npm-run-lint always fail in a very strange way: it seems it fails on "cat npm-shrinkwrap.json" with error "cat: write error: Resource temporarily unavailable" | 15:05 |
| vkramskikh | does anybody have any idea why it happens? | 15:06 |
| vkramskikh | krotscheck: do you know why we need to create shrinkwrap.json and then cat it? | 15:06 |
| *** nwkarsten has joined #openstack-infra | 15:07 | |
| *** ajmiller has joined #openstack-infra | 15:07 | |
| clarkb | only the ubuntu-trusty upload worked against rax-ord last night. The remaining four uploads I queued all failed | 15:07 |
| vkramskikh | can I modify this job to avoid generating shrinkwrap if it already exists? (fuel-ui repo has it) | 15:07 |
| clarkb | but thats ok, all of the other regions have the images we need and rax-ord can catch upafter we switch servers | 15:07 |
| *** j_king_ is now known as j_king | 15:07 | |
| *** thorst________ is now known as thorst_ | 15:08 | |
| openstackgerrit | Derek Higgins proposed openstack-infra/tripleo-ci: Nothing to see here https://review.openstack.org/111011 | 15:08 |
| openstackgerrit | Derek Higgins proposed openstack-infra/tripleo-ci: Temporarily disable mistral on the undercloud https://review.openstack.org/302832 | 15:08 |
| *** fitoduarte has quit IRC | 15:10 | |
| *** Guest74036 has quit IRC | 15:10 | |
| *** nwkarsten has quit IRC | 15:11 | |
| vkramskikh | tsufiev: maybe you know about shrinkwrap? | 15:11 |
| *** moravec has joined #openstack-infra | 15:12 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Use noop-jobs for openstack/fuel-ui https://review.openstack.org/302835 | 15:12 |
| *** rkukura has joined #openstack-infra | 15:13 | |
| tsufiev | vkramskikh, unfortunately no | 15:14 |
| *** maishsk has joined #openstack-infra | 15:14 | |
| *** nwkarste_ has joined #openstack-infra | 15:16 | |
| tsufiev | krotscheck, I wonder how this patch https://review.openstack.org/#/c/300446/ manages to hit npm-run issues again and again :) | 15:16 |
| tsufiev | a jinxed patch | 15:16 |
| tsufiev | I think it's 3rd failure in a row | 15:17 |
| vkramskikh | it seems there is failure on downloading phantomjs | 15:17 |
| tsufiev | ah, that's true | 15:18 |
| tsufiev | still it's a mystery to me | 15:18 |
| tsufiev | what's so special about it? | 15:18 |
| clarkb | its hosted on bitbucket iirc and bitbucket says go away | 15:18 |
| clarkb | which is why we don't allow external link packages for python packages | 15:19 |
| *** baoli has quit IRC | 15:19 | |
| openstackgerrit | Merged openstack-infra/tripleo-ci: Use split cirros image for ping test https://review.openstack.org/301951 | 15:20 |
| *** akaszuba has joined #openstack-infra | 15:20 | |
| *** nwkarste_ has quit IRC | 15:20 | |
| tsufiev | clarkb, perhaps part of our distributed system was banned on bitbucket? | 15:21 |
| openstackgerrit | Merged openstack-infra/project-config: Migrate rally from bare-precise https://review.openstack.org/301652 | 15:21 |
| *** amitgandhinz has joined #openstack-infra | 15:21 | |
| *** nwkarsten has joined #openstack-infra | 15:21 | |
| openstackgerrit | Merged openstack-infra/project-config: Migrate rally to centos-7 https://review.openstack.org/301334 | 15:22 |
| openstackgerrit | Merged openstack-infra/project-config: revoke-sudo after bindep for groups jobs https://review.openstack.org/302784 | 15:22 |
| clarkb | tsufiev: I think this is a general issue and not one on our end | 15:22 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Revert "Switch npm-lint to non-voting for fuel-ui" https://review.openstack.org/302854 | 15:22 |
| clarkb | tsufiev: we don't really reuse IPs that aggressively | 15:22 |
| *** zeih has quit IRC | 15:22 | |
| clarkb | nor do we talk to bitbucket | 15:22 |
| rcarrillocruz | nice | 15:22 |
| AJaeger | igorbelikov: See https://review.openstack.org/#/c/302854/ | 15:23 |
| rcarrillocruz | long time i don't try to simulate a devstack-gate locally | 15:23 |
| rcarrillocruz | and got a freeze | 15:23 |
| rcarrillocruz | my memory told me i hit this some point | 15:23 |
| rcarrillocruz | https://review.openstack.org/#/c/160480/1 | 15:23 |
| rcarrillocruz | clarkb: :-) | 15:23 |
| *** zeih has joined #openstack-infra | 15:23 | |
| clarkb | rcarrillocruz: apt-get is expecting input, use the reproduce.sh script | 15:23 |
| rcarrillocruz | will give it a try again in a sec | 15:23 |
| clarkb | rcarrillocruz: the fix is to close stdin | 15:23 |
| AJaeger | vkramskikh: see also https://review.openstack.org/302743 | 15:23 |
| *** achanda has joined #openstack-infra | 15:24 | |
| openstackgerrit | Aliaksandr Buhayeu proposed openstack-infra/jenkins-job-builder: Clarify exception message for missed trigger properties https://review.openstack.org/302864 | 15:24 |
| *** ddieterly is now known as ddieterly[away] | 15:25 | |
| *** alex_xu has quit IRC | 15:25 | |
| AJaeger | pabelanger: https://review.openstack.org/#/c/302835 is not what we do normally | 15:26 |
| AJaeger | pabelanger: I think my -2 was too eager | 15:26 |
| *** yamahata has quit IRC | 15:26 | |
| *** kmartin has joined #openstack-infra | 15:27 | |
| AJaeger | pabelanger: if this would be in the gate queue, we would have a real problem since now all jobs having noop in gate would be in shared queue. For check it might be harmless - but still, I don't think we need this and we have other examples of the pattern. | 15:27 |
| *** cloudtrainme has quit IRC | 15:28 | |
| *** zeih has quit IRC | 15:28 | |
| *** Swami has joined #openstack-infra | 15:28 | |
| *** alex_xu has joined #openstack-infra | 15:29 | |
| *** armax has joined #openstack-infra | 15:30 | |
| igorbelikov | AJaeger: thanks, and thanks for workflow-1, It’ll be better to wait till fuel-ui passes npm-lint | 15:31 |
| *** cloudtrainme has joined #openstack-infra | 15:31 | |
| *** dangers_away is now known as dangers | 15:32 | |
| openstackgerrit | Merged openstack-infra/project-config: Allow fuel-ui-core to merge into stable branches https://review.openstack.org/301548 | 15:32 |
| vgridnev | AJaeger, is it true that python34 job cannot be added as non-voting? | 15:32 |
| AJaeger | igorbelikov: please tell me in the review once it's fine | 15:32 |
| AJaeger | vgridnev: sure, it can. | 15:32 |
| AJaeger | vgridnev: which review? | 15:33 |
| vgridnev | https://review.openstack.org/#/c/302647/ | 15:33 |
| *** tphummel has quit IRC | 15:33 | |
| vgridnev | this one | 15:33 |
| openstackgerrit | Merged openstack-infra/project-config: Watcher multinode - disable plugin in subnode https://review.openstack.org/299809 | 15:33 |
| vikram | AJaeger: Have doubt about your comment on https://review.openstack.org/#/c/268726/ | 15:33 |
| igorbelikov | AJaeger: sure | 15:33 |
| AJaeger | vgridnev: that one adds the template and *also* a single job | 15:33 |
| AJaeger | that double, use either option. For a non-voting, use the single job and not the template. For voting the template and not the job | 15:34 |
| vikram | AJaeger: This has two branches, do you really want to import stable/mitaka? I suggest to delete it instead - it will only confuse folks later --- You would like me to delete the stable/mitaka branch? | 15:34 |
| AJaeger | the template adds the job to both check and gate and we do not have non-voting jobs in gate | 15:34 |
| pabelanger | AJaeger: fair enough, I can abandon it. I just noticed it in passing, since no other job had noop only gate | 15:34 |
| *** Nikolay_St has quit IRC | 15:34 | |
| AJaeger | vikram: yes, my suggestion is to delete that branch. Once a repo is imported, branch deletion is a manual step. | 15:35 |
| AJaeger | pabelanger: there should be some - I remember approving some in the past;) | 15:35 |
| vikram | AJaeger: Ok.. Will do | 15:35 |
| *** wfoster has quit IRC | 15:35 | |
| AJaeger | vikram: since you have not done a release for mitaka from that repo, I suggest to remove it. | 15:35 |
| *** jsavak has quit IRC | 15:35 | |
| *** links has joined #openstack-infra | 15:36 | |
| *** sc68cal has quit IRC | 15:36 | |
| vikram | AJaeger: Ok got it | 15:36 |
| pabelanger | AJaeger: okay, I'll abandon based on your comments. | 15:36 |
| vikram | AJaeger: Done! | 15:36 |
| openstackgerrit | Vitaly Kramskikh proposed openstack-infra/project-config: Do not run npm shrinkwrap if npm-shrinkwrap.json already exists https://review.openstack.org/302900 | 15:37 |
| *** mtanino has joined #openstack-infra | 15:37 | |
| openstackgerrit | Vitaly Gridnev proposed openstack-infra/project-config: [sahara] setup additional jobs for sahara-dashboard https://review.openstack.org/302647 | 15:37 |
| *** achanda has quit IRC | 15:37 | |
| *** ddieterly[away] is now known as ddieterly | 15:37 | |
| fungi | clarkb: rcarrillocruz: there are also apt-get command-line options that tell it to run noninteractively | 15:38 |
| clarkb | fungi: iirc they stopped working at some point | 15:38 |
| *** nelsnelson has joined #openstack-infra | 15:38 | |
| clarkb | fungi: we never noticed with jenkins because it doesn't give it a working stdin, but when you start from a bash shell on your local box it does? | 15:39 |
| *** jcooley_ has joined #openstack-infra | 15:39 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/project-config: Implement periodic job to update Puppet OpenStack constraints https://review.openstack.org/301375 | 15:39 |
| *** nelsnels_ has quit IRC | 15:39 | |
| openstackgerrit | Vitaly Kramskikh proposed openstack-infra/project-config: Do not run npm shrinkwrap if npm-shrinkwrap.json already exists https://review.openstack.org/302900 | 15:40 |
| fungi | clarkb: more likely someone neglected to add the right options to some new apt-get call they introduced | 15:40 |
| fungi | i've never seen those options fail to be respected | 15:40 |
| *** wfoster has joined #openstack-infra | 15:40 | |
| *** vincentll has quit IRC | 15:40 | |
| EmilienM | pabelanger: I commented on https://review.openstack.org/#/c/301375/ - please re-consider your vote | 15:40 |
| *** jsavak has joined #openstack-infra | 15:40 | |
| *** Qiming has quit IRC | 15:41 | |
| pabelanger | EmilienM: no -1 from me, just a note about the history | 15:42 |
| clarkb | fungi: ya its possible we just don't know what the entire list of required flags are or if they changed sometime in the last 4 years or so didn't notice because it works in a jenkins env | 15:42 |
| EmilienM | pabelanger: if you ready my commit message and the code I'm adding, what I'm doing and what you did 6 months ago are 2 very different things | 15:42 |
| *** mikelk has quit IRC | 15:43 | |
| anteaya | ttx: just following up on the merge tag conversation in backscroll, it looks like searchlight: 302579 zaqar: 302580 and congress: 302558 have yet to merge their patches | 15:43 |
| *** afranc has joined #openstack-infra | 15:44 | |
| *** ddieterly is now known as ddieterly[away] | 15:44 | |
| pabelanger | EmilienM: Right, I think the issue was with the automation of it over the functionality. Either way, going to hold off on voting until other infra-root review the patch. | 15:44 |
| fungi | clarkb: back when i was remotely triggering devstack jobs by hand, i had to explicitly close stdin for other reasons that caused it to hang, so i don't think it's particularly new | 15:44 |
| clarkb | fungi: ah | 15:44 |
| EmilienM | pabelanger: fair enough, thanks | 15:45 |
| fungi | clarkb: this was several years ago at least that it was necessary | 15:45 |
| pabelanger | EmilienM: FWIW: I'd like to see some automation too just need to make sure everybody is on board with it | 15:45 |
| EmilienM | pabelanger: me too, moreover, other projects already have this same automation. | 15:46 |
| *** kzaitsev_mb has joined #openstack-infra | 15:46 | |
| *** ddieterly[away] is now known as ddieterly | 15:46 | |
| pabelanger | EmilienM: /me nods | 15:47 |
| openstackgerrit | Zhipeng Huang proposed openstack-infra/project-config: add new project Coupler to openstack repo https://review.openstack.org/302791 | 15:47 |
| *** jaosorior has quit IRC | 15:47 | |
| *** scheuran has quit IRC | 15:47 | |
| *** jaosorior has joined #openstack-infra | 15:47 | |
| *** jsavak has quit IRC | 15:47 | |
| *** jsavak has joined #openstack-infra | 15:48 | |
| *** links has quit IRC | 15:48 | |
| *** ramishra has quit IRC | 15:48 | |
| *** baoli has joined #openstack-infra | 15:49 | |
| *** matrohon has quit IRC | 15:49 | |
| AJaeger | pabelanger, fungi: We have several of these proposal jobs like EmilienM's - we should discuss these again... | 15:49 |
| pabelanger | clarkb: fungi: only review left to migrate away from bare-precise (minus our jobs) https://review.openstack.org/#/c/301677/ | 15:50 |
| AJaeger | I meant: 3 open reviews | 15:50 |
| clarkb | pabelanger: looking | 15:50 |
| pabelanger | AJaeger: agreed | 15:50 |
| EmilienM | AJaeger: yes, please | 15:50 |
| * AJaeger puts it on agenda of next meeting and will mark changes as WIP for now... | 15:51 | |
| EmilienM | AJaeger: do we need to wait meeting? | 15:51 |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Add experimental bindep JJB for kolla https://review.openstack.org/302909 | 15:51 |
| EmilienM | AJaeger: I mean, we can discuss it on Gerrit | 15:51 |
| clarkb | pabelanger: I will actually defer to fungi on that one as he is most familiar with the state of things there | 15:51 |
| pabelanger | clarkb: WFM | 15:51 |
| pabelanger | so only kolla is left for devstack-centos7 | 15:52 |
| pabelanger | testing some experimental jobs today to confirm it will be a smooth cut over | 15:53 |
| AJaeger | EmilienM: this needs some broader discussion and fungi's signoff IMHO. | 15:53 |
| *** pcaruana has quit IRC | 15:53 | |
| anteaya | jhesketh: it happens quite frequently that a root needs to dig in statusbot logs to find channel topics and reset them | 15:53 |
| *** derekh has quit IRC | 15:53 | |
| pabelanger | AJaeger: EmilienM: Could be a fishbowl session at summit too | 15:53 |
| anteaya | jhesketh: if you are inclined to write a tool I think it would be welcomed | 15:53 |
| EmilienM | AJaeger: I have no doubt about it | 15:54 |
| *** maishsk has quit IRC | 15:54 | |
| EmilienM | pabelanger: can you also re-visit https://review.openstack.org/#/c/297469/ please? AFIK release is done | 15:54 |
| AJaeger | pabelanger: feel free to add to etherpad ;) | 15:54 |
| *** mrmartin has quit IRC | 15:55 | |
| EmilienM | AJaeger, pabelanger: I don't understand why we would need to wait Summit for that, other projects already have automation that sync the requirements, and that's the purpose of my patch | 15:55 |
| *** moravec has quit IRC | 15:55 | |
| pabelanger | EmilienM: sure, let me confirm that with release team | 15:55 |
| AJaeger | EmilienM: those jobs all run on the proposal slae that has access of passwords. | 15:57 |
| AJaeger | The jobs we currently do that should have been reviewed for security implications. | 15:57 |
| openstackgerrit | Zara proposed openstack-infra/python-storyboardclient: WIP Proof of Concept Storyboard Updating from Gerrit https://review.openstack.org/302912 | 15:57 |
| AJaeger | EmilienM: this is a broader security discussion on what to run. | 15:58 |
| *** sridhar_ram has joined #openstack-infra | 15:58 | |
| AJaeger | EmilienM: Feel free to discuss with fungi and the rest of the team, if you're unhappy with my -1. For now I want this to be a team discussion and like to have a clear guideline on how I shall review this. | 15:59 |
| anteaya | AJaeger: thank you for encouraging the discussion, can't join in now but thank you | 15:59 |
| *** apoorvad has joined #openstack-infra | 16:00 | |
| EmilienM | AJaeger: sure, no worries about -1. I did not have full background | 16:00 |
| *** moravec1 has joined #openstack-infra | 16:00 | |
| *** nmagnezi has joined #openstack-infra | 16:01 | |
| AJaeger | WIP ;) | 16:01 |
| *** xarses has quit IRC | 16:01 | |
| zaro | morning | 16:02 |
| *** xarses has joined #openstack-infra | 16:02 | |
| *** mtanino has quit IRC | 16:02 | |
| *** rvasilets has joined #openstack-infra | 16:02 | |
| *** armax has quit IRC | 16:02 | |
| tristanC | it does sounds risky to have a jenkins jobs submit change... what if those new changes trigger the same job ? :) | 16:02 |
| *** armax has joined #openstack-infra | 16:02 | |
| *** mtanino has joined #openstack-infra | 16:03 | |
| *** avarner_ has joined #openstack-infra | 16:03 | |
| AJaeger | tristanC: jenkins *proposes* changes and thus for sure will trigger new jobs | 16:03 |
| AJaeger | But that's a proposal, so no commit | 16:03 |
| *** tphummel has joined #openstack-infra | 16:04 | |
| *** angdraug has joined #openstack-infra | 16:04 | |
| openstackgerrit | Merged openstack/requirements: Updated from generate-constraints https://review.openstack.org/302539 | 16:04 |
| *** jamesmcarthur has quit IRC | 16:04 | |
| AJaeger | bbl | 16:05 |
| openstackgerrit | Merged openstack/requirements: Bump python-cinderclient to 1.6.0 https://review.openstack.org/295662 | 16:05 |
| *** julim has quit IRC | 16:05 | |
| *** yamamoto has quit IRC | 16:06 | |
| *** annegent_ has joined #openstack-infra | 16:08 | |
| *** akshai has quit IRC | 16:08 | |
| *** yamahata has joined #openstack-infra | 16:08 | |
| fungi | i'll try to take a look, but still very patchy cell signal here (just came out of a 15+ minute dead zone) | 16:08 |
| *** e0ne has quit IRC | 16:08 | |
| *** e0ne has joined #openstack-infra | 16:09 | |
| *** nwkarsten has quit IRC | 16:10 | |
| krotscheck | tsufiev Checking | 16:12 |
| krotscheck | tsufiev Oh | 16:12 |
| krotscheck | tsufiev Yes, have encountered that. Lemme tell ya | 16:12 |
| krotscheck | tsufiev, clarkb: Turns out that the binary for phantomjs is hosted on bintray, and it tends to hit over quota issues. The recommended fix they advocate is "Hey go use this mirror in china", which is... LOLworthy. | 16:13 |
| *** maishsk has joined #openstack-infra | 16:13 | |
| *** jsavak has quit IRC | 16:14 | |
| krotscheck | tsufiev, clarkb: My recommended fix is: Stop using phantomjs. it's not a real browser, and we run chrome and firefox in the gate. | 16:14 |
| *** fhubik has quit IRC | 16:14 | |
| *** dtantsur is now known as dtantsur|afk | 16:14 | |
| *** jsavak has joined #openstack-infra | 16:14 | |
| matt-borland | krotscheck, we can certainly modify the config to run chrome/ff if needed. Also, seems npm troubles have lifted this morning, may be ephemeral though. | 16:15 |
| matt-borland | nah, not lifted :) | 16:15 |
| * matt-borland strikes that last statement | 16:16 | |
| clarkb | krotscheck: any idea why they don't just host on npm propre? | 16:16 |
| clarkb | krotscheck: solve the issue and everyone can move on with life? | 16:16 |
| krotscheck | matt-borland: Yep, def not lifted, but we're no node4 and npm2 now, so that should be shaking out. | 16:16 |
| matt-borland | krotscheck, thank you | 16:16 |
| krotscheck | clarkb: I think it's because phantomJS is not actually a node thing. | 16:17 |
| krotscheck | clarkb: It's just used a lot by node things. | 16:17 |
| *** jamesmcarthur has joined #openstack-infra | 16:17 | |
| *** ramishra has joined #openstack-infra | 16:17 | |
| clarkb | but if npm can install it then its fair game to be hosted by npm | 16:17 |
| *** mylu has quit IRC | 16:19 | |
| *** jlanoux_ has quit IRC | 16:19 | |
| *** EricGonczer_ has quit IRC | 16:20 | |
| openstackgerrit | Merged openstack/requirements: Bump neutron-lib to 0.0.3 minimum https://review.openstack.org/285395 | 16:20 |
| *** mylu has joined #openstack-infra | 16:20 | |
| *** nwkarsten has joined #openstack-infra | 16:20 | |
| fungi | stopping for lunch, then planning to take over driving, but should be back online from a proper connection in another couple hours | 16:20 |
| clarkb | fungi: great we can work through nodepool switchover then | 16:20 |
| *** vgridnev has quit IRC | 16:21 | |
| *** vgridnev has joined #openstack-infra | 16:21 | |
| *** mrmartin has joined #openstack-infra | 16:21 | |
| clarkb | my rough plan is nova shutdown old host, switch dns, double check firewalls update, start service on new server, work through alien cleanup | 16:21 |
| clarkb | and I should make sure our ttl is 5 minutes, doing that now | 16:22 |
| pabelanger | clarkb: so, if you have time, I'd be interested to see which devstack-gate job we need to add for experimental ubuntu-xenial | 16:22 |
| krotscheck | clarkb: Sadly I don't own the phantomjs node module. | 16:22 |
| pabelanger | clarkb: after nodepool migration | 16:22 |
| avarner_ | Have there been any changes in gerrit in the last 2 days? I can't access any code reviews now. | 16:22 |
| *** avarner_ is now known as avarner | 16:22 | |
| clarkb | krotscheck: right, when we had this problem with pypi we went to the packages and asked them to change | 16:22 |
| clarkb | krotscheck: those that didn't we stopped using | 16:22 |
| *** ddieterly is now known as ddieterly[away] | 16:23 | |
| krotscheck | clarkb: I did. They said no. My recommendation is to stop using phantomjs | 16:23 |
| clarkb | pabelanger: I would add base dsvm tempest job, neutron dsvm tempest job, postgres temest job, and grenade | 16:23 |
| clarkb | pabelanger: and add those jobs to devstack-gate, devstack, tempest, and grenade to start | 16:23 |
| clarkb | once that is working we can add in multinode | 16:23 |
| krotscheck | clarkb: If I recall correctly the reason was *handwave handwave* it's hard *handwave* different binaries for different operating systems *handwave* | 16:24 |
| *** trown is now known as trown|lunch | 16:24 | |
| pabelanger | avarner: not yet, but changes are coming: http://lists.openstack.org/pipermail/openstack-dev/2016-April/091274.html | 16:24 |
| openstackgerrit | Merged openstack/requirements: Updated magnum client version to 2.0.0 https://review.openstack.org/296810 | 16:25 |
| openstackgerrit | Merged openstack/requirements: Bump zaqarclient and barbicanclient versions https://review.openstack.org/297103 | 16:25 |
| *** EricGonczer_ has joined #openstack-infra | 16:25 | |
| *** jcoufal has quit IRC | 16:26 | |
| openstackgerrit | Merged openstack/requirements: Bump python-ironic-inspector-client to 1.5.0 https://review.openstack.org/299367 | 16:26 |
| openstackgerrit | Merged openstack/requirements: oslo.config>=3.9.0 https://review.openstack.org/291175 | 16:26 |
| clarkb | krotscheck: oh its shipping precompiled binaries ya that can make things a little trickier though not impossible | 16:26 |
| openstackgerrit | Merged openstack/requirements: Require at least python-neutronclient 4.1.1 https://review.openstack.org/297826 | 16:26 |
| openstackgerrit | Merged openstack/requirements: Bump openstacksdk version to 0.8.4 https://review.openstack.org/301094 | 16:26 |
| tsufiev | krotscheck, thank you for the info. Need to talk to r1charj0nes about that | 16:26 |
| *** shashank_hegde has joined #openstack-infra | 16:27 | |
| clarkb | nodepool dns is set to 5 minute ttls | 16:27 |
| openstackgerrit | Merged openstack-infra/tripleo-ci: ssh less https://review.openstack.org/295973 | 16:28 |
| krotscheck | tsufiev: There are build macros that make sure that the browsers are available and that xvfb is started and configured. LMK if you have any questions, you can use the jobs in javascript.yml as a template. | 16:29 |
| *** jsavak has quit IRC | 16:29 | |
| *** bpokorny has joined #openstack-infra | 16:29 | |
| *** jsavak has joined #openstack-infra | 16:29 | |
| *** sridhar_ram has quit IRC | 16:32 | |
| openstackgerrit | Costin Galan proposed openstack/requirements: Add Windows as supported operating system https://review.openstack.org/302945 | 16:32 |
| tsufiev | krotscheck, sure. phantomjs dependency is r1chardj0nes' child, so - if we're going to remove it - I'd like first discuss it with him | 16:32 |
| anteaya | clarkb: looks like you are in a good place for the switchover when fungi is next online? | 16:32 |
| clarkb | anteaya: yes | 16:34 |
| *** yaume_ has quit IRC | 16:34 | |
| clarkb | I think my brain just exploded, you have to delete a file before renaming it in windows | 16:34 |
| openstackgerrit | Merged openstack-infra/tripleo-ci: Fix image caching logic https://review.openstack.org/299537 | 16:34 |
| *** moravec has joined #openstack-infra | 16:34 | |
| clarkb | oh the dest side | 16:34 |
| clarkb | still | 16:34 |
| anteaya | :/ | 16:35 |
| anteaya | that ws for the windows thing | 16:35 |
| anteaya | and yay on the you are in a good place thing for nodepool switchover | 16:36 |
| *** mylu has quit IRC | 16:36 | |
| clarkb | I wonder if three is a different way to do atomic rename type deals | 16:36 |
| anteaya | I have to go run errands in 20 minutes, I am out of food as I didn't plan on being home this week | 16:36 |
| clarkb | because not being able to do that is sadness | 16:36 |
| *** david-lyle has quit IRC | 16:36 | |
| *** moravec2 has joined #openstack-infra | 16:36 | |
| anteaya | sorry I am not understanding | 16:37 |
| anteaya | why do you have to rename windows files | 16:37 |
| *** LaT0rtue has joined #openstack-infra | 16:37 | |
| *** Nikolay_St has joined #openstack-infra | 16:37 | |
| clarkb | anteaya: the patch above for adding windows support to requirements does this | 16:37 |
| *** moravec1 has quit IRC | 16:37 | |
| ociuhandu | clarkb: it’s not a simple rename there, it’s trying to rename one file with the name of an already existing one | 16:37 |
| clarkb | anteaya: I was just reading it out of curiousity | 16:38 |
| clarkb | ociuhandu: yes that is a very common pattern and iirc on linux you get an atomic rename on the same fs | 16:38 |
| clarkb | which is super handy | 16:38 |
| anteaya | clarkb: ah, now I understand | 16:38 |
| anteaya | sorry thought it had something to do with nodepool | 16:38 |
| *** moravec has quit IRC | 16:38 | |
| *** yaume has joined #openstack-infra | 16:39 | |
| *** lakshmiS has joined #openstack-infra | 16:39 | |
| zigo | clarkb: Hi there! | 16:40 |
| clarkb | zigo: hello | 16:40 |
| zigo | clarkb: Could you review this trivial patch? https://review.openstack.org/#/c/302610/ | 16:40 |
| zigo | clarkb: I believe that's the last one before we can actually build a first package. | 16:41 |
| ociuhandu | clarkb: I know, but on windows python throws an error that the target already exists. Does not look like a windows issue, rather the python implementation | 16:41 |
| openstackgerrit | Merged openstack-infra/subunit2sql: Fix lock range, again https://review.openstack.org/302515 | 16:41 |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Add experimental devstack-jobs for ubuntu-xenial https://review.openstack.org/302949 | 16:42 |
| *** mylu has joined #openstack-infra | 16:42 | |
| *** e0ne has quit IRC | 16:42 | |
| pabelanger | clarkb: I believe that guess what you asked for ^ | 16:42 |
| *** shashank_hegde has quit IRC | 16:43 | |
| clarkb | zigo: I am not sure if that is sufficient but we can try it (I do not know if jenkins creates a new login shell for each shell builder) | 16:43 |
| clarkb | zigo: is that prefered over newgrp? | 16:43 |
| *** daemontool has quit IRC | 16:44 | |
| *** ddieterly[away] is now known as ddieterly | 16:46 | |
| *** xyang1 has quit IRC | 16:47 | |
| *** nwkarsten has quit IRC | 16:48 | |
| *** annegent_ has quit IRC | 16:49 | |
| *** amitgandhinz has quit IRC | 16:49 | |
| *** pblaho has quit IRC | 16:49 | |
| *** nwkarsten has joined #openstack-infra | 16:49 | |
| *** yaume has quit IRC | 16:49 | |
| *** annegentle has joined #openstack-infra | 16:49 | |
| *** pblaho has joined #openstack-infra | 16:51 | |
| *** rcernin has quit IRC | 16:52 | |
| *** moravec2 has quit IRC | 16:53 | |
| *** moravec has joined #openstack-infra | 16:53 | |
| *** julim has joined #openstack-infra | 16:53 | |
| *** nwkarsten has quit IRC | 16:53 | |
| *** darvon has quit IRC | 16:54 | |
| *** ZZelle_ has joined #openstack-infra | 16:55 | |
| *** pfallenop has quit IRC | 16:55 | |
| *** pfallenop has joined #openstack-infra | 16:56 | |
| *** darvon has joined #openstack-infra | 16:57 | |
| *** cloudtrainme has quit IRC | 16:57 | |
| *** akaszuba has quit IRC | 16:57 | |
| *** abregman has quit IRC | 16:57 | |
| *** angdraug has quit IRC | 16:58 | |
| *** dprince has quit IRC | 16:58 | |
| *** dprince has joined #openstack-infra | 16:59 | |
| openstackgerrit | Armando Migliaccio proposed openstack/requirements: Tempest 11.0.0 is released https://review.openstack.org/302958 | 16:59 |
| *** stevebaker has quit IRC | 17:00 | |
| *** oanson has joined #openstack-infra | 17:01 | |
| *** jistr has quit IRC | 17:01 | |
| *** mylu has quit IRC | 17:02 | |
| *** xyang1 has joined #openstack-infra | 17:02 | |
| *** jamesmcarthur_ has joined #openstack-infra | 17:02 | |
| *** jamesmcarthur has quit IRC | 17:03 | |
| *** jamesmcarthur_ is now known as jamesmcarthur | 17:03 | |
| *** rossella_s has quit IRC | 17:03 | |
| *** mylu has joined #openstack-infra | 17:03 | |
| *** esker has joined #openstack-infra | 17:03 | |
| *** rossella_s has joined #openstack-infra | 17:03 | |
| *** nadya has quit IRC | 17:03 | |
| *** vgridnev has quit IRC | 17:04 | |
| *** ramishra has quit IRC | 17:05 | |
| *** yamahata has quit IRC | 17:06 | |
| *** david-lyle has joined #openstack-infra | 17:06 | |
| *** oanson has quit IRC | 17:06 | |
| *** yamamoto has joined #openstack-infra | 17:07 | |
| *** pcaruana has joined #openstack-infra | 17:08 | |
| *** ddieterly is now known as ddieterly[away] | 17:08 | |
| *** BobBall is now known as BobBall_AWOL | 17:10 | |
| *** david-lyle has quit IRC | 17:11 | |
| amrith | hi folks, is the gerritbot down? | 17:11 |
| *** salv-orlando has joined #openstack-infra | 17:12 | |
| *** mat128 has joined #openstack-infra | 17:13 | |
| *** ddieterly[away] is now known as ddieterly | 17:13 | |
| *** annegentle has quit IRC | 17:14 | |
| *** oanson has joined #openstack-infra | 17:14 | |
| *** MarkAtwood has joined #openstack-infra | 17:14 | |
| *** annegentle has joined #openstack-infra | 17:14 | |
| *** annegentle has quit IRC | 17:15 | |
| *** flepied1 has quit IRC | 17:15 | |
| *** annegentle has joined #openstack-infra | 17:15 | |
| *** salv-orlando has quit IRC | 17:15 | |
| *** yamamoto has quit IRC | 17:16 | |
| *** baoli has quit IRC | 17:16 | |
| *** migi has joined #openstack-infra | 17:16 | |
| *** baoli has joined #openstack-infra | 17:17 | |
| *** fitoduarte has joined #openstack-infra | 17:17 | |
| *** david-lyle has joined #openstack-infra | 17:17 | |
| *** jamesmcarthur has quit IRC | 17:18 | |
| *** briancline has quit IRC | 17:18 | |
| *** moravec has quit IRC | 17:19 | |
| *** xarses has quit IRC | 17:20 | |
| *** kushal has quit IRC | 17:20 | |
| *** cdent has quit IRC | 17:20 | |
| pabelanger | amrith: openstackgerrit should we working | 17:20 |
| *** fitoduarte has quit IRC | 17:20 | |
| *** briancline has joined #openstack-infra | 17:20 | |
| sc` | i recently submitted a change and openstackgerrit didn't call it out | 17:20 |
| sc` | within the last 5 minutes or so | 17:20 |
| sc` | i saw it pop up in the zuul status page, so it's enqueued | 17:21 |
| amrith | pabelanger, thx | 17:21 |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci: Dump end of logs to console.log on failure https://review.openstack.org/302999 | 17:22 |
| sc` | there it is again | 17:22 |
| pabelanger | glitch in the matrix | 17:22 |
| sc` | i knew i saw my black cat cross my doorway twice this morning | 17:22 |
| *** cschwede has quit IRC | 17:23 | |
| *** baoli has quit IRC | 17:23 | |
| *** david-lyle has quit IRC | 17:23 | |
| *** amitgandhinz has joined #openstack-infra | 17:24 | |
| SpamapS | hm | 17:24 |
| *** esikachev has quit IRC | 17:24 | |
| SpamapS | I can't seem to pip install tempest using constraints | 17:24 |
| *** trown|lunch is now known as trown | 17:24 | |
| *** moravec has joined #openstack-infra | 17:25 | |
| SpamapS | http://paste.openstack.org/show/493381/ | 17:25 |
| SpamapS | ^^ | 17:25 |
| SpamapS | lifeless: ^ does that error make much sense to you? | 17:25 |
| *** slagle has quit IRC | 17:26 | |
| *** dizquierdo has quit IRC | 17:26 | |
| SpamapS | mordred: ^ pbr/pip thing confusing me | 17:26 |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci: TEST: Delete the overcloud when finished https://review.openstack.org/297328 | 17:27 |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci: Test overcloud deletion with net-iso in periodic job https://review.openstack.org/296765 | 17:27 |
| openstackgerrit | Merged openstack-infra/project-config: Switch ironic-inspector jobs to iPXE https://review.openstack.org/302172 | 17:28 |
| *** flongo has quit IRC | 17:29 | |
| *** sambetts is now known as sambetts|afk | 17:30 | |
| SpamapS | mtreinish: you around? | 17:31 |
| SpamapS | mtreinish: I'm trying to figure out why tempest is at 11.0.0 in HEAD, but ===10.0.0 in upper-constraints | 17:32 |
| *** rcernin has joined #openstack-infra | 17:33 | |
| *** tqtran has joined #openstack-infra | 17:33 | |
| *** moravec has quit IRC | 17:35 | |
| *** fitoduarte has joined #openstack-infra | 17:36 | |
| *** ddieterly is now known as ddieterly[away] | 17:36 | |
| greghaynes | SpamapS: 11.0 uploaded on 2016-04-07 | 17:36 |
| greghaynes | SpamapS: so the update just hasnt been run yet | 17:36 |
| greghaynes | https://review.openstack.org/#/c/302539/ was the most recent | 17:37 |
| *** pvaneck has joined #openstack-infra | 17:37 | |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci: Add undercloud idempotency test https://review.openstack.org/279218 | 17:38 |
| SpamapS | greghaynes: oh, so you're thinking tempest hasn't actually landed a patch since they tagged at 11.0.0 ? | 17:38 |
| SpamapS | ergo, the constraints are still at 10 | 17:38 |
| SpamapS | that's kind of a weird side-effect of release day | 17:38 |
| greghaynes | SpamapS: or that they dont test that they work with constraints in their gate | 17:38 |
| clarkb | constraints is updated by a daily job or manual updates | 17:38 |
| AJaeger | SpamapS, greghaynes: We update upper-constraints once every day - via a periodic job that gets queued around 6:00 UTC | 17:39 |
| clarkb | that ^ | 17:39 |
| AJaeger | and then it needs to be approved and merged in | 17:39 |
| AJaeger | Yeah, as clarkb says | 17:39 |
| SpamapS | clarkb: OH! I thought it was updated every time the gate passed. | 17:39 |
| AJaeger | greghaynes, SpamapS : So, that job was not run. You can update manually if you need it now - propose a change for requirmenets repo... | 17:39 |
| greghaynes | nope, openstack poposal bot | 17:39 |
| SpamapS | GOT IT | 17:39 |
| SpamapS | TIL | 17:39 |
| AJaeger | SpamapS: proposal bot and review that everything still works | 17:39 |
| clarkb | fungi I am about to run that errand now should be back in 45 minutes ot so | 17:40 |
| *** degorenko is now known as _degorenko|afk | 17:40 | |
| *** BassT has joined #openstack-infra | 17:40 | |
| SpamapS | it's odd though, that the constraint not bumping, makes git uninstallable | 17:40 |
| greghaynes | theres already a requirements patch up to do that https://review.openstack.org/#/c/302958/ | 17:40 |
| greghaynes | and in theory that should fail | 17:41 |
| greghaynes | with the error we are seeing | 17:41 |
| SpamapS | well.. no | 17:41 |
| SpamapS | greghaynes: no it should pass! | 17:41 |
| *** ihrachys has quit IRC | 17:41 | |
| SpamapS | because it has 11.0.0 in upper-constraints | 17:41 |
| fungi | clarkb: cool, i'm around now and catching up so should be ready to collaborate on nodepoolectomy shortly | 17:41 |
| fungi | clarkb: i _think_ alien cleanup will just "happen" | 17:42 |
| fungi | we haven't namespaced it | 17:42 |
| *** lezbar has joined #openstack-infra | 17:42 | |
| *** lezbar__ has quit IRC | 17:42 | |
| clarkb | fungi great, for prep we do need to reinstall nodepool to the older commit, fix the nodepool.yaml yo use zuul.o.o for geard, and stop my geard and nodepool builder | 17:42 |
| clarkb | fungi images too? uh that may be a ptoblrm of my uploaded images were deleted :/ | 17:43 |
| pabelanger | fungi: if your review queue is clear: https://review.openstack.org/#/c/302909/ will help with centos-7 testing for kolla | 17:43 |
| greghaynes | SpamapS: ah, right. We just hit a narrow window where the two didnt match | 17:43 |
| clarkb | fungi can you see if yhe new images are still in the clouds? | 17:43 |
| SpamapS | greghaynes: yeah I wonder what other projects we might hit such a window on | 17:43 |
| pabelanger | AJaeger: https://review.openstack.org/#/c/302949/ adds experimental ubuntu-xenial jobs for devstack-gate | 17:43 |
| greghaynes | SpamapS: exactly my thought :/ | 17:44 |
| SpamapS | so, cherry picking that in, doesn't fix it | 17:44 |
| SpamapS | Could not satisfy constraints for 'tempest': installation from path or url cannot be constrained to a version | 17:44 |
| greghaynes | SpamapS: Yea, so there might be something else going on | 17:44 |
| AJaeger | pabelanger: LGTM | 17:45 |
| SpamapS | greghaynes: that error in particular smells wrong | 17:45 |
| SpamapS | I wonder if I have to add a #egg thing | 17:45 |
| fungi | clarkb: i don't think it does alien image deletes, just instances | 17:46 |
| fungi | we can cleanup images though, yes | 17:46 |
| *** shashank_hegde has joined #openstack-infra | 17:46 | |
| fungi | clarkb: checking to be sure, yes | 17:46 |
| SpamapS | nope, #egg=tempest doesn't help | 17:46 |
| *** jordanP has quit IRC | 17:46 | |
| fungi | SpamapS: the idea with constraints was that if having a newer release of something reflected in constraints was urgent, someone would manually propose that update and get it fast-tracked through review | 17:47 |
| krotscheck | jtomasek: That npm prune fix merged, can you recheck tripleo-ui to see if your issue's been resolved? | 17:47 |
| AJaeger | krotscheck, jtomasek let's wait 60 mins after merge so that all jenkins are updated before you recheck | 17:48 |
| SpamapS | fungi: ACK. | 17:48 |
| krotscheck | AJaeger: Ok, so another 33 minutes | 17:48 |
| SpamapS | greghaynes: so, at worst, given the way we're doing CD, it means our CI stops testing things until the proposal bot does it. | 17:48 |
| fungi | SpamapS: so... if it's urgent for you, then go for it. there are no taboos with proposing constraints bumps manually | 17:48 |
| SpamapS | but, I still can't install tempest | 17:48 |
| SpamapS | fungi: there's already one proposed | 17:48 |
| SpamapS | queued for 43 min ;) | 17:48 |
| SpamapS | or, 50 now | 17:48 |
| SpamapS | things are .. a bit backed up ;) | 17:49 |
| greghaynes | SpamapS: Well, you mentioned that pulling in that change locally didnt fix it? | 17:49 |
| SpamapS | greghaynes: right | 17:49 |
| SpamapS | so I suspect something else | 17:49 |
| AJaeger | krotscheck: 60 mins is my rule of thumb, could be faster - and if it takes longer, we have a problem ;) | 17:49 |
| AJaeger | krotscheck: is https://review.openstack.org/302900 still needed? | 17:50 |
| *** SumitNaiksatam has joined #openstack-infra | 17:51 | |
| *** lakshmiS has quit IRC | 17:52 | |
| krotscheck | AJaeger Looks like it's updated everywhere except jenkins02 | 17:52 |
| *** esker has quit IRC | 17:52 | |
| krotscheck | AJaeger I... looking at the console, I can't actually tell why that failed. | 17:52 |
| mtreinish | SpamapS: right, I just pushed the release a couple of hours ago | 17:53 |
| *** e0ne has joined #openstack-infra | 17:53 | |
| AJaeger | wow, 270 changes in check queue and 99 in post - our systems are busy | 17:53 |
| mtreinish | I don't believe any commits have landed since then | 17:53 |
| AJaeger | pabelanger: could you check whether jenkins02 needs a manual JJB run, please? See krotscheck's comment above | 17:53 |
| mtreinish | SpamapS: fwiw: https://review.openstack.org/#/c/302958/ | 17:54 |
| krotscheck | AJaeger: It won't hurt anything. Let me ponder whether it makes logical sense... | 17:54 |
| *** ifarkas has quit IRC | 17:55 | |
| *** maishsk_ has joined #openstack-infra | 17:55 | |
| *** kzaitsev_mb has quit IRC | 17:55 | |
| *** david-lyle has joined #openstack-infra | 17:56 | |
| pabelanger | AJaeger: sure | 17:56 |
| *** david-lyle has quit IRC | 17:56 | |
| krotscheck | AJaeger: So, the purpose of running npm shrinkwrap in the gate is basically just to tell us which versions a package was being built with. The shrinkwrap file, if already included, basically puts the onus of generating that file on the project itself. I see no reason why adding a switch would harm anything. | 17:56 |
| *** maishsk has quit IRC | 17:56 | |
| *** maishsk_ is now known as maishsk | 17:56 | |
| *** xarses has joined #openstack-infra | 17:56 | |
| krotscheck | AJaeger: I did downvote it for other reasons though | 17:57 |
| fungi | the answer to "is jenkins02 broken?" is almost always "yes" | 17:57 |
| *** david-lyle has joined #openstack-infra | 17:58 | |
| fungi | it seems to be the one most frequently hit by jenkins api timeouts, which sometimes causes jjb updates not to complete, which causes them to pile up more often, which means it eventually gets bad enough they exceed the execution timeout from our automation | 17:58 |
| *** vgridnev has joined #openstack-infra | 17:58 | |
| openstackgerrit | Merged openstack-infra/project-config: Add experimental devstack-jobs for ubuntu-xenial https://review.openstack.org/302949 | 17:59 |
| fungi | after that, it pretty much always requires some manual intervention | 17:59 |
| fungi | both jenkins01 and jenkins02 could stand to be rebuilt on newer nova flavors | 17:59 |
| *** dprince has quit IRC | 18:00 | |
| *** vikram has quit IRC | 18:00 | |
| *** dprince has joined #openstack-infra | 18:00 | |
| *** cloudtrainme has joined #openstack-infra | 18:01 | |
| *** jsavak has quit IRC | 18:01 | |
| *** jamesmcarthur has joined #openstack-infra | 18:02 | |
| *** jsavak has joined #openstack-infra | 18:02 | |
| *** e0ne has quit IRC | 18:03 | |
| *** julim has quit IRC | 18:04 | |
| *** ociuhandu has quit IRC | 18:04 | |
| *** piet has quit IRC | 18:06 | |
| *** jcooley_ has quit IRC | 18:06 | |
| openstackgerrit | amrith proposed openstack/requirements: Update upper-constraints for python-troveclient v2.2.0 https://review.openstack.org/292652 | 18:07 |
| pabelanger | AJaeger: re-running JJB on jenkins02 | 18:07 |
| *** moravec has joined #openstack-infra | 18:08 | |
| *** mrmartin has quit IRC | 18:09 | |
| greghaynes | SpamapS: yea, so as I read this it has nothing to do with the current constraints not matching | 18:10 |
| *** shardy has quit IRC | 18:12 | |
| AJaeger | thanks, pabelanger | 18:12 |
| greghaynes | SpamapS: ok, so services are not listed in constraints | 18:12 |
| greghaynes | SpamapS: We are installing tempest from a path, that means it cant determine if that patch fulfills the constraint | 18:12 |
| SpamapS | greghaynes: ok, so we have to use not-constraints for that | 18:13 |
| *** e0ne has joined #openstack-infra | 18:13 | |
| greghaynes | SpamapS: yep | 18:13 |
| SpamapS | greghaynes: kinda sux | 18:13 |
| SpamapS | but ok | 18:13 |
| greghaynes | SpamapS: or we have to remove tempest from our constraints when we use constraints | 18:13 |
| *** dprince has quit IRC | 18:13 | |
| *** rajinir has joined #openstack-infra | 18:13 | |
| *** sridhar_ram has joined #openstack-infra | 18:14 | |
| *** e0ne has quit IRC | 18:14 | |
| *** dprince has joined #openstack-infra | 18:15 | |
| *** annegentle has quit IRC | 18:16 | |
| *** annegentle has joined #openstack-infra | 18:16 | |
| openstackgerrit | Mark Vanderwiel proposed openstack-infra/project-config: Enable lbaasv2 jobs in check and gate https://review.openstack.org/303017 | 18:17 |
| fungi | clarkb: a spot check says that the new nodepool's uuid for the ubuntu-trusty image in rax-dfw appears in nova image-list output there, so we should be fine wrt it not "cleaning up" our images from the new server | 18:18 |
| fungi | though the nodepool alien-image-list cli subcommand looks like it may be broken | 18:19 |
| *** ociuhandu has joined #openstack-infra | 18:21 | |
| pabelanger | AJaeger: krotscheck: and done | 18:22 |
| krotscheck | pabelanger: Cool, we'll check. | 18:22 |
| *** oanson has quit IRC | 18:24 | |
| *** sdake_ has joined #openstack-infra | 18:26 | |
| *** doug-fish has joined #openstack-infra | 18:26 | |
| *** sdake has quit IRC | 18:26 | |
| jtomasek | krotscheck: ack | 18:27 |
| krotscheck | jtomasek: link to job so I can watch it? | 18:27 |
| krotscheck | jtomasek: oh. 250 things in the gate. Weeee | 18:28 |
| jtomasek | krotscheck: http://status.openstack.org//zuul/ change id 301137 | 18:28 |
| *** nwkarsten has joined #openstack-infra | 18:30 | |
| sdague | there was also a fixtures release that ... made a lot of things unhappy | 18:30 |
| *** esker has joined #openstack-infra | 18:32 | |
| *** salv-orlando has joined #openstack-infra | 18:32 | |
| *** cody-somerville has joined #openstack-infra | 18:33 | |
| *** tonytan4ever has quit IRC | 18:34 | |
| *** tonytan4ever has joined #openstack-infra | 18:34 | |
| *** ddieterly[away] has quit IRC | 18:36 | |
| *** amitgandhinz has quit IRC | 18:37 | |
| *** salv-orlando has quit IRC | 18:38 | |
| *** jsavak has quit IRC | 18:38 | |
| *** nihilifer has quit IRC | 18:39 | |
| *** slagle has joined #openstack-infra | 18:39 | |
| *** jsavak has joined #openstack-infra | 18:40 | |
| *** cdent has joined #openstack-infra | 18:40 | |
| openstackgerrit | Mark Vanderwiel proposed openstack-infra/project-config: Enable lbaasv2 jobs into check queue https://review.openstack.org/303017 | 18:40 |
| *** ddieterly has joined #openstack-infra | 18:42 | |
| *** akshai has joined #openstack-infra | 18:42 | |
| *** sambetts|afk has quit IRC | 18:42 | |
| *** dprince has quit IRC | 18:42 | |
| *** dprince has joined #openstack-infra | 18:43 | |
| clarkb | fungi: I am back now, I will work on getting the right version of nodepool installed, fix the config file, and stop the services I had started to get images up | 18:44 |
| *** angdraug has joined #openstack-infra | 18:44 | |
| *** ddieterly is now known as ddieterly[away] | 18:44 | |
| fungi | right version of nodepool? did we have the wrong version on the new server? | 18:45 |
| fungi | anyway, not sure if you saw in scrollback, but your new images are safe and sound | 18:45 |
| clarkb | afdd58c10a8b85ea1466d32c3a29c0d7a1b0aea9 is the sha1 I will be installing | 18:45 |
| clarkb | fungi: the issue is the use shade for all interactions doesn't work yet and has not been reverted | 18:46 |
| clarkb | so old nodepool was running a manually installed older version (the sha above I think but I cannot confirm on old server) | 18:46 |
| clarkb | fungi: I did ty | 18:46 |
| openstackgerrit | Paul Belanger proposed openstack-infra/devstack-gate: Test commit https://review.openstack.org/303032 | 18:46 |
| *** sambetts has joined #openstack-infra | 18:46 | |
| *** esker has quit IRC | 18:46 | |
| *** ddieterly[away] is now known as ddieterly | 18:46 | |
| fungi | oh, got it. i though we had officially reverted again | 18:47 |
| fungi | not just manually rolled the server back to an older commit | 18:47 |
| fungi | s/though/thought/ | 18:47 |
| clarkb | nodepool==0.2.1.dev71 # git sha afdd58c so thats done | 18:48 |
| clarkb | fungi: thats it for prep I think | 18:49 |
| clarkb | now its just a matter of shutting down the old server and updating dns | 18:49 |
| clarkb | then whatever cleanup we need to do | 18:49 |
| *** annegentle has quit IRC | 18:49 | |
| clarkb | I will get on puppet master to do the nova shutdown | 18:49 |
| *** annegentle has joined #openstack-infra | 18:50 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config: Add support for ubuntu-xenial wheel mirror https://review.openstack.org/303033 | 18:51 |
| *** SumitNaiksatam has left #openstack-infra | 18:51 | |
| clarkb | fungi: are you ready? do you want to get the dns updates? | 18:52 |
| clarkb | I didn't set up reverse records for new server yet eitehr | 18:52 |
| *** salv-orlando has joined #openstack-infra | 18:52 | |
| *** annegentle has quit IRC | 18:52 | |
| fungi | clarkb: sure, i can do dns updates (forward and reverse). i expect we're going to set zuul back a bit since it's got a backlog and we're currently pegging all our quotas | 18:53 |
| clarkb | fungi: do you think we should wait for zuul to catch up? | 18:53 |
| fungi | jobs running on machines which get alien-deleted are hopefully going to restart automatically | 18:53 |
| clarkb | nodepool is running on the other server we will just have to do this if anything goes wrong | 18:53 |
| clarkb | fungi: yup they should | 18:53 |
| openstackgerrit | Matt Riedemann proposed openstack/requirements: Temporarily cap fixtures<2.0 https://review.openstack.org/303034 | 18:54 |
| mriedem | sdague: dims: ^ | 18:54 |
| clarkb | mostly want ot get this done before it becomes pressing in an emergency, there may not be a good time at all so just doing it is best | 18:54 |
| fungi | yeah, let's just go for it | 18:54 |
| *** jsavak has quit IRC | 18:54 | |
| fungi | i think it'll be transparent | 18:54 |
| clarkb | agreed | 18:55 |
| dims | mriedem : +2, sdague : can you please +2A? | 18:55 |
| clarkb | fungi: lte me know when you are ready to start the dns updates and I will nova stop the old server | 18:55 |
| sdague | dims done | 18:55 |
| sdague | fungi: there is currently no integrated gate | 18:55 |
| fungi | sdague: yeah, just a ton of check/post | 18:55 |
| sdague | fungi: as such, would you mind promoting https://review.openstack.org/303034 to gate | 18:55 |
| sdague | as it won't stomp out anything | 18:56 |
| fungi | sdague: once we're done replacing nodepool | 18:56 |
| sdague | ok | 18:56 |
| dims | fungi : 2 projects affected :) | 18:56 |
| *** chem has quit IRC | 18:56 | |
| sdague | dims: well, a bunch of neutron projects, so it's more than 2 | 18:56 |
| dims | true | 18:56 |
| clarkb | fungi: we may need to service iptables restart to pick up the changed ip addr on zuul.o.o and jenkins masters | 18:57 |
| *** jsavak has joined #openstack-infra | 18:57 | |
| zigo | clarkb: newgrp? I'm not sure, here, it's sbuild stuff, it's a bit special, I'm not really sure how it works. On my Jenkins server, I really do have to logout and login again (ie: new ssh session). Others in infra told me doing another shell stuff will do another ssh session. Let's try, as you wrote... | 18:57 |
| fungi | clarkb: addresses updated in dns to 23.253.73.160 and 2001:4800:7815:103:be76:4eff:fe06:95e2 just now | 18:57 |
| zigo | (and thanks for your +2) | 18:57 |
| clarkb | fungi: ok stopping old server now | 18:57 |
| zigo | fungi: Can you have a look as well? https://review.openstack.org/#/c/302610/ | 18:57 |
| *** hashar has joined #openstack-infra | 18:58 | |
| fungi | zigo: i have a bit of a review backlog which i'm already hoping to find time for after we finish this maintenance | 18:58 |
| fungi | but i'll see | 18:58 |
| clarkb | ERROR (CommandError): Unable to stop the specified server(s). | 18:58 |
| openstackgerrit | Roland Hochmuth proposed openstack-infra/project-config: Add openstack/monasca-transform project https://review.openstack.org/300742 | 18:58 |
| clarkb | There is no such action: os-stop (HTTP 400) | 18:58 |
| zigo | fungi: it's a tiny small patch... :) | 18:58 |
| clarkb | something something interop | 18:58 |
| * clarkb looks into how to actually stop this thing | 18:58 | |
| zigo | fungi: Just dong 2 - shell: instead of one, to get a new login session after sbuild is installed. | 18:59 |
| zigo | doing | 18:59 |
| fungi | clarkb: there's "nova stop" | 18:59 |
| fungi | i assume that's the same thing openstack server stop is trying to call though | 18:59 |
| clarkb | fungi: ya nova stop is what I ran | 19:00 |
| *** sambetts has quit IRC | 19:00 | |
| clarkb | looks like we have suspend and pause as options too | 19:00 |
| *** e0ne has joined #openstack-infra | 19:00 | |
| *** ddieterly is now known as ddieterly[away] | 19:00 | |
| clarkb | and shelve | 19:00 |
| clarkb | so we have 4 different verbs | 19:00 |
| clarkb | at least one doesn't work | 19:00 |
| clarkb | sdague: ^ is there a recommended one for "turn off this server" | 19:01 |
| *** salv-orlando has quit IRC | 19:01 | |
| fungi | clarkb: in the rackspace webui, the only thing that looks likely to "stop" the server is delete | 19:01 |
| sdague | clarkb: what are you going to do with the server after you turn it off? | 19:01 |
| sdague | because, delete is the normal workflow | 19:01 |
| fungi | sdague: possibly turn it back on if this doesn't go well | 19:01 |
| clarkb | sdague: probably will delete it, but want to have the disk around just in case we need to mount it forsome reason | 19:01 |
| sdague | normally I'd then suggest snapshot + delete | 19:02 |
| *** esikachev has joined #openstack-infra | 19:02 | |
| fungi | i suppose we could snapshot it | 19:02 |
| fungi | yeah, that | 19:02 |
| clarkb | wow | 19:02 |
| clarkb | so as a user thats broken | 19:02 |
| clarkb | we have 4 verbs that are stop this thing without deleting it | 19:02 |
| clarkb | but snapshot and delete is what we need to do... | 19:02 |
| sdague | well, you have a provider that didn't implement some of them | 19:03 |
| fungi | well, pause and suspend and shelve will... probably not work in rax | 19:03 |
| sdague | because, openstack was built of extensions | 19:03 |
| clarkb | ya I don't know that its necessarily nova's fault just weird in general | 19:03 |
| clarkb | fungi: I will work on snapshotting it | 19:03 |
| sdague | shelve would probably also be fine | 19:03 |
| *** sambetts has joined #openstack-infra | 19:03 | |
| sdague | that kind of does snapshot + stop | 19:03 |
| clarkb | looks like we have multiple snapshot verbs too | 19:04 |
| sdague | and I agree, the verbs are all over the place | 19:04 |
| fungi | it's possible rax supports shelve but doesn't expose it in their dashboard | 19:04 |
| clarkb | 'backup' and 'image-create' | 19:04 |
| *** shu-mutou-AFK has quit IRC | 19:04 | |
| sdague | but, that all happened before I got here | 19:04 |
| * clarkb does image-create | 19:04 | |
| sdague | fungi: that is possible, I know it's accessable to rax people | 19:04 |
| clarkb | should I try shelving as a user? | 19:05 |
| clarkb | maybe I will try that first | 19:05 |
| fungi | clarkb: worth a shot | 19:05 |
| *** esker has joined #openstack-infra | 19:05 | |
| *** esker has quit IRC | 19:05 | |
| fungi | clarkb: one difference i've spotted | 19:05 |
| *** esker has joined #openstack-infra | 19:06 | |
| clarkb | ERROR (BadRequest): There is no such action: shelve (HTTP 400) | 19:06 |
| fungi | clarkb: the old server's cinder volume used ssd, new one is sata | 19:06 |
| openstackgerrit | Armando Migliaccio proposed openstack-infra/project-config: Update Gerrit ACLs for vendor specific networking-* projects https://review.openstack.org/303039 | 19:06 |
| clarkb | fungi: oh | 19:06 |
| clarkb | does openstack volume list specify that difference? I might have just missed it | 19:06 |
| fungi | i know we switched to ssd at one point because of performance concerns, not sure if they are still concerns | 19:06 |
| clarkb | fungi: based on build times yesterday I don't think it is an issue | 19:06 |
| fungi | i expect that if you didn't see poor performance during image updates, then sata is good enough, yes | 19:07 |
| *** nikhil has quit IRC | 19:07 | |
| clarkb | if it persists it is lvm'd and we can swap it around and do the volume mv thing | 19:07 |
| fungi | absolutely, so let's ignore that for now | 19:07 |
| clarkb | fungi: most of our time is spent uploading not building | 19:07 |
| clarkb | ok image-creating now | 19:07 |
| sdague | "openstack volume type list" is where you'd find details out about it I think | 19:07 |
| openstackgerrit | Vitaly Kramskikh proposed openstack-infra/project-config: Do not run npm shrinkwrap if npm-shrinkwrap.json already exists https://review.openstack.org/302900 | 19:07 |
| fungi | i added reverse dns for the new server now too | 19:07 |
| clarkb | really we need bigger pipes to rax-ord and ovh-bhs if we want to speedup the entire image lifecycle | 19:09 |
| *** jcooley_ has joined #openstack-infra | 19:09 | |
| clarkb | it is "SAVING" | 19:10 |
| *** thiagop_ has joined #openstack-infra | 19:10 | |
| *** thiagop_ has quit IRC | 19:10 | |
| *** jcooley_ has quit IRC | 19:11 | |
| *** nikhil has joined #openstack-infra | 19:11 | |
| *** hdd has joined #openstack-infra | 19:12 | |
| clarkb | I have no idea how long this will take, in theory not too bad since we used performance flavors will smaller / | 19:13 |
| clarkb | fungi: image is called nodepool-no-ssh-host-keys | 19:13 |
| *** david-lyle_ has joined #openstack-infra | 19:13 | |
| fungi | it's giving me time to catch up on the "non-atc recognition working group" meeting | 19:13 |
| *** vgridnev has quit IRC | 19:14 | |
| *** salv-orlando has joined #openstack-infra | 19:14 | |
| clarkb | what if it somehow snapshots the half terabyte /opt | 19:15 |
| *** david-lyle has quit IRC | 19:15 | |
| * clarkb hopes it is smarter than that | 19:15 | |
| *** bkeller` has joined #openstack-infra | 19:16 | |
| *** sigmavirus24 is now known as sigmavirus24_awa | 19:17 | |
| fungi | it should only snapshot the rootfs afaik | 19:17 |
| *** Sukhdev has joined #openstack-infra | 19:18 | |
| *** david-lyle_ has quit IRC | 19:19 | |
| clarkb | still saving | 19:21 |
| * clarkb reads email and tries to be patient | 19:21 | |
| *** ddieterly[away] is now known as ddieterly | 19:21 | |
| *** bpokorny_ has joined #openstack-infra | 19:23 | |
| *** ihrachys has joined #openstack-infra | 19:24 | |
| openstackgerrit | Ryan Hallisey proposed openstack-infra/tripleo-ci: Allow the container job to run again https://review.openstack.org/288915 | 19:24 |
| *** briancline has quit IRC | 19:24 | |
| *** briancline has joined #openstack-infra | 19:25 | |
| *** bpokorny has quit IRC | 19:26 | |
| *** jaosorior has quit IRC | 19:27 | |
| *** lunarlamp has joined #openstack-infra | 19:28 | |
| *** david-lyle has joined #openstack-infra | 19:29 | |
| *** bkeller` has quit IRC | 19:30 | |
| *** david-lyle has quit IRC | 19:31 | |
| *** david-lyle has joined #openstack-infra | 19:31 | |
| *** Jeffrey4l_ has joined #openstack-infra | 19:32 | |
| clarkb | still going maybe I should find lunch | 19:33 |
| *** julim has joined #openstack-infra | 19:33 | |
| pvaneck | fungi: hey, I wanted to revisit this patch you submitted way back for enabling the mysql-backup cron job for refstack if you have a moment: https://review.openstack.org/#/c/254426 | 19:33 |
| *** Jeffrey4l has quit IRC | 19:34 | |
| pvaneck | Seeing your last comment in that review, I submitted a patch to puppet-refstack https://review.openstack.org/#/c/301405 , but yolanda makes the point that these cron jobs are typically declared in system-config. | 19:34 |
| clarkb | yes, I think the idea behind putting backup info in system-config is that it is site specific and not application specific | 19:34 |
| *** arxcruz has quit IRC | 19:35 | |
| fungi | clarkb: pvaneck: the reason it ended up being not straightforward for refstack is that it has no dedicated puppet class in openstack_project | 19:35 |
| clarkb | fungi: ah | 19:35 |
| fungi | it's just a node declaration in the global site manifest | 19:35 |
| *** tonytan_brb has joined #openstack-infra | 19:35 | |
| nibalizer | pabelanger: around? | 19:35 |
| *** mylu has quit IRC | 19:35 | |
| pabelanger | nibalizer: I am | 19:35 |
| nibalizer | puppetdb? | 19:36 |
| pabelanger | nibalizer: we should probably wait until new nodepool.o.o is online | 19:36 |
| *** tonytan4ever has quit IRC | 19:36 | |
| fungi | so to do it like "most" servers we need a shim class added to openstack_project in the system-config repo that instantiates the refstack module and also instantiates the mysql_backup::backup_remote class | 19:36 |
| pabelanger | nibalizer: so, likely next monday now | 19:36 |
| clarkb | fungi: or we could put it in the site.pp | 19:36 |
| clarkb | that starts to get ugly though | 19:36 |
| nibalizer | pabelanger: wfm | 19:36 |
| fungi | clarkb: well, i tried that with 254426 | 19:36 |
| pvaneck | fungi: Seems having it in system-config is the way to go. I think your patch set one is the way to do it | 19:37 |
| nibalizer | fungi: yea a shim class is generally the way to do it | 19:37 |
| nibalizer | keeps site.pp under control | 19:37 |
| nibalizer | but see a host like nodepool where it clearly didn't work | 19:37 |
| pvaneck | you were just missing an apostrophe in the Class['refstack] | 19:37 |
| *** david-lyle has quit IRC | 19:37 | |
| fungi | pvaneck: hah, so i was | 19:37 |
| *** cloudtrainme has quit IRC | 19:37 | |
| fungi | i'll push an update there--good eye | 19:37 |
| pvaneck | just tested it out locally to see how it would work and caught that | 19:38 |
| pvaneck | with that closing apostophe it works for me | 19:38 |
| clarkb | ok snapshot is ACTIVE | 19:38 |
| *** mylu has joined #openstack-infra | 19:38 | |
| clarkb | fungi: ready for me to delete the old host? | 19:38 |
| pvaneck | patchset 1 that is | 19:38 |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/system-config: Add database backups for refstack.openstack.org https://review.openstack.org/254426 | 19:39 |
| fungi | pvaneck: nibalizer: ^ | 19:39 |
| fungi | clarkb: go for it | 19:39 |
| openstackgerrit | Khai Do proposed openstack-infra/system-config: Customize Gerrit's download command and schema https://review.openstack.org/300603 | 19:39 |
| *** tonytan_brb has quit IRC | 19:39 | |
| *** baoli has joined #openstack-infra | 19:39 | |
| *** tonytan4ever has joined #openstack-infra | 19:39 | |
| clarkb | your request for deletion has been accepted | 19:40 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/shade: Update func tests for latest devstack flavors https://review.openstack.org/302767 | 19:40 |
| lifeless | SpamapS: yes, it does | 19:40 |
| fungi | clarkb: watching for ping responses to cease | 19:40 |
| lifeless | SpamapS: you need to run edit-constraints, or remove it from the constraints file | 19:40 |
| clarkb | and its gone | 19:40 |
| lifeless | SpamapS: since you're basically saying | 19:40 |
| clarkb | at leats according to nova | 19:40 |
| *** bpokorny_ has quit IRC | 19:40 | |
| lifeless | tempest == somever + tempest == someotherver | 19:41 |
| lifeless | SpamapS: and thats an impossible situation | 19:41 |
| clarkb | fungi: I get no ping repsonses, do you? | 19:41 |
| *** bpokorny has joined #openstack-infra | 19:41 | |
| fungi | clarkb: oh, hah, i was still doing it by name, which is now resolving to the new server ;) | 19:41 |
| fungi | yes, old addresses no longer respond, so i think we're good to press forward | 19:41 |
| clarkb | fungi: ok I am starting services on the new server now | 19:41 |
| *** dprince has quit IRC | 19:42 | |
| *** hdd has quit IRC | 19:42 | |
| clarkb | not seeing new image upload jobs in gearman so I think firewalls do need bouncing | 19:43 |
| fungi | clarkb: we will likely end up with a ton of defunct slaves on the jenkins masters after the alien node cleanup completes | 19:43 |
| pabelanger | greghaynes: krotscheck: when you added the wheel_mirror_slave.pp to system-config, was there a specific reason for not setting thin => true for openstack_project::slave? | 19:43 |
| clarkb | fungi: yup, I am going to bounce iptables service on zuul and the jenkinses | 19:44 |
| pabelanger | greghaynes: krotscheck: trying to see if it is safe to remove the dependency on think_slave.pp | 19:44 |
| fungi | just restart the iptables-persistent "service"? | 19:44 |
| fungi | i can do some of them | 19:44 |
| krotscheck | pabelanger: That decision was made before I took over the patches. greghaynes ? | 19:44 |
| clarkb | fungi: yup | 19:44 |
| clarkb | I just did jenkins 07 | 19:44 |
| fungi | i'll start with jenkins and 01 up | 19:45 |
| fungi | did up to 03 | 19:46 |
| *** BassT has quit IRC | 19:46 | |
| clarkb | doing zuul now | 19:46 |
| fungi | assumed you were counting downward | 19:46 |
| clarkb | there we go | 19:46 |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config: [WIP] Remove dependency on think_slave for wheel mirrors https://review.openstack.org/303047 | 19:46 |
| clarkb | starting nodepoold now | 19:46 |
| fungi | expecting eta on a bunch of changes to go back to unknown | 19:47 |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config: Add support for ubuntu-xenial wheel mirror https://review.openstack.org/303033 | 19:47 |
| clarkb | it is doing its things so as quickly as the aliens can be deleted | 19:47 |
| clarkb | probably within the netx 5 minutes or so | 19:47 |
| fungi | clarkb: okay for me to fix the cached ssh host key on puppetmaster and clear the ansible inventory cache? | 19:48 |
| pabelanger | clarkb: did we roll back nodepool to the pre-shade commit? | 19:48 |
| fungi | pabelanger: yep | 19:48 |
| pabelanger | nice | 19:48 |
| clarkb | fungi: yes I think it is | 19:49 |
| clarkb | we are hitting quota issues but that is expected until aliens go away | 19:49 |
| clarkb | and the snapshot builds have started | 19:49 |
| fungi | okay, `sudo ssh nodepool.openstack.org` works from puppetmaster.o.o now and /var/cache/ansible-inventory/ansible-inventory.cache has been truncated | 19:50 |
| nibalizer | crinkle: re the pip hax | 19:52 |
| clarkb | fungi: does nodepool cleanup aliens periodically? | 19:52 |
| fungi | yes | 19:52 |
| nibalizer | a) i think this is the right amount of hax | 19:52 |
| *** bpokorny has quit IRC | 19:52 | |
| clarkb | fungi: ah found it cleanupLeakedInstances | 19:52 |
| clarkb | fungi: which is why git grep -i alien didn't find it | 19:53 |
| *** matt-borland has quit IRC | 19:53 | |
| nibalizer | if you look at the diff here https://github.com/puppetlabs/puppet/pull/3984/files getting what we want out of the json api is actually super easy | 19:53 |
| *** bpokorny has joined #openstack-infra | 19:53 | |
| nibalizer | and dstufft told me that the json api makes him happy | 19:53 |
| fungi | not that said, i don't know if we've ever tested what happens when we spontaneously "leak" some 500-600 instances | 19:53 |
| fungi | s/not/now/ | 19:53 |
| dstufft | yes, json api is better | 19:53 |
| SpamapS | lifeless: so, it looks like on April 1, tempest got added to upper-constraints | 19:53 |
| dstufft | than the XMLRPC one anyways, we can't cache XMLRPC in Fastly | 19:53 |
| clarkb | we have building, ready and used instances so that all looks well | 19:53 |
| SpamapS | lifeless: which is what broke us ultimately. | 19:54 |
| fungi | SpamapS: sounds like an april fool's joke someone forgot to revert? | 19:54 |
| *** alex_xu has quit IRC | 19:54 | |
| clarkb | and snapshot images are building | 19:55 |
| clarkb | vexxhost images uploading | 19:55 |
| clarkb | lgtm so far | 19:55 |
| SpamapS | fungi: yeah I dunno, the commit message looks legitimate.. things depend on it to work. But at the same time, it means we can't install tempest from a git dir with constraints without making a separate constraints file without tempest in it. | 19:55 |
| *** alex_xu has joined #openstack-infra | 19:55 | |
| *** tonytan_brb has joined #openstack-infra | 19:55 | |
| clarkb | tempest itself is installed to a venv without constraints iirc | 19:56 |
| fungi | SpamapS: yep, i looked at https://review.openstack.org/300263 and it does seem legit | 19:56 |
| SpamapS | yeah, we were trying to do that _with_ constraints. | 19:56 |
| mtreinish | clarkb: right, we don't constrain tempest when we run it in the gate | 19:56 |
| clarkb | ya thats how I would do it elsewhere then | 19:56 |
| SpamapS | I'm a bit curious at why pip can't just use the version from the dir to match the constraints though. | 19:56 |
| mtreinish | we do honor constraints for running some config utils as part of the devstack run IIRC | 19:56 |
| mtreinish | https://github.com/openstack-dev/devstack/blob/master/lib/tempest#L347 | 19:57 |
| clarkb | fungi: we should check for fip leakage on osic and bluebox once things settle a bit more | 19:57 |
| lifeless | SpamapS: or editing your constraints file | 19:58 |
| lifeless | SpamapS: see devstack for examples of editing constraints | 19:58 |
| fungi | clarkb: great point | 19:58 |
| mtreinish | but that's kinda cheating because tox will make a venv unconstrained and then we pip install with constraints in the same venv after | 19:58 |
| *** tonytan4ever has quit IRC | 19:59 | |
| SpamapS | lifeless: so I wonder is it a violation of the model to allow constraining the git dir one is pip installing, or an implementation limitation? | 20:00 |
| SpamapS | I can sdist the git dir, and pip install the resulting tarball just fine using constraints. | 20:00 |
| pabelanger | nodepool.o.o doesn't appear to be logging to statsd anymore | 20:00 |
| *** yamahata has joined #openstack-infra | 20:00 | |
| lifeless | SpamapS: as long as the version matches the constraints you can | 20:00 |
| clarkb | pabelanger: oh need to kick iptables there too | 20:00 |
| lifeless | SpamapS: if the constraints don't match the version, you should get an equivalent error | 20:01 |
| clarkb | done | 20:01 |
| SpamapS | lifeless: right, which is fine. I got into a situation where once we updated constraints, it still did not work. | 20:01 |
| SpamapS | Could not satisfy constraints for 'tempest': installation from path or url cannot be constrained to a version | 20:01 |
| *** esikachev has quit IRC | 20:01 | |
| lifeless | SpamapS: pastebin the constraints file you had? | 20:01 |
| SpamapS | lifeless: https://review.openstack.org/#/c/302958/1 <-- that one | 20:02 |
| *** jtomasek has quit IRC | 20:03 | |
| *** baoli has quit IRC | 20:03 | |
| *** baoli has joined #openstack-infra | 20:04 | |
| *** jtomasek has joined #openstack-infra | 20:05 | |
| fungi | pabelanger: are you sure? i see building count spiking | 20:06 |
| greghaynes | SpamapS: lifeless: https://github.com/pypa/pip/blob/develop/pip/req/req_set.py#L262 was raising because existing_req.link.path was None and install_req.link.path was set | 20:06 |
| fungi | pabelanger: and used count falling | 20:06 |
| clarkb | fungi: I am going to start fip things | 20:06 |
| fungi | clarkb: you doing osic first? | 20:06 |
| * SpamapS doesn't give a fip | 20:06 | |
| fungi | i can take bbox if so | 20:06 |
| greghaynes | for tempest, since existing req was in constraints, and install_req was the path we were intsalling | 20:06 |
| *** lakshmiS has joined #openstack-infra | 20:07 | |
| clarkb | fungi: yup doing osic since they hav elarger quota | 20:08 |
| fungi | i've got fip cleanup underway in bbox now | 20:08 |
| *** cloudtrainme has joined #openstack-infra | 20:08 | |
| lifeless | SpamapS: thats not an edited file | 20:08 |
| clarkb | fungi: ty | 20:08 |
| lifeless | SpamapS: I want the actual file off of disk | 20:08 |
| lifeless | SpamapS: the one you say you updated | 20:08 |
| fungi | as predicted lots of stuff in check now has an unknown eta. that's good (we didn't insta-fail everything!) | 20:08 |
| lifeless | SpamapS: I want to make sure that the edit was done correctly | 20:08 |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Make sure we run bindep for all puppet-jobs https://review.openstack.org/303052 | 20:09 |
| lifeless | greghaynes: yes, thats deliberate | 20:09 |
| pabelanger | fungi: I was looking on grafana.o.o | 20:09 |
| pabelanger | EmilienM: we likely need that ^ | 20:09 |
| fungi | pabelanger: shouldn't it pull from the same sources? | 20:09 |
| pabelanger | fungi: it should, let me double check | 20:09 |
| fungi | grafana statsd i mean | 20:09 |
| clarkb | pabelanger: fungi I kicked iptables on graphite.o.o | 20:09 |
| pabelanger | fungi: okay, working now | 20:09 |
| SpamapS | lifeless: oh, I did not update one, I mean, I tried this one, and it failed. | 20:09 |
| jtomasek | krotscheck: now running https://jenkins05.openstack.org/job/gate-tripleo-ui-npm-run-test/9/console | 20:09 |
| clarkb | the ackets were getting nacked | 20:09 |
| krotscheck | jtomasek: Finally | 20:10 |
| SpamapS | lifeless: and 11.0.0 is the same version as the sdist produces | 20:10 |
| fungi | clarkb: oh, i was about to wonder if that's what we were missing | 20:10 |
| fungi | good call | 20:10 |
| EmilienM | pabelanger: excellent! | 20:10 |
| *** ddieterly is now known as ddieterly[away] | 20:10 | |
| greghaynes | SpamapS: lifeless I was reproing using upper constraints from that review (http://paste.openstack.org/show/493414/) and installing tempest from git... | 20:11 |
| SpamapS | I can still reproduce that right now | 20:11 |
| fungi | pabelanger: EmilienM: out of curiosity, what packages were missing? this is likely to preinstall a bunch more packages which weren't installed for you on devstack-trusty | 20:11 |
| fungi | not that it likely matters much | 20:11 |
| SpamapS | I assumed it was just some limitation of installing from git. | 20:11 |
| EmilienM | fungi: http://logs.openstack.org/57/296557/32/experimental/gate-puppet-openstack-integration-4-scenario001-tempest-centos-7/672e134/console.html#_2016-04-07_17_13_34_911 | 20:12 |
| greghaynes | SpamapS: not from git, from a path on disk | 20:12 |
| greghaynes | SpamapS: although git would do the same I think | 20:12 |
| SpamapS | git would have a version | 20:12 |
| pabelanger | EmilienM: once that lands, we can start playing with other-requirements.txt, adding an empty file to ensure the puppet modules install everything that is required | 20:12 |
| EmilienM | fungi: it's only in our puppet4 jobs (experimental) | 20:12 |
| fungi | EmilienM: pabelanger: i see, we were preinstalling libfacter on devstack-trusty? | 20:13 |
| pabelanger | Ya, need to see where libfacter is coming from | 20:13 |
| pabelanger | fungi: don't know yet | 20:13 |
| *** jed56 has quit IRC | 20:13 | |
| fungi | i don't think our bindep fallback list includes it either, but maybe it does. i haven't looked | 20:13 |
| *** Nikolay_St has quit IRC | 20:13 | |
| *** hdd has joined #openstack-infra | 20:14 | |
| clarkb | fungi: osic fips are cleaned up | 20:15 |
| *** ddieterly[away] is now known as ddieterly | 20:15 | |
| openstackgerrit | Ihar Hrachyshka proposed openstack-infra/project-config: Run a set of lightweight neutron jobs on upper-constraints.txt changes https://review.openstack.org/303054 | 20:15 |
| krotscheck | jtomasek: Hrm, that failed. | 20:15 |
| pabelanger | fungi: https://tickets.puppetlabs.com/browse/FACT-1123 | 20:15 |
| jtomasek | krotscheck: :/ | 20:15 |
| pabelanger | wonder if it is a locale issue | 20:15 |
| krotscheck | But _why_ | 20:15 |
| *** bpokorny has quit IRC | 20:16 | |
| fungi | clarkb: bbox deletes still going, but should be done in moments | 20:16 |
| fungi | clarkb: as predicted, it was 100% of our fip quota leaked | 20:16 |
| *** annegentle has joined #openstack-infra | 20:16 | |
| pabelanger | EmilienM: which review is that failure from? | 20:17 |
| pabelanger | EmilienM: want to recheck and hold the image, so I can see what is going on | 20:17 |
| EmilienM | pabelanger: https://review.openstack.org/296557 | 20:17 |
| krotscheck | jtomasek: You don't happen to know if there's something weird with cat? It' somewhat surprised that the shrinkwrap file wasn't entirely output in th ebuild | 20:18 |
| jtomasek | krotscheck: only error I see is [31m07 04 2016 20:14:24.080:ERROR [phantomjs.launcher]: [39mFontconfig warning: ignoring C.UTF-8: not a valid language tag | 20:18 |
| EmilienM | pabelanger: check experimental | 20:18 |
| *** ayoung has quit IRC | 20:18 | |
| jtomasek | krotscheck: no idea | 20:18 |
| fungi | pabelanger: i'm not seeing any locale-related errors in the job log. are you assuming they're being silenced somehow? | 20:18 |
| clarkb | jtomasek: do you ru nthat on centos? | 20:18 |
| pabelanger | fungi: maybe, going to hold the node and take a peak | 20:18 |
| clarkb | jtomasek: that isn't a valid local on centos, you can use C or something like en_US.utf-8 | 20:18 |
| jtomasek | clarkb: the job runs on ubuntu-trusty afaik | 20:19 |
| clarkb | should be valid there iirc, but maybe my memory is wrong | 20:19 |
| jtomasek | clarkb: I am not sure this is the error which causes the failure though | 20:19 |
| clarkb | fungi: I think the only outstanding thing in new nodepool is images. We have image builds for snapshots and the regions I didn't bother with yesterday due to max servers set low. And cleaning out the alien images | 20:19 |
| *** kgiusti has left #openstack-infra | 20:20 | |
| fungi | clarkb: i think precise is the only snapshot image we still care about now? | 20:20 |
| fungi | bare-precise i mean | 20:20 |
| clarkb | fungi: and tripleo's | 20:20 |
| fungi | but yeah, that may mean a long wait for any precise-based jobs | 20:20 |
| fungi | aha | 20:20 |
| *** maishsk has quit IRC | 20:21 | |
| clarkb | the others are devstack-centos7 in rax beacuse it isn't dib there but is going away soon and vexxhost and rax-ord | 20:21 |
| pabelanger | fungi: Ya, this will be a rubygem not distro package | 20:21 |
| clarkb | I am going to start working on figuring out the alien image list | 20:21 |
| *** mrmartin has joined #openstack-infra | 20:21 | |
| nibalizer | crinkle: haha nvm | 20:21 |
| fungi | clarkb: you mean the exception it's raising when you try to run it? | 20:21 |
| krotscheck | clarkb: Is there something about how the logfiles are accessed by jenkins that would throw up if someone did a `cat` on a 294kb text file? | 20:22 |
| clarkb | fungi: haven't even gotten that far :) | 20:22 |
| clarkb | krotscheck: shouldn't | 20:22 |
| clarkb | krotscheck: but it might be ebtter to log that file separately | 20:22 |
| *** eharney has quit IRC | 20:22 | |
| clarkb | krotscheck: if you want its contents | 20:22 |
| fungi | clarkb: yeah, i tested it a little bit ago when you were erranding and i think it's bitrotted | 20:22 |
| mtreinish | nibalizer: did we ever get the cacti thing fixed? I still don't see subunit-worker02.o.o listed there | 20:23 |
| *** sdake_ is now known as sdake | 20:23 | |
| clarkb | fungi: oh fun, I see it now | 20:23 |
| *** maishsk has joined #openstack-infra | 20:23 | |
| ihrachys | AJaeger: hi. could you please check https://review.openstack.org/303054 ? I am not really sure in the syntax. | 20:24 |
| EmilienM | pabelanger: I found more package issue http://logs.openstack.org/76/296076/9/check/gate-puppet-tripleo-puppet-syntax-3-centos-7/04d81d8/console.html#_2016-04-06_19_57_44_723 | 20:25 |
| jtomasek | krotscheck: yeah, the cat is not complete and right after it says 'step 'Execute shell' marked build as failure' | 20:26 |
| *** tonytan_brb is now known as tonytan4ever | 20:26 | |
| *** eharney has joined #openstack-infra | 20:26 | |
| krotscheck | jtomasek: I find that suspicious. | 20:26 |
| krotscheck | fuel's got the same problem. | 20:26 |
| pabelanger | EmilienM: do a recheck, we already fixed that | 20:26 |
| EmilienM | pabelanger: ack | 20:26 |
| pabelanger | EmilienM: TL;DR, removing rdo-release will break bindep, so we need to call bindep first | 20:26 |
| *** hashar has quit IRC | 20:27 | |
| fungi | krotscheck: jtomasek: is the example a log you're looking at on logs.o.o or on one of the jenkins masters directly? | 20:27 |
| EmilienM | pabelanger: ok thx | 20:27 |
| jtomasek | fungi: https://jenkins07.openstack.org/job/gate-tripleo-ui-npm-run-lint/9/consoleFull | 20:27 |
| clarkb | fungi: http://paste.openstack.org/show/493415/ that is an artisanal hand made alien image list for osic | 20:27 |
| clarkb | fungi: any reason to not start deleting those that you can think of? new nodepool seems to be functional | 20:28 |
| fungi | clarkb: nope, double-checked and none of those uuids show up in the new nodepool image-list output | 20:29 |
| nibalizer | mtreinish: um likely not | 20:29 |
| *** tongli has joined #openstack-infra | 20:29 | |
| fungi | clarkb: should be fine to go for it | 20:30 |
| *** moravec has quit IRC | 20:30 | |
| nibalizer | mtreinish: this is the review | 20:31 |
| nibalizer | https://review.openstack.org/#/c/284466/ | 20:31 |
| clarkb | fungi: deletes running for osic, I will work on the others | 20:31 |
| clarkb | fungi: I think the alien list thing is related to shade stuff and it will likely end up being a mile long thread to pull on | 20:31 |
| clarkb | so not worrying about it yet | 20:31 |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/project-config: Do not cat npm-shrinkwrap.json https://review.openstack.org/303059 | 20:31 |
| krotscheck | jtomasek, fungi, clarkb: Maybe this'll work? ^^ | 20:31 |
| krotscheck | ¯\_(ツ)_/¯ | 20:31 |
| *** _ari_ has quit IRC | 20:32 | |
| ddieterly | what is a username/password that can be used in the gates to ssh into the DevStack localhost? | 20:32 |
| jtomasek | krotscheck: he:) the contents of shrinkwrap.json is pretty large, so maybe cat fails because of it | 20:32 |
| krotscheck | jtomasek: Well, I tried running cat on my little trusty vm here, and it didn't seem to have any problems. | 20:33 |
| mtreinish | nibalizer: cool, +1 :) | 20:33 |
| ddieterly | basically, i need to ssh into the same machine (devstack) inside a test | 20:33 |
| fungi | ddieterly: i think they're set by devstack. but i'm looking | 20:33 |
| nibalizer | fungi: clarkb i can babysit https://review.openstack.org/#/c/284466/6https://review.openstack.org/#/c/284466/6 if you want to look at it | 20:34 |
| nibalizer | er 284466 | 20:34 |
| ddieterly | fungi thanks | 20:34 |
| fungi | ddieterly: oh, ssh into devstack localhost? that i don't think is set at all | 20:34 |
| fungi | ddieterly: the hosts which run these don't enable password auth | 20:35 |
| fungi | ddieterly: i think most of the cases where we do this, we have the account set its own keys in authorized_keys so it can ssh localhost without a password | 20:35 |
| clarkb | fungi: ddieterly correct there is a macro for it in the puppet beaker rspec tests | 20:36 |
| *** jcooley_ has joined #openstack-infra | 20:36 | |
| clarkb | nibalizer: kind of swamped with nodepool stuff still :/ | 20:36 |
| *** julim has quit IRC | 20:36 | |
| nibalizer | mk | 20:36 |
| ddieterly | fungi, ok, thanks | 20:37 |
| clarkb | fungi: osic and bluebox are done. Internap is in progress | 20:40 |
| *** ericksonsantos has joined #openstack-infra | 20:41 | |
| clarkb | thats interesting | 20:41 |
| *** baoli has quit IRC | 20:41 | |
| clarkb | on internap glance image-list and nova image-list do not agree | 20:41 |
| clarkb | mgagne: ^ fyi not sure if that is expected behavior. But it broke my glance image-delete loop | 20:41 |
| mgagne | clarkb: checking | 20:41 |
| ddieterly | fungi clarkb is openssh setup and running in devstack in the gate jobs? | 20:42 |
| clarkb | basically all of our images show up under nova image-list but only your cloud wide images show up on glance image-list | 20:42 |
| clarkb | ddieterly: yes jenkisn communicates to the slaves via ssh | 20:42 |
| ddieterly | ok, so then in the test, i can setup a user with keys and use that user inside devstack? | 20:43 |
| clarkb | ddieterly: yes there is already a jjb macro to do this and is used by the puppet beaker rspec jobs | 20:43 |
| *** matt-borland has joined #openstack-infra | 20:43 | |
| ddieterly | ok, no idea what jjb macro is nor puppet beaker spec job | 20:43 |
| ddieterly | yes, i'm a lamo ;-) | 20:44 |
| clarkb | jenkins job builder, the tool we use to configure the jenkins jobs | 20:44 |
| krotscheck | jtomasek: I'm about to close out for the day. | 20:45 |
| ddieterly | it looks like jenkins is setup on devstack to be able to ssh in | 20:45 |
| *** baoli has joined #openstack-infra | 20:45 | |
| lifeless | SpamapS: greghaynes: right, what you are describing is the expected behavour | 20:46 |
| mgagne | clarkb: I don't see any different between the 2. | 20:46 |
| *** baoli_ has joined #openstack-infra | 20:46 | |
| openstackgerrit | Derek Higgins proposed openstack-infra/tripleo-ci: Fix patch to deploy.env for mirror puppet modules https://review.openstack.org/303067 | 20:46 |
| clarkb | mgagne: huh definitely different for out user/tenant. Maybe its a client version thing? | 20:46 |
| fungi | ddieterly: allow-local-ssh-root http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/jobs/macros.yaml#n812 | 20:46 |
| mgagne | clarkb: will check | 20:46 |
| *** mriedem has quit IRC | 20:46 | |
| clarkb | we have a lot of leaked images in ovh-gra1 | 20:47 |
| SpamapS | lifeless: kk, thanks for taking a look. :) | 20:47 |
| clarkb | I think because uploads there fail frequently and we don't cleanup after reliably | 20:47 |
| fungi | ddieterly: it's used by jobs like this one http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/jobs/puppet-module-jobs.yaml#n71 | 20:47 |
| *** Sukhdev has quit IRC | 20:47 | |
| *** burgerk has quit IRC | 20:47 | |
| *** rfolco has quit IRC | 20:47 | |
| lifeless | SpamapS: you need to use edit-constraints to tell pip you want a non-version-constrained tempest | 20:47 |
| clarkb | mordred: this is a problem with the glance image upload 2 step process fwiw | 20:47 |
| lifeless | SpamapS: again, see devstack for an example | 20:47 |
| ddieterly | fungi thanks | 20:47 |
| clarkb | mordred: and I cna't think of a good reason it needs ot be that way but shade should handle it | 20:47 |
| SpamapS | lifeless: yeah, that's exactly what I think we'll do. | 20:48 |
| * zigo just finished uploading all of Mitaka final to Debian Sid (list of services in this release which I support: http://paste.debian.net/426194/) | 20:49 | |
| *** ddieterly is now known as ddieterly[away] | 20:49 | |
| mgagne | clarkb: http://paste.openstack.org/show/493419/ | 20:49 |
| mgagne | clarkb: I sorted the output for better debug | 20:49 |
| *** baoli has quit IRC | 20:50 | |
| clarkb | mordred: 139 leaked iamges in ovh-gra1 | 20:50 |
| *** baoli_ has quit IRC | 20:50 | |
| *** baoli has joined #openstack-infra | 20:51 | |
| ianw | SpamapS / greghaynes: I'd really like to get the stack from https://review.openstack.org/#/c/302509/ in a point release soon because a) it's totally broken and b) making nodepool logs about 10x bigger than they need to be | 20:51 |
| fungi | still no new fips leaked in bbox, so that was presumably just from the cutover | 20:52 |
| *** ddieterly[away] has quit IRC | 20:52 | |
| clarkb | mordred: we should consider forcing glance v1 | 20:53 |
| *** mylu has quit IRC | 20:54 | |
| *** nihilifer has joined #openstack-infra | 20:54 | |
| *** mylu has joined #openstack-infra | 20:54 | |
| clarkb | assuming v1 is less silly, I hvaen't checked that | 20:54 |
| *** david-lyle has joined #openstack-infra | 20:56 | |
| clarkb | seeing quite a few launch fails in gra1 | 20:57 |
| *** abregman has joined #openstack-infra | 20:57 | |
| clarkb | fungi: ^ maybe you can look into that while I clenaup images? | 20:58 |
| fungi | clarkb: yep, digging now. thanks | 20:58 |
| clarkb | I suppose its possible I derped the image deletes in gra1 and it is trying to use an image that dne | 20:59 |
| clarkb | if that happaned just nodepool image-delete the image and force it to queue an upload | 20:59 |
| fungi | LaunchStatusException: Server 8062923f-0708-4d24-89e6-3b6bbfe74484 for node id: 3040 status: ERROR | 20:59 |
| *** maishsk_ has joined #openstack-infra | 21:00 | |
| clarkb | ya I tried nova show and got nothing | 21:00 |
| clarkb | maybe try manually booting an instnace that is otherwise the same as that one was? | 21:00 |
| fungi | i'll check the gra1 list of images against nova image list | 21:00 |
| *** maishsk has quit IRC | 21:00 | |
| *** maishsk_ is now known as maishsk | 21:00 | |
| *** pcaruana has quit IRC | 21:01 | |
| crinkle | nibalizer: what is haha nvm, the json api? | 21:01 |
| *** davideagnello has quit IRC | 21:02 | |
| *** davideagnello has joined #openstack-infra | 21:02 | |
| nibalizer | crinkle: ya, we don't have a json api on our mirrors | 21:02 |
| nibalizer | im not sure why I thought we would | 21:02 |
| *** bpokorny has joined #openstack-infra | 21:02 | |
| openstackgerrit | Merged openstack-infra/project-config: Execute sbuild after relogin https://review.openstack.org/302610 | 21:02 |
| crinkle | nibalizer: yeah that is what i figured, it didn't seem like the xml api worked | 21:02 |
| *** rossella_s has quit IRC | 21:03 | |
| crinkle | nibalizer: that is why the gross regex | 21:03 |
| fungi | clarkb: that failure was for an ubuntu-trusty boot, and the ubuntu-trusty image in nodepool for gra1 is showing active in nova image-list output still | 21:03 |
| *** weshay has quit IRC | 21:03 | |
| *** lucasagomes has quit IRC | 21:03 | |
| clarkb | fungi: ok | 21:03 |
| nibalizer | yah | 21:03 |
| crinkle | nibalizer: is that something that could be enabled or is it just not supported? | 21:03 |
| clarkb | fungi: probably something on their end then. | 21:03 |
| fungi | so doesn't seem to be an image deleted out from under nodepool afaict. still digging for other possibilities | 21:03 |
| *** rossella_s has joined #openstack-infra | 21:03 | |
| nibalizer | crinkle: i have no idea | 21:03 |
| nibalizer | i think our mirrors are a dumb http server so we'd have to compute the json and spray it out into some directories | 21:04 |
| *** fitoduarte has quit IRC | 21:04 | |
| *** mylu has quit IRC | 21:04 | |
| *** psilvad_ has quit IRC | 21:04 | |
| *** e0ne has quit IRC | 21:04 | |
| fungi | nibalizer: crinkle: what our mirrors provide is what pypi calls it's "simple" api | 21:04 |
| *** |-paul-| has joined #openstack-infra | 21:05 | |
| *** jcooley_ has quit IRC | 21:05 | |
| *** eharney has quit IRC | 21:05 | |
| fungi | which basically means file indices under /simple/packagename | 21:05 |
| *** rajinir has quit IRC | 21:05 | |
| *** trown is now known as trown|outtypewww | 21:06 | |
| crinkle | fungi: i made https://review.openstack.org/#/c/302396/ to try to make our puppet use the simple api on the mirrors instead of looking at pypi.python.org, would love your feedback when you have some time | 21:06 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder: Fix disk usage report https://review.openstack.org/302509 | 21:07 |
| pabelanger | fungi: EmilienM: failed to initialize logging system due to a locale error: Invalid or unsupported charset:ANSI_X3.4-1968 when you run facter for puppet 4 | 21:08 |
| pabelanger | so, something related to locale | 21:08 |
| *** hdd has quit IRC | 21:09 | |
| *** jamesmcarthur has quit IRC | 21:09 | |
| *** maishsk has quit IRC | 21:09 | |
| clarkb | ovh-bhs1 leaked iamges are being deleted now | 21:09 |
| *** lucasagomes has joined #openstack-infra | 21:11 | |
| *** e0ne has joined #openstack-infra | 21:11 | |
| *** thorst_ has quit IRC | 21:12 | |
| fungi | lifeless: dstufft: just to confirm, there's no dry run option for pip install where you could do something like `pip install --upgrade --no-deps --dry-run somepackage` to find out what version of a package it _would_ install for that request, right? | 21:12 |
| dstufft | fungi: closest you can get is pip download | 21:13 |
| *** maishsk has joined #openstack-infra | 21:13 | |
| fungi | lifeless: dstufft: wondering whether there's some way to exercise the index scan pip install would do without actually downloading/installing anything, to get the "latest" version of a package from a mirror only providing the simple api | 21:14 |
| persia | A reputed rune is `virtualenv /tmp/venv; /tmp/venv/bin/pip install foo; rm -rf /tmp/venv ` | 21:14 |
| *** aeng has joined #openstack-infra | 21:14 | |
| dstufft | fungi: wellllll | 21:14 |
| *** ldnunes has quit IRC | 21:14 | |
| dstufft | pip has no public API, but you could always just use the PackageFinder class if you wanted | 21:14 |
| dstufft | it might break on pip upgrades though, so I wouldn't ship it somewhere that you don't have control over | 21:15 |
| fungi | the "not downloading" part is more from a performance perspective, as the context here is a puppet module that determines whether it needs to upgrade a package or not | 21:15 |
| dstufft | pip list -o ? | 21:16 |
| fungi | i guess the alternative would be to call pip install --upgrade every time, and only report "changed" when that exec actually installed something | 21:16 |
| fungi | oh, that might do the trick | 21:16 |
| fungi | crinkle: ^ considered as an option? | 21:16 |
| clarkb | rax-iad in progress now too | 21:16 |
| fungi | crinkle: from pip list --help: -o, --outdated List outdated packages | 21:16 |
| crinkle | fungi: that might be better | 21:17 |
| fungi | i suppose that also assumes a new enough pip version that pip list is provided | 21:17 |
| *** tongli has quit IRC | 21:17 | |
| fungi | dstufft: do you recall when you implemented --outdated, or should i hunt in changelogs? | 21:17 |
| *** thorst_ has joined #openstack-infra | 21:18 | |
| dstufft | I don't recall exactly, I think it was prior to 1.5 tho | 21:18 |
| *** salv-orl_ has joined #openstack-infra | 21:18 | |
| *** nwkarsten has quit IRC | 21:18 | |
| crinkle | fungi: we can generally assume we have latest pip on everything, right? | 21:18 |
| fungi | i'd call that ancient enough history for our purposes | 21:18 |
| *** e0ne has quit IRC | 21:18 | |
| fungi | crinkle: on everything we build, yes | 21:19 |
| nibalizer | i think using a pip command to find the latest would be generally in line with how every other provider works | 21:19 |
| *** mylu has joined #openstack-infra | 21:19 | |
| fungi | nibalizer: yeah, that's what this seems to do | 21:19 |
| fungi | fungi@dhole:~$ work/openstack/venvs/virtualenv/bin/pip list --outdated | 21:19 |
| fungi | setuptools (2.2) - Latest: 20.6.7 [wheel] | 21:19 |
| *** Nikolay_St has joined #openstack-infra | 21:19 | |
| crinkle | yes that would be way better, i didn't know about pip list (pip search was not doing the right thing) | 21:19 |
| nibalizer | if we say that we don't need this in puppet core and we just need it for us we can use whatever pip version we like | 21:20 |
| nibalizer | crinkle: i do think this hack was neat | 21:20 |
| *** mylu has quit IRC | 21:20 | |
| fungi | yeah, i'm certainly not opposed to the hack | 21:20 |
| *** thorst__ has joined #openstack-infra | 21:21 | |
| fungi | (haven't even finished looking through it, was just questioning assumptions from the commit message first) | 21:21 |
| dstufft | fungi: https://github.com/pypa/pip/pull/752 fwiw | 21:21 |
| fungi | dstufft: thanks! | 21:22 |
| dstufft | looks like it was added in 1.3 | 21:22 |
| *** jsavak has quit IRC | 21:22 | |
| fungi | wow, so ubuntu trusty debs even new enough to have this | 21:22 |
| *** thorst_ has quit IRC | 21:22 | |
| *** salv-orlando has quit IRC | 21:22 | |
| dstufft | I think uh, anything newer than Precise, Debian oldstable, etc era is good to go | 21:22 |
| *** jsavak has joined #openstack-infra | 21:23 | |
| lifeless | fungi: you can't deal with transitive deps without downloading | 21:23 |
| fungi | lifeless: right, we don't expect to anyway | 21:23 |
| nibalizer | lifeless: i don't think that is required for this | 21:23 |
| lifeless | fungi: don't expect to transitive? | 21:23 |
| *** mylu has joined #openstack-infra | 21:24 | |
| nibalizer | ya looking at the output of pip list --outdated it looks like exactly what we need | 21:24 |
| fungi | lifeless: this is more from the perspective of "my puppet manifest says i should always automatically upgrade to the latest foopackage, so figure out if there's a newer one" | 21:24 |
| clarkb | 165 leaked images in rax-dfw | 21:24 |
| *** markvan has quit IRC | 21:24 | |
| nibalizer | and if its in trusty, then maybe it might make it in to core after all | 21:24 |
| dstufft | at some point we need to add some --json or something to our commands | 21:24 |
| nibalizer | at least for puppet 4 | 21:24 |
| dstufft | I'm not sure that the output of pip list is stable :( | 21:24 |
| *** mrmartin has quit IRC | 21:24 | |
| clarkb | fungi: that makes meworry that we may need to do cleanup in our swift container too | 21:24 |
| lifeless | dstufft: its not a contract today, no. | 21:24 |
| clarkb | anyways dfw deletes in progress now | 21:25 |
| fungi | lifeless: in this case puppet users don't, i think, consider ensure=>latest to necessarily mean "and also update all transitive dependencies even if this package hasn't updated" | 21:25 |
| *** Sukhdev has joined #openstack-infra | 21:25 | |
| dstufft | looks like it's mostly stayed the same though | 21:25 |
| lifeless | fungi: ack, though I thought puppet was all about reproducability :) | 21:25 |
| *** bpokorny has quit IRC | 21:25 | |
| *** thorst__ has quit IRC | 21:25 | |
| fungi | lifeless: normally i'd agree, but i don't think the pip provider has ever really had that guarantee since dependencies are open-ended | 21:26 |
| dstufft | oh no it hasn't, but close enough I think you can hack it | 21:26 |
| *** bpokorny has joined #openstack-infra | 21:26 | |
| dstufft | https://bpaste.net/show/120f204d05ad | 21:26 |
| dstufft | and I guess if all you care about is "should I upgrade", you can just test for the name of the thing being in the output | 21:26 |
| dstufft | rather than needing to parse out the version | 21:26 |
| *** hdd has joined #openstack-infra | 21:26 | |
| *** markvan has joined #openstack-infra | 21:27 | |
| clarkb | fungi: maybe you want to take a peak into the rax swift image containers to make sure we don't have 500 images in there all crufty | 21:27 |
| pabelanger | EmilienM: Will have to follow up with puppetlabs tomorrow and see what is going on. But cfacter is not happy | 21:27 |
| zigo | fungi: lifeless: clarkb: The patch I just made to attempt for the build script to re-login after sbuild is setup, just failed. What do you suggest here, moving forward? | 21:27 |
| EmilienM | _ody: ^ | 21:27 |
| clarkb | zigo: fix the failure? | 21:27 |
| zigo | https://jenkins02.openstack.org/job/pkgdeb-build-openstack-pkg-tools/4/console | 21:27 |
| pabelanger | EmilienM: do we need to revert to devstack-centos7? | 21:27 |
| EmilienM | pabelanger: no, the jobs are experimental | 21:28 |
| zigo | clarkb: The failure is because one is supposed to logout, and I don't know how this can be done with infra. | 21:28 |
| pabelanger | EmilienM: okay, I'll jump into puppetlabs tomorrow and see if we can get support for facter | 21:28 |
| EmilienM | pabelanger: cool | 21:28 |
| clarkb | zigo: I already told you how in my review... | 21:28 |
| zigo | Ok, reading... | 21:28 |
| fungi | zigo: yeah, clarkb and i both mentioned alternative solutions | 21:28 |
| fungi | in review comments | 21:29 |
| annegentle | fungi: or clarkb: or jeblair: or someone who knows about the Rackspace cloud account, can I chat with one of you privately for a minute? | 21:29 |
| fungi | annegentle: sure, i'm free | 21:29 |
| annegentle | fungi: thanks! | 21:29 |
| *** mylu has quit IRC | 21:29 | |
| zigo | clarkb: newgrp wont help | 21:29 |
| clarkb | zigo: why not? | 21:31 |
| zigo | clarkb: Hum... I can try! | 21:31 |
| *** denisra_ has joined #openstack-infra | 21:31 | |
| clarkb | also I don't know what fungi's suggestion was but its proably good too | 21:31 |
| *** ayoung has joined #openstack-infra | 21:32 | |
| zigo | clarkb: He suggested using exec. | 21:32 |
| zigo | Both of which I can try without a patch to infra. | 21:32 |
| pabelanger | clarkb: fungi: zigo: Heres how I handled the issue when I was trying to get packaging: https://review.openstack.org/#/c/185680/6/modules/openstack_project/manifests/slave_pkg.pp | 21:32 |
| pabelanger | I did it with puppet before jenkins connected | 21:32 |
| anteaya | clarkb fungi nice work on the nodepool cut over | 21:33 |
| pabelanger | see line #20 | 21:33 |
| *** hdd has quit IRC | 21:33 | |
| *** nwkarsten has joined #openstack-infra | 21:33 | |
| clarkb | pabelanger: I don't think we should do it that way | 21:33 |
| clarkb | pabelanger: just like with package deps jobs should manage group membership directly | 21:33 |
| zigo | pabelanger: I'm a puppet blind ... :/ | 21:33 |
| openstackgerrit | Fei Long Wang proposed openstack-infra/project-config: Add Tempest gate job for Zaqar https://review.openstack.org/302487 | 21:34 |
| *** denisra has quit IRC | 21:34 | |
| pabelanger | clarkb: right, just giving another, other option :) | 21:35 |
| clarkb | ya but I will -2 that option now :) | 21:35 |
| clarkb | we shouldn't get this far into making jobs manage their own dependencies to add stuff like that | 21:35 |
| zigo | clarkb: Doing something like "newgrp - sbuild" will not modify the current env, it will just initialize a new interactive shell, no? Wont it just either wait forever, or exit back to the upper shell? | 21:36 |
| *** tiswanso has quit IRC | 21:37 | |
| *** lucasagomes has quit IRC | 21:37 | |
| *** nwkarsten has quit IRC | 21:37 | |
| *** baoli has quit IRC | 21:37 | |
| openstackgerrit | Colleen Murphy proposed openstack-infra/puppet-pip: Add new pip provider to use pypi mirrors https://review.openstack.org/302396 | 21:37 |
| *** sridhar_ram has quit IRC | 21:37 | |
| zigo | clarkb: I really don't see how it may help... | 21:38 |
| *** baoli has joined #openstack-infra | 21:38 | |
| clarkb | zigo: the first paragraph of the man page says exactly what happens | 21:38 |
| zigo | clarkb: I get it, though isn't this for interactive shell only? | 21:38 |
| *** jcooley_ has joined #openstack-infra | 21:38 | |
| clarkb | I don't think so | 21:38 |
| *** jcooley_ has quit IRC | 21:39 | |
| *** tphummel has quit IRC | 21:39 | |
| zigo | Ah indeed... (I just tested it in a local script) | 21:39 |
| zigo | Thanks, trying this. | 21:39 |
| _ody | pabelanger: You noticed facter broken now on your failing expiremental job? | 21:40 |
| *** cloudtrainme has quit IRC | 21:41 | |
| _ody | Ah. Ok. I see it. I'll look into what might be happening. | 21:42 |
| pabelanger | _ody: failed to initialize logging system due to a locale error: Invalid or unsupported charset:ANSI_X3.4-1968 | 21:42 |
| pabelanger | _ody: when you invoke facter | 21:42 |
| pabelanger | _ody: have to run tonight, but will be working on it tomorrow too | 21:43 |
| clarkb | pabelanger: do we need to set the locale to C.utf-8 in our image builds? | 21:43 |
| clarkb | assuming thats xenial? | 21:43 |
| pabelanger | clarkb: this is centos-7 | 21:43 |
| pabelanger | clarkb: not explicitly, we should likely populate /etc/locale.conf to it | 21:43 |
| pabelanger | sorry, have to run | 21:44 |
| *** thorst_ has joined #openstack-infra | 21:44 | |
| *** lucasagomes has joined #openstack-infra | 21:45 | |
| *** weshay has joined #openstack-infra | 21:45 | |
| *** tphummel has joined #openstack-infra | 21:47 | |
| *** erlon has quit IRC | 21:47 | |
| *** Goneri has quit IRC | 21:47 | |
| *** thorst_ has quit IRC | 21:48 | |
| *** lucasagomes has quit IRC | 21:50 | |
| fungi | centos 7 lacks C.UTF-8 | 21:50 |
| fungi | fedora (22?) added it | 21:50 |
| fungi | but pretty much all distros are adding it by patching newer glibc | 21:51 |
| fungi | upstream is still waffling on whether to make it an actual locale iirc | 21:51 |
| *** mylu has joined #openstack-infra | 21:52 | |
| *** Sukhdev has quit IRC | 21:52 | |
| clarkb | openstack image delete not working against vexxhost :/ | 21:53 |
| clarkb | oh wait | 21:54 |
| clarkb | is it paging | 21:54 |
| clarkb | ugh | 21:54 |
| clarkb | ya I think that is a major anti feature | 21:55 |
| *** nwkarsten has joined #openstack-infra | 21:56 | |
| *** lucasagomes has joined #openstack-infra | 21:56 | |
| *** matt-borland has quit IRC | 21:56 | |
| *** regXboi has quit IRC | 21:58 | |
| *** denisra_ has quit IRC | 21:58 | |
| *** denisra_ has joined #openstack-infra | 21:58 | |
| clarkb | I think it should do like git and direct to $PAGER if interactive | 21:59 |
| zigo | clarkb: As I expected, "newgrp - sbuild" didn't help! | 21:59 |
| zigo | clarkb: http://logs.openstack.org/82/302382/4/check/pkgdeb-build-openstack-pkg-tools/8fcf2ae/console.html | 21:59 |
| zigo | clarkb: Any other idea? | 21:59 |
| *** ybathia has joined #openstack-infra | 22:00 | |
| clarkb | zigo: start a new login shell | 22:00 |
| *** xyang1 has quit IRC | 22:00 | |
| clarkb | bash --loging or whatever | 22:00 |
| *** bpokorny_ has joined #openstack-infra | 22:01 | |
| *** bpokorny_ has quit IRC | 22:01 | |
| *** ddieterly has joined #openstack-infra | 22:01 | |
| *** ayoung has quit IRC | 22:01 | |
| *** bpokorny_ has joined #openstack-infra | 22:02 | |
| fungi | zigo: it doesn't look like you ran those commands under newgrp, you merely ran them after newgrp | 22:02 |
| fungi | oh, i see, it's supposed to modify the current environment, not spawn a subshell | 22:02 |
| *** nmagnezi has quit IRC | 22:02 | |
| clarkb | ya | 22:02 |
| fungi | zigo: nevermind, yeah i don't know that newgrp is going to pick up groups your current login predates | 22:03 |
| fungi | seems like it won't | 22:03 |
| *** pvaneck has quit IRC | 22:03 | |
| nibalizer | ah, sad | 22:04 |
| *** bpokorny has quit IRC | 22:04 | |
| *** jcooley_ has joined #openstack-infra | 22:05 | |
| *** ihrachys has quit IRC | 22:05 | |
| *** maishsk has quit IRC | 22:05 | |
| fungi | newgrp is working in my limited testing here. playing around with it a little more | 22:07 |
| clarkb | observation time: we almost never ever leak snapshot builds | 22:08 |
| fungi | also you probably want to reset your group back to the original one after calling newgrp to the new one | 22:08 |
| clarkb | dib iamge uploads seem to leak at the rate of "lots" | 22:08 |
| *** mylu has quit IRC | 22:08 | |
| clarkb | all of our cloud regions should have spring cleaned glance image lists | 22:08 |
| clarkb | I have no looked at the rax image swift container because I am scared | 22:09 |
| greghaynes | clarkb: I wonder if we should periodically delete non public images that the builders dont know about | 22:09 |
| clarkb | greghaynes: I think first step is fix shade's handling of fialure | 22:10 |
| clarkb | then ya if it continues to be bad we periodically clean them out | 22:10 |
| fungi | oh, indeed newgrp does in fact (per my testing) create a new subshell | 22:10 |
| openstackgerrit | Armando Migliaccio proposed openstack-infra/project-config: Make gate-neutron-dsvm-api job non-voting https://review.openstack.org/303100 | 22:13 |
| *** briancline has quit IRC | 22:14 | |
| armax | anyone kind enough to have a look at ^ | 22:15 |
| armax | we’re trying to put some sanity back into the Neutron CI until we get things sorted | 22:15 |
| armax | thanks! | 22:15 |
| *** briancline has joined #openstack-infra | 22:17 | |
| anteaya | armax: +2 | 22:18 |
| *** mikal has quit IRC | 22:18 | |
| armax | anteaya: thanks! | 22:18 |
| anteaya | welcome | 22:18 |
| *** mikal has joined #openstack-infra | 22:18 | |
| *** ddieterly is now known as ddieterly[away] | 22:18 | |
| anteaya | I hope the spliting out of these test suites continues and is effective | 22:18 |
| *** flepied has joined #openstack-infra | 22:19 | |
| armax | anteaya: we’re getting there | 22:19 |
| *** camunoz has joined #openstack-infra | 22:19 | |
| anteaya | yay progress! | 22:19 |
| anteaya | armax: good work | 22:19 |
| *** ddieterly[away] is now known as ddieterly | 22:19 | |
| *** knikolla has quit IRC | 22:19 | |
| clarkb | another perspective. It is early in the cycle when little work gets done. YOu basically have 2 weeks to fix it while few notice | 22:20 |
| clarkb | make it non voting and that 2 weeks means more regression potential | 22:20 |
| *** jcooley_ has quit IRC | 22:20 | |
| *** Sukhdev_ has joined #openstack-infra | 22:21 | |
| *** flepied has quit IRC | 22:21 | |
| *** doug-fish has quit IRC | 22:23 | |
| *** briancline has quit IRC | 22:25 | |
| *** ayoung has joined #openstack-infra | 22:26 | |
| *** jcooley_ has joined #openstack-infra | 22:26 | |
| *** Sukhdev_ has quit IRC | 22:26 | |
| *** briancline has joined #openstack-infra | 22:27 | |
| *** jcooley_ has quit IRC | 22:27 | |
| *** jcooley_ has joined #openstack-infra | 22:27 | |
| *** yaume has joined #openstack-infra | 22:27 | |
| *** dimtruck is now known as zz_dimtruck | 22:28 | |
| *** rcernin has quit IRC | 22:29 | |
| *** ddieterly is now known as ddieterly[away] | 22:30 | |
| *** ddieterly[away] is now known as ddieterly | 22:30 | |
| *** yaume has quit IRC | 22:30 | |
| *** cloudtrainme has joined #openstack-infra | 22:31 | |
| anteaya | clarkb: that is true, I'm willing to cut armax some slack here as he and I did have a very long, earnest and mutually beneficial discussion at the last neutron mid-cycle about https://review.openstack.org/#/c/274182/ | 22:31 |
| armax | clarkb: believe me, I am fully aware of the circumstances :) | 22:32 |
| *** cloudtrainme has quit IRC | 22:32 | |
| anteaya | he listened when I explained my personal concern about care of the gate and regressions and he took it to heart and spent some time learning more about elastic recheck and has been submitting some fingerprints for the elastic recheck repo | 22:32 |
| anteaya | thank you for that armax | 22:32 |
| *** angdraug has quit IRC | 22:32 | |
| armax | clarkb: but I think we’d want to rework some of the internal plumbings of the job and that requires us to make it non voting | 22:32 |
| *** cloudtrainme has joined #openstack-infra | 22:32 | |
| armax | clarkb: I am thinking like moving gate hooks etc | 22:33 |
| anteaya | I feel armax is feeling caught between two uncomfortable places, not entirely of his own making, and is looking for a show of support as he tries to make things better | 22:33 |
| pleia2 | anteaya: forgot to follow up yesterday, our wikimedia module is very much incomplete and is based on a 4 year old version of Ubuntu (12.04), so I'm not surprised that you're having dependency/missing packages issues | 22:33 |
| anteaya | so I'm inclined to show him some support | 22:33 |
| anteaya | pleia2: awesome, thanks for letting me know :) | 22:33 |
| fungi | the incomplete part is most of what's held us back from rebuilding/upgrading it before now | 22:34 |
| * pleia2 nods | 22:34 | |
| anteaya | pleia2: I'm keeping notes about package dependenices here: https://review.openstack.org/#/c/302456/1 | 22:34 |
| pleia2 | anteaya: oh, great | 22:34 |
| anteaya | at this moment, I'm wondering how I can get my hands on a vm with a xenial image to test the puppet module on | 22:34 |
| fungi | in fact, the situation that server wound up in is one of the reasons we've gotten stricter about new services being fully automated before being in production | 22:35 |
| pleia2 | anteaya: do you have the ability locally to spin up an image? | 22:35 |
| anteaya | as it is possible all the badness might just disappear, in a puff of smoke like | 22:35 |
| anteaya | pleia2: I don't know | 22:35 |
| *** annegentle has quit IRC | 22:35 | |
| pleia2 | anteaya: like, in a virtualbox or with kvm? | 22:35 |
| anteaya | pleia2: I will try to figure out if I do tomorrow | 22:35 |
| anteaya | yes I have kvm | 22:35 |
| pleia2 | xenial hasn't been released yet, so it's early days for clouds to support it | 22:35 |
| anteaya | right | 22:35 |
| pleia2 | but you can grab a daily iso from the ubuntu dev folks | 22:35 |
| anteaya | cool, I will consider that tomorrow's project | 22:35 |
| anteaya | thank you | 22:36 |
| anteaya | fungi: glad it was the bad example to create a better workflow | 22:36 |
| pleia2 | sure, feel free to let me know if you have trouble finding the image | 22:36 |
| anteaya | pleia2: thanks I will do | 22:36 |
| anteaya | probably by the time I dig up my kvm notes again you will be online :) | 22:37 |
| *** cloudtrainme has quit IRC | 22:37 | |
| *** tpsilva has quit IRC | 22:37 | |
| pleia2 | hehe | 22:37 |
| anteaya | basically image magik is used by image scaler which requires ffmpeg | 22:37 |
| anteaya | which went away in trusty and is back in xenial | 22:38 |
| pleia2 | ah, yeah, ffmpeg is complicated | 22:38 |
| anteaya | yeah | 22:38 |
| anteaya | so I really really would like to use a distro package for it | 22:38 |
| fungi | i saw that in scrollback from yesterday and meant to reply | 22:38 |
| armax | clarkb: is my knowledge still current when I believe that we can’t use depends-on between project-config and other openstack projects like neutron? | 22:38 |
| anteaya | I'm listening, fungi | 22:39 |
| fungi | the situation with ffmpeg packaging in debian (and transitively ubuntu) is (or was at least) complicated indeed | 22:39 |
| * pleia2 nods | 22:39 | |
| clarkb | armax: depends on what you are wanting to change. You cannot do it with jenkins jobs because jenkins jobs must be vetted before they are updated | 22:39 |
| clarkb | armax: but in theory other bits could be depends on'd | 22:39 |
| fungi | started out with mpeg encoding patents possibly being enforceable and debian not wanting to get itself (or its users) sued for shipping the library | 22:39 |
| anteaya | good reason | 22:40 |
| armax | clarkb: ack | 22:40 |
| pleia2 | libav tends to be recommended ahead of ffmpeg even now, but feature parity is hard, and getting packages to switch over is too | 22:40 |
| fungi | fraunhofer encoding patent specifically i think? | 22:40 |
| fungi | and yeah, thus was born the libavcodec fork in debian | 22:40 |
| armax | clarkb: I am thinking in situations were a change in the pre and post hooks may need to go hand in hand with the hooks located in the project repo itself | 22:40 |
| bkero | Dealing with binary distros and linking with new filenames can be a big pain too | 22:40 |
| armax | clarkb: maybe I am talking nonsense | 22:41 |
| fungi | which lagged behind over time and ffmpeg moved on, and also teh licensing/patent situation got clarified | 22:41 |
| clarkb | armax: I think you just enable the hooks in the jobs then change them however you want in neutron | 22:41 |
| pleia2 | fungi: yeah, at least for a while debian compilied without fraunhofer, I remember a bug report in Ubuntu that complained about it | 22:41 |
| *** tonytan4ever has quit IRC | 22:41 | |
| clarkb | armax: I don't think that needs tight coupling between the two. Once you get the hooks enabled you should never need ot touch them again | 22:41 |
| fungi | also, for much of this time, debian-multimedia.org hosted unofficial packages of ffmpeg and lots of stuff recompiled against ffmpeg | 22:42 |
| armax | clarkb: I have seen past situations where we did not fully consider the execution of the jobs across all the branches | 22:42 |
| armax | and that has burned us a bit | 22:42 |
| * anteaya really enjoys storytime with fungi | 22:42 | |
| armax | nothing catastrophic though | 22:42 |
| anteaya | so sounds like we are all in agreement that a distro package for ffmpeg is preferred | 22:44 |
| fungi | anteaya: yeah, so according to https://trac.ffmpeg.org/wiki/CompilationGuide/Ubuntu the ffmpeg package in xenial should be actual ffmpeg | 22:44 |
| anteaya | fungi: wooooo, real ffmpeg! | 22:45 |
| *** abregman has quit IRC | 22:45 | |
| pleia2 | yeah, and if we do a migration (rather than upgrade) it sounds like trying to go directly to xenial will make sense | 22:46 |
| *** lakshmiS has quit IRC | 22:46 | |
| *** briancline has quit IRC | 22:46 | |
| *** gordc has quit IRC | 22:46 | |
| openstackgerrit | Anita Kuno proposed openstack-infra/puppet-mediawiki: Updates to move off precise https://review.openstack.org/302456 | 22:47 |
| anteaya | pleia2: might as well | 22:47 |
| fungi | i think debian jessie also wound up with actual ffmpeg packages too, not dummy packages depending on the forked libav | 22:47 |
| fungi | or that might only be in jessie-backports | 22:47 |
| anteaya | getting the module in shape for trusty might be more work that for xenial | 22:47 |
| anteaya | but anyway, so far I'm hearing support for testing the module on xenial which will be my next step | 22:48 |
| anteaya | thank you | 22:48 |
| *** daemontool has joined #openstack-infra | 22:48 | |
| fungi | ahh, yeah just in jessie backports (stable-bpo) | 22:48 |
| fungi | https://tracker.debian.org/pkg/ffmpeg | 22:49 |
| fungi | so it'll be in stretch, barring any release-critical bugs which demand its removal from testing before then | 22:49 |
| anteaya | is stretch the release name after jessie? | 22:49 |
| pleia2 | yes | 22:50 |
| anteaya | okay great, thanks | 22:50 |
| fungi | but anyway, that's the debian ffmpeg story in a nutshell, minus years of drama on the debian-devel mailing list and elsewhere | 22:50 |
| pleia2 | it's a purple octopus | 22:50 |
| anteaya | fungi: I was picturing you dramatically waving your hands about | 22:50 |
| anteaya | fungi: so thanks for not leaving all of it out | 22:50 |
| anteaya | pleia2: I don't know the purple octopus | 22:50 |
| *** |-paul-| has quit IRC | 22:50 | |
| pleia2 | anteaya: https://www.flickr.com/photos/pleia2/23535402942/sizes/l | 22:51 |
| pleia2 | toy story 3 character | 22:51 |
| jhesketh | Morning | 22:51 |
| *** baoli has quit IRC | 22:52 | |
| anteaya | pleia2: nice :) | 22:52 |
| anteaya | morning jhesketh | 22:52 |
| *** briancline has joined #openstack-infra | 22:52 | |
| *** mylu has joined #openstack-infra | 22:52 | |
| fungi | yes, "stretch" was a stretchy purple octopus in toy story 3 | 22:54 |
| fungi | just as "jessie" was a cowgirl doll | 22:54 |
| *** sdake has quit IRC | 22:54 | |
| *** rbrndt_ has quit IRC | 22:56 | |
| *** zz_dimtruck is now known as dimtruck | 22:56 | |
| *** sdague has quit IRC | 22:57 | |
| *** sdake has joined #openstack-infra | 22:58 | |
| clarkb | fungi: look at the jobs launched per hour graph on zuul status | 22:58 |
| clarkb | fungi: barely a blip | 22:58 |
| * clarkb calls this a success and stops thinking about computers for a bit | 22:59 | |
| fungi | indeed, success! | 23:00 |
| *** moravec has joined #openstack-infra | 23:00 | |
| *** dangers is now known as dangers_away | 23:00 | |
| fungi | clarkb: we do indeed have many, many, many files in the images container, some dating back as far as august | 23:02 |
| *** briancline has quit IRC | 23:02 | |
| *** BillCr5 has joined #openstack-infra | 23:02 | |
| fungi | clarkb: those are only temporarily created as part of the upload process and should be removed at the end normally, yes? | 23:03 |
| clarkb | fungi: I think we can delete all of those except for those from the last 2 days or so | 23:03 |
| clarkb | fungi: yes | 23:03 |
| clarkb | ideally by shde since it creates them | 23:03 |
| fungi | most of these show as 0 bytes | 23:03 |
| clarkb | huh failures maybe? | 23:04 |
| fungi | 198 files, 142.52 GB total | 23:05 |
| fungi | there' | 23:05 |
| clarkb | fungi: I/we should actually make sure that the ssh host keys are working properly now | 23:05 |
| fungi | there's also a images_segments container | 23:05 |
| fungi | it has 2804 files, 2.66 TB total | 23:05 |
| *** briancline has joined #openstack-infra | 23:07 | |
| fungi | oh, ugh, the contents of that are trees of folders nested about 5 deep | 23:07 |
| clarkb | different rsa host keys on centos7 | 23:07 |
| fungi | template-centos-7-1458125147 | 23:07 |
| *** ybathia has quit IRC | 23:08 | |
| clarkb | different on ubuntu-trusty too | 23:08 |
| clarkb | so ya lgtm | 23:08 |
| clarkb | fungi: fun | 23:08 |
| fungi | has template-centos-7-1458125147/1458125129.391523/16780599808/1073741824 as a path under it with 16 segment files | 23:08 |
| clarkb | fungi: I think there is md5summing involved somewhere too | 23:08 |
| clarkb | which might be part of that pathing? | 23:08 |
| greghaynes | clarkb: hah, thats a lot better ;) | 23:08 |
| *** Qiming has joined #openstack-infra | 23:09 | |
| *** akshai has quit IRC | 23:09 | |
| fungi | clarkb: yeah, i'm not real sure how to go about using shade to clean this up without embarking on a fair amount of development | 23:10 |
| clarkb | fungi: ya... probably can be dealt with by applying hammer to mordred | 23:10 |
| fungi | or looking into setting expiration policies? | 23:11 |
| *** dingyichen has joined #openstack-infra | 23:11 | |
| clarkb | fungi: I think shade should just delete the image from swift after it is successfully imported to glance | 23:12 |
| fungi | the main thing i want to know is whether these are actually ephemeral (temporary staging location for glance import task) or reused (nova actually being instructed by glance to pull from that url into its cache when needed) | 23:12 |
| *** sdake has quit IRC | 23:12 | |
| clarkb | same as shade should delete the image from glance when uploads fail | 23:12 |
| *** moravec has quit IRC | 23:12 | |
| clarkb | fungi: I am almost positive it is temporary staging because the glance import task takes forever | 23:12 |
| clarkb | implying all the bits are copied | 23:12 |
| *** jsavak has quit IRC | 23:13 | |
| fungi | if that's the case, then it should just be safe to blow all of this away as long as no rax image uploads are in progress | 23:13 |
| fungi | though their dashboard also doesn't present a recursive delete option | 23:13 |
| clarkb | fungi: sudo -H -u nodepool nodepool image-list | grep rax | grep building shows nothing | 23:13 |
| greghaynes | if only the code was upstream then we'd know... | 23:13 |
| fungi | heh | 23:13 |
| *** mriedem has joined #openstack-infra | 23:14 | |
| fungi | i think the only "easy" way to clear the entire images_segments container's thousands upon thousands of nested directories and files is to delete the container and recreate it | 23:15 |
| *** tosky has quit IRC | 23:15 | |
| clarkb | fungi: shade will recreate it by itself too I think | 23:16 |
| clarkb | fungi: maybe get mordred to confirm that his knowledge of this system is that it is a copy | 23:16 |
| clarkb | the nwe delete | 23:16 |
| fungi | though it's also possible that the images container's zero-byte files are really pointers into those images_containers directories, and that swift would be smart enough to remove them by reference? | 23:16 |
| fungi | er, into the images_segments directories | 23:16 |
| clarkb | oh right because the images are too big to be a single swift object | 23:16 |
| clarkb | so it does the magical large objcet support stuff however that works | 23:16 |
| openstackgerrit | Merged openstack-infra/irc-meetings: Add Nate Johnston as neutron-qos co-chair https://review.openstack.org/302706 | 23:17 |
| fungi | it might be that if i just delete all the files from the images container, everything referenced in the images_segments container will also be cleaned up | 23:17 |
| notmyname | swift doesn't keep any reference counting, so it can't automatically remove a large object manifest if the segments are missing | 23:17 |
| fungi | notmyname: thanks for confirming. that's a "no" then ;) | 23:17 |
| notmyname | fungi: but yeah, that. if you delete a large object, it can delete the referenced segments for you | 23:17 |
| fungi | oh? | 23:17 |
| fungi | got it, so other way around won't work (deleting segments won't clear the pointer) | 23:18 |
| notmyname | you can't remove the segments and have swift remove a manifest. right | 23:18 |
| fungi | but removing the pointer will let it clean up the segments | 23:18 |
| notmyname | right | 23:18 |
| notmyname | let me find the doc on that | 23:18 |
| fungi | okay, in that case this is easy | 23:18 |
| clarkb | fungi: but ya I would wait to have modred or rax confirm that those are temp files and not backing the nova use of the images directly | 23:18 |
| clarkb | because it will be sad times if we kill all our images in rax | 23:18 |
| fungi | and yeah, in this case as long as mordred confirms it's safe to do so, we can just blow away everything in these | 23:19 |
| openstackgerrit | Merged openstack-infra/irc-meetings: Update Horizon meeting times https://review.openstack.org/301503 | 23:19 |
| notmyname | fungi: http://docs.openstack.org/developer/swift/overview_large_objects.html#deleting-a-large-object | 23:19 |
| fungi | notmyname: thanks! that's useful. exactly what i was hoping for | 23:20 |
| fungi | ?multipart-manifest=delete | 23:20 |
| *** flwang has left #openstack-infra | 23:20 | |
| *** aysyd has quit IRC | 23:21 | |
| *** jcooley_ has quit IRC | 23:21 | |
| notmyname | fungi: there's 2 kinds of manifests in swift. let me make sure that works on both | 23:22 |
| *** briancline has quit IRC | 23:22 | |
| *** salv-orl_ has quit IRC | 23:24 | |
| *** briancline has joined #openstack-infra | 23:25 | |
| *** ayoung has quit IRC | 23:26 | |
| *** jamesmcarthur has joined #openstack-infra | 23:27 | |
| *** tqtran has quit IRC | 23:27 | |
| notmyname | fungi: hmm...bad news. that only works on static large objects (SLOs). to be fair, that's what glance *should* be using. but the ?multipar-manifest=delete doesn't work to recursively delete dynamic large objects (DLOs) | 23:28 |
| *** markvoelker has quit IRC | 23:29 | |
| fungi | notmyname: is there a particular format to the segment trees which can be used to identify slo vs dlo? | 23:29 |
| notmyname | fungi: based on the way the segments are stored? no. but if you do a GET with ?multipart-manifest=get, you'll see the manifest file itself | 23:30 |
| fungi | notmyname: oh, excellent. thanks again | 23:31 |
| *** sridhar_ram has joined #openstack-infra | 23:31 | |
| notmyname | fungi: and a DLO has an empty body with the x-object-manifest header. an SLO has a json body and the x-static-large-object: true header | 23:31 |
| fungi | awesome. i'll query this one here in a bit | 23:32 |
| *** ddieterly is now known as ddieterly[away] | 23:33 | |
| *** jamesmcarthur has quit IRC | 23:33 | |
| *** ddieterly[away] has quit IRC | 23:33 | |
| fungi | oh, in fact, the header details on one in the rax dashboard also show that | 23:33 |
| *** briancline has quit IRC | 23:34 | |
| fungi | unfortunately: X-Object-Manifest images_segments/template-ce... | 23:34 |
| *** jamielennox|away is now known as jamielennox | 23:36 | |
| *** ddieterly has joined #openstack-infra | 23:36 | |
| *** nwkarsten has quit IRC | 23:37 | |
| fungi | as an experiment, i've tried deleting one dlo manifest through the rackspace dashboard, and am watching to see if the segments tree for it eventually vanishes | 23:38 |
| fungi | it was from march, so shouldn't impact anything current anyway | 23:38 |
| *** mylu has quit IRC | 23:38 | |
| *** briancline has joined #openstack-infra | 23:38 | |
| fungi | so far it's still there | 23:39 |
| *** bpokorny has joined #openstack-infra | 23:40 | |
| *** bpokorny_ has quit IRC | 23:40 | |
| *** ZZelle_ has quit IRC | 23:41 | |
| *** Jeffrey4l_ has quit IRC | 23:42 | |
| anteaya | tonyb: does this have any hope of merging before the end of next week? https://review.openstack.org/#/c/296825/ | 23:44 |
| anteaya | fungi: sorry was on the phone to my parents, they fly home tomorrow | 23:44 |
| anteaya | sounds like they are in good shape for the flight | 23:44 |
| *** mylu has joined #openstack-infra | 23:45 | |
| tonyb | anteaya: 95% by the end of *next* week. | 23:45 |
| *** xarses has quit IRC | 23:45 | |
| fungi | still there. i don't think rackspace's dashboard integration for swift cleans up dlo segments either | 23:46 |
| *** bpokorny has quit IRC | 23:46 | |
| tonyb | anteaya: we need https://review.openstack.org/#/c/302543/ or land *or* for me to change irc-meetings to pull from git rather than pypi | 23:46 |
| * tonyb will publish that chnage today and ask ttx to +2 it pending resolution on 302543 | 23:47 | |
| *** bpokorny has joined #openstack-infra | 23:47 | |
| anteaya | tonyb: thank you | 23:48 |
| anteaya | tonyb: let me know if I can do anything to support | 23:48 |
| anteaya | tonyb: go you! | 23:48 |
| *** akshai has joined #openstack-infra | 23:48 | |
| *** ddieterly is now known as ddieterly[away] | 23:48 | |
| *** gildub has joined #openstack-infra | 23:48 | |
| mordred | fungi: reading | 23:49 |
| anteaya | dhellmann: hedgehog cafe! http://www.cbc.ca/news/world/peek-inside-tokyo-s-latest-animal-caf%C3%A9-hint-it-s-full-of-hedgehogs-1.3525251 | 23:49 |
| mordred | fungi, clarkb: I actually have no clue if deleting the swift objects has any effect on the image | 23:50 |
| mordred | it's unclear and one thing I do not like about task-import | 23:50 |
| mordred | however, I _think_ that someone told me at one point that glance does, in fact, copy the image from the user's swift into its own storage area | 23:50 |
| clarkb | mordred: ok, did you see the other issue with 2 step glance image uploadss? we should delete the imgae if upload fails | 23:50 |
| clarkb | because glance does not | 23:50 |
| mordred | clarkb: with 2-step PUTs? | 23:51 |
| fungi | do we know whether rackspace is using the default import task (such that we could infer this from public source code)? | 23:51 |
| fungi | or who we can ask? | 23:51 |
| clarkb | mordred: using glance you do an aimge create then an image upload. When the image upload fails we need to delete the image | 23:51 |
| mordred | fungi: rosmaita is the person to ask | 23:51 |
| clarkb | mordred: we had somewhere around 600 leaked image | 23:51 |
| *** moravec has joined #openstack-infra | 23:51 | |
| mordred | clarkb: neat! | 23:51 |
| clarkb | mordred: the vast majority (over 95% were uploads not snapshots) | 23:52 |
| clarkb | and many were in a SAVING state which I think means that the upload for them failed and we never cleaned them up | 23:52 |
| mordred | notmyname: fwiw, these are not Large Objects created by glance -- we upload them | 23:53 |
| mordred | notmyname: so we can upload them differently if there is a better way to upload them | 23:53 |
| fungi | mordred: oh, shade actually chooses dlo explicitly for this? | 23:53 |
| mordred | shade explcitly uploads using SwiftService | 23:53 |
| fungi | sounds like using slo instead of dlo might make it easier to clean up after the fact if something goes bad | 23:54 |
| fungi | since you don't have to perform your own mapping from the manifest header back to the segment files | 23:54 |
| *** akshai_ has joined #openstack-infra | 23:54 | |
| mordred | fungi: http://git.openstack.org/cgit/openstack-infra/shade/tree/shade/openstackcloud.py#n4055 | 23:55 |
| mordred | swiftclient.service.SwiftUploadObject(source=filename, object_name=name) | 23:56 |
| *** Sukhdev_ has joined #openstack-infra | 23:57 | |
| *** akshai has quit IRC | 23:58 | |
| notmyname | mordred: ah ok. so SLOs are almost always better to use. you should be doing that. they're explicit, faster, and had the nice recursive delete feature | 23:58 |
| *** markvoelker has joined #openstack-infra | 23:59 | |
| notmyname | mordred: and a cool new thing is that you can actually reference *ranges* of other objects in a SLO now | 23:59 |
| openstackgerrit | Colleen Murphy proposed openstack-infra/puppet-pip: Add new pip provider to use pypi mirrors https://review.openstack.org/302396 | 23:59 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!