| clarkb | so its really nice to fixproperly upfront if possible imo | 00:00 |
|---|---|---|
| armax | experience thought me the hard way that the longer it takes to fix a gate issue, the more disgruntled people may get, especially during a busy time like RC | 00:01 |
| armax | I am not in the position to deliver the right fix right now | 00:01 |
| prometheanfire | this is devstack, so bash is ok? | 00:02 |
| * prometheanfire hasn't worked on devstack | 00:02 | |
| *** Apoorva has quit IRC | 00:03 | |
| clarkb | prometheanfire: yes its devstack and bash | 00:03 |
| * clarkb writes a patch | 00:03 | |
| StevenK | I've got a little bash script to find the interface name if that helps | 00:03 |
| fungi | armax: oh, you abandoned it? i was about to push up a new patchset for you | 00:04 |
| *** devananda is now known as devananda|dinner | 00:04 | |
| prometheanfire | for interface in $(find /sys/class/net/ -maxdepth 1 -type l | rev | cut -d '/' -f 1 | rev); do if [[ $(cat /sys/class/net/${interface}/type) -eq 1 ]]; then echo ${interface}; fi; done | 00:04 |
| fungi | at least that way you have an alternative if the rabbit hole on interface names goes much deeper | 00:05 |
| clarkb | StevenK: no its already in devstack | 00:05 |
| prometheanfire | for interface in $(find /sys/class/net/ -maxdepth 1 -type l | rev | cut -d '/' -f 1 | rev); do if [[ $(cat /sys/class/net/"${interface}"/type) -eq 1 ]]; then echo "${interface}"; fi; done | 00:05 |
| clarkb | StevenK: all I should have to do is edit one line in devstack to set the default for LB_PHYSICAL_INTERFACE to the interface devsatck knows to use | 00:05 |
| prometheanfire | now with quotes | 00:05 |
| clarkb | I jsut ahve to find what the variable name for that is | 00:05 |
| prometheanfire | not sure that's enough for you | 00:06 |
| prometheanfire | you could probably filter on the first character being e | 00:06 |
| clarkb | this is what I was trying to say devstack already knows... so it should be really simple | 00:06 |
| armax | fungi: I thought we didn’t want to pursue that route | 00:06 |
| clarkb | we had to use the same info when fixing RAs for osic | 00:06 |
| fungi | armax: i was still writing it up as a fallback, but i guess if this solution doesn't work out as a quick fix i can push up the regex change to move the xenial version to experimental and still run the trusty version of that job on all branches | 00:07 |
| prometheanfire | this returns my only ethernet interface | 00:08 |
| fungi | so that you're not missing out on test coverage while you ping the experimental xenial version for a few days (if it ends up dragging on that long) | 00:08 |
| prometheanfire | for interface in $(find /sys/class/net/ -maxdepth 1 -type l | rev | cut -d '/' -f 1 | rev); do if [[ $(cat /sys/class/net/"${interface}"/type) -eq 1 ]]; then echo "${interface}" | grep -E '^e.*'; fi; done | 00:08 |
| armax | prometheanfire: I am not sure that determine the interface in such a way works in all cases though | 00:08 |
| openstackgerrit | Graham Hayes proposed openstack-infra/project-config: Add new worker-model job, and remove 2 legacy dvsm jobs https://review.openstack.org/360695 | 00:09 |
| armax | prometheanfire: might be fine in the gate though | 00:09 |
| armax | fungi: I can restore it if you want it as an option | 00:09 |
| prometheanfire | armax: given that names can be WHATEVER, it's not perfect, but likely fine for the gate | 00:09 |
| clarkb | https://review.openstack.org/367699 there thats based on what we did for RAs | 00:09 |
| * armax looks | 00:10 | |
| *** shu-mutou has joined #openstack-infra | 00:10 | |
| clarkb | we'll have to edit the job to stop hard setting the value so this isn't actually self testing... | 00:10 |
| armax | clarkb: we’d need to make this depend on the cleanup of .. | 00:10 |
| armax | clarkb: right | 00:10 |
| openstackgerrit | Merged openstack-infra/jenkins-job-builder: Update clone-workspace plugin https://review.openstack.org/366894 | 00:11 |
| clarkb | I am going to push a child patch that hardsets it in devstack to test | 00:11 |
| clarkb | which we can abandon when we do the other things | 00:11 |
| prometheanfire | clarkb: that prints my wireless, but is likely fine for gate | 00:11 |
| armax | clarkb, prometheanfire: the caveat with the initialization LB_PHYSICAL_INTERFACE=${LB_PHYSICAL_INTERFACE:-$default_route_dev} | 00:12 |
| clarkb | https://review.openstack.org/367701 is the next one that will work | 00:12 |
| clarkb | prometheanfire: that means your default route goes out your wireless network :) | 00:12 |
| armax | clark, prometheanfire is that one should not the physical interface for traffic be the one associated to the default route | 00:12 |
| *** SumitNaiksatam has quit IRC | 00:12 | |
| fungi | worth noting, the workaround in 367699 specifically looks for an ipv4 default route. if you had a v6-only system this needed to work on, you'd need a couple extra commands | 00:12 |
| armax | but I guess one has still the ability to override the value | 00:12 |
| clarkb | armax: eth0 is the default route interface on our trusty images | 00:13 |
| prometheanfire | clarkb: of course :P | 00:13 |
| clarkb | armax: so at least for testing that is how we were using it | 00:13 |
| *** kvcobb has joined #openstack-infra | 00:13 | |
| prometheanfire | fungi: true | 00:13 |
| armax | clarkb: yeah, I think this is fine | 00:13 |
| clarkb | fungi: well it was good enough for RAs... | 00:13 |
| armax | both in the gate and not | 00:13 |
| *** markvoelker has joined #openstack-infra | 00:13 | |
| armax | because the variable needed to be set explicitely anyway in order for Linuxbridge to work | 00:14 |
| *** zul has quit IRC | 00:14 | |
| clarkb | I do think however that a good long term plan for testing of this stuff is to only rely on things you can control by default | 00:15 |
| clarkb | definitely make it configurable so that you can use devstack on your local machine to get externally networked VMs | 00:16 |
| clarkb | but seems like by default what is important is that the host is not broken and the VMs get conenctivity which is why I like the detached setup that is managed directly | 00:16 |
| *** tqtran has quit IRC | 00:16 | |
| clarkb | and we haev had really good results with it for multinode testing | 00:16 |
| *** adrian_otto has quit IRC | 00:17 | |
| clarkb | armax: prometheanfire fungi telnet://23.253.48.59:19885 is the job running against the devstack change that should work. If you want to follow it | 00:21 |
| clarkb | fungi: is there a project-config chagne I can review while we wait for devstack to do its thing? | 00:21 |
| prometheanfire | neat | 00:22 |
| *** senk has quit IRC | 00:23 | |
| EmilienM | clarkb: ack your comment on https://review.openstack.org/#/c/367551/9/run_tests.sh i'll work on it to make it better | 00:25 |
| *** fguillot has joined #openstack-infra | 00:25 | |
| clarkb | EmilienM: its a minor thing fwiw but fallocate is immedaite and dd for gigabytes is not :) | 00:25 |
| EmilienM | ack | 00:25 |
| armax | clarkb: I can post the project-config change that removes the hardcoding to eth0 if that’s what you’re asking | 00:26 |
| armax | clarkb: shouldn’t 367699 be dependent on it for the LinuxBridge job to pass? | 00:26 |
| *** kvcobb has quit IRC | 00:26 | |
| clarkb | armax: that chaneg but also if we still want to make it non voting or experimental until devstack can review and approve the fix | 00:27 |
| *** kvcobb has joined #openstack-infra | 00:27 | |
| clarkb | armax: no we don't test with the proposed jenkins job upadtes when using depends on so it wont' really affect anything. I pushed a second change to devstack that only ever uses that calculated value which will test if it works despite the job config that uses eth0 | 00:27 |
| armax | clarkb: ack | 00:28 |
| armax | clarkb: as for the linuxbridge to non-voting, it’s all down to how quickly we can get 367699 in the gate | 00:29 |
| armax | if it’s a matter of an hour or so I guess it’s probably not worth it | 00:29 |
| clarkb | armax: ya and for that I can't help (not devstack core) | 00:29 |
| armax | clarkb: you probably should become one :) | 00:29 |
| *** piet has quit IRC | 00:34 | |
| openstackgerrit | Armando Migliaccio proposed openstack-infra/project-config: Remove hardcoding to eth0 in LinuxBridge job configuration https://review.openstack.org/367704 | 00:35 |
| armax | clarkb: ^ | 00:35 |
| clarkb | armax: cool | 00:35 |
| *** shu-mutou is now known as shu-mutou-AFK | 00:36 | |
| *** gyee has quit IRC | 00:37 | |
| clarkb | armax: at what point in the linuxbridge job was it failing? eg how far along does 367701 have to get for us to know it works? | 00:37 |
| clarkb | just past devstacking? | 00:38 |
| armax | clarkb: the l2 agent failed to start | 00:38 |
| armax | clarkb: so devstack wouldn’t finish | 00:38 |
| fungi | clarkb: armax: yeah, if we need to make it experimental only then if armax wants to un-abandon 367693 i can push up a version with the branches adjusted to run trusty on master | 00:38 |
| clarkb | ok, if that job passes devstack maybe we can get a devstack core to approve then it can be enqueued | 00:38 |
| armax | we’d fail like this: | 00:38 |
| armax | http://logs.openstack.org/31/367331/1/gate/gate-tempest-dsvm-neutron-linuxbridge-ubuntu-xenial/bdba22e/logs/screen-q-agt.txt.gz | 00:38 |
| armax | fungi: I restored it if we want the option | 00:39 |
| clarkb | fungi: armax why don't we push that too just in case we can't get devstack change merged | 00:39 |
| clarkb | or if it doesn't work for some reason | 00:39 |
| *** kaisers has joined #openstack-infra | 00:40 | |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/project-config: Fix linuxbridge job breakage on xenial https://review.openstack.org/367693 | 00:40 |
| *** kaisers1 has quit IRC | 00:40 | |
| armax | clarkb: sure, if that’s okay with you | 00:40 |
| fungi | armax: clarkb: ^ (the xml comparison job result should tell us if it's overriding in the right direction) | 00:40 |
| clarkb | we should know very soon the job thats running has been devstacking for ~12 minutes | 00:40 |
| armax | clarkb: I am pretty positive that https://review.openstack.org/#/c/367701/ will pass | 00:41 |
| armax | the incognito is how long it is going for https://review.openstack.org/#/c/367699/ to go through the usual churn | 00:42 |
| armax | hoping no intermittent failures set us back | 00:42 |
| clarkb | armax: we can promote it too | 00:42 |
| anteaya | that is an interesting use of incognito | 00:43 |
| * anteaya goes to a dictionary | 00:43 | |
| prometheanfire | :( | 00:43 |
| armax | anteaya: I misspoke, I meant Unknown | 00:44 |
| anteaya | it has undisclosed and unidentified as synonyms | 00:44 |
| armax | incognita being the Italian word for it | 00:44 |
| anteaya | no, no actually it is really good | 00:44 |
| anteaya | I've just never seen it used like that before | 00:44 |
| anteaya | thank you | 00:44 |
| clarkb | armax: the dvr job is ok on xenial? | 00:45 |
| armax | clarkb: looks like it is | 00:45 |
| anteaya | glad to learn the italian, thanks | 00:45 |
| armax | anteaya: heh | 00:45 |
| anteaya | :) | 00:45 |
| *** gongysh has quit IRC | 00:45 | |
| fungi | whee... oscon is the same week as the pike summit | 00:45 |
| *** ilyashakhat_mobi has joined #openstack-infra | 00:46 | |
| anteaya | same city? | 00:46 |
| clarkb | anteaya: no | 00:46 |
| anteaya | thank goodness | 00:46 |
| clarkb | oscon is staying in austin | 00:46 |
| prometheanfire | pike summit is atl right? | 00:46 |
| fungi | boston | 00:46 |
| anteaya | pike gathering is atlanta | 00:46 |
| anteaya | summit is boston | 00:46 |
| prometheanfire | so confusing | 00:46 |
| clarkb | armax: prometheanfire ok it started tempest so I think tahts the fix | 00:47 |
| clarkb | or possible fix. I have also +2'd the project-config change | 00:47 |
| prometheanfire | clarkb: I saw that, I hope the revert is quick is all | 00:47 |
| armax | clarkb: I’ll get a message out on the ML | 00:47 |
| clarkb | is anyone from devstack core willing to review 367699 (and its child?) | 00:47 |
| clarkb | ianw: mtreinish ^ | 00:47 |
| clarkb | if we can get a devsatck core willing to ack 367699 then I will approve the change to remove eth0 from the job config | 00:48 |
| anteaya | so do we want 367693 now? | 00:48 |
| anteaya | or wait? | 00:48 |
| clarkb | otherwise we can approve https://review.openstack.org/367693 | 00:48 |
| clarkb | anteaya: if a devstack core doesn't say anything in the next minute lets go ahead with 367693 | 00:48 |
| anteaya | clarkb: very good | 00:48 |
| * anteaya stands by | 00:48 | |
| clarkb | the neveryone can go back to their evenings and we can beg for devstack reviews in the morning | 00:49 |
| *** kzaitsev_mb has joined #openstack-infra | 00:49 | |
| clarkb | *then everyone | 00:49 |
| anteaya | I liked the concept of the neveryone | 00:50 |
| anteaya | begs a story to go with | 00:50 |
| fungi | neveryone of the nethernet | 00:51 |
| anteaya | sounds like we have our title | 00:51 |
| prometheanfire | rename newton to nethernet | 00:52 |
| prometheanfire | sed should get the job done | 00:52 |
| clarkb | well I don't hear anyone from devstack probably best to just approve the workaround. We have a fix and a way to test it we just need reviews which aren't happening | 00:52 |
| prometheanfire | ya, can revert if needed | 00:52 |
| fungi | did anyone get a chance to check the xml from that change? | 00:52 |
| clarkb | anteaya: want to review https://review.openstack.org/367693 | 00:52 |
| clarkb | fungi: no not yet | 00:53 |
| *** ilyashakhat_mobi has quit IRC | 00:53 | |
| clarkb | fungi: but it looks right | 00:53 |
| clarkb | also no xml involved because no jjb | 00:53 |
| anteaya | clarkb: reviewed already I just hadn't vote, will approve | 00:53 |
| anteaya | voted | 00:53 |
| fungi | clarkb: er, not xml but the layout job, sorry | 00:53 |
| fungi | i'll have a peek at the log now to be sure | 00:53 |
| anteaya | off it goes | 00:54 |
| *** thorst has joined #openstack-infra | 00:54 | |
| *** rajinir has quit IRC | 00:55 | |
| clarkb | fungi: I am not sure I know how to parse that job log to do the thing you want to do | 00:55 |
| anteaya | I just tried and I can't | 00:56 |
| fungi | yeah, i'm trying to figure that out myself. was pretty sure for each pipeline it lists the matching for a job | 00:57 |
| anteaya | four instances of gate-tempest-dsvm-neutron-linuxbridge-ubuntu-trusty in the log all followed by a cloud of matches | 00:57 |
| fungi | trying to see what branches are matching it | 00:57 |
| fungi | hopefully .* | 00:57 |
| anteaya | yeah I tried to look for branch information too, can't see in the noise | 00:57 |
| anteaya | 3 are independent pipeline manager one is dependent pipeline manager | 00:58 |
| clarkb | also I WIP'd https://review.openstack.org/#/c/367704/1 because we wnat to coordinate merging that with merging devstack fix | 00:58 |
| clarkb | don't want one without the other | 00:58 |
| anteaya | is a depends on in order here? | 00:59 |
| clarkb | anteaya: its sort ofa mutual depends on | 00:59 |
| anteaya | okay | 00:59 |
| fungi | the file matchers make it really hard to follow | 01:00 |
| anteaya | indeed | 01:00 |
| fungi | the dependent one is the gate, independents are (i think) check, experimental and silent? | 01:00 |
| anteaya | I have no basis to refute so I'll just accpet | 01:01 |
| anteaya | accept | 01:01 |
| *** mtreinish has quit IRC | 01:01 | |
| anteaya | as it sounds reasonable to me | 01:01 |
| anteaya | we have FileMatcher and ProjectMatcher but no BranchMatcher | 01:01 |
| *** thorst has quit IRC | 01:02 | |
| *** kzaitsev_mb has quit IRC | 01:02 | |
| fungi | i think the fact that those lines _don't_ end with ^(?!stable/(liberty|mitaka)).*$ tells us it's correct? | 01:02 |
| clarkb | maybe? | 01:02 |
| fungi | or, rather, start with | 01:02 |
| fungi | compare to a grep for the xenial version of the job | 01:03 |
| openstackgerrit | Merged openstack-infra/project-config: Fix linuxbridge job breakage on xenial https://review.openstack.org/367693 | 01:03 |
| anteaya | I got this from the dependent pipeline manager line: http://paste.openstack.org/show/569494/ | 01:03 |
| fungi | aha! yes, the match lines for the trusty version start with ^.* | 01:04 |
| fungi | it's just almost impossible to see with all those project/file matches after it | 01:04 |
| fungi | so we should be good | 01:04 |
| anteaya | oh good | 01:04 |
| *** zhurong has joined #openstack-infra | 01:05 | |
| *** mtreinish has joined #openstack-infra | 01:05 | |
| fungi | anyway, i guess we'll know soon enough since it merged a couple minutes ago | 01:05 |
| *** tonytan4ever has joined #openstack-infra | 01:05 | |
| * clarkb wonders if we need to be a lot more explicot about noy assuming things on thr test nodes | 01:06 | |
| clarkb | dont assume interface names or count, dont assume routable ipv4, dont assume swap, dont assume cpu count, dont assume architecture (I wish) etc | 01:07 |
| *** jamesdenton has quit IRC | 01:07 | |
| *** jamesdenton has joined #openstack-infra | 01:08 | |
| clarkb | dont assume cpu flags | 01:08 |
| *** thorst has joined #openstack-infra | 01:09 | |
| prometheanfire | cloudnull: yep :P | 01:10 |
| clarkb | things you can rely on: 8GB of ram at least 80gb of disk, root access, ext4 filesystem | 01:10 |
| *** gongysh has joined #openstack-infra | 01:11 | |
| ianw | clarkb: i remember we had a change out that was complaining about failing when no default route on compute nodes | 01:11 |
| clarkb | ianw: really? we should always have default routes on all our current machines | 01:12 |
| clarkb | otherwise how would zuul work | 01:12 |
| ianw | clarkb: that was what i said ... finding it ... | 01:12 |
| fungi | well, _a_ default route (currently at least ipv4, but someday could be v6-only) | 01:12 |
| ianw | clarkb: https://review.openstack.org/#/c/361572/ | 01:13 |
| *** ociuhandu has quit IRC | 01:13 | |
| fungi | our dual-stack nodes do technically have more than one default route (one for each configured address family) | 01:13 |
| clarkb | ianw thats for randompeople doing random things :) I think we can tell them to set up machines properly | 01:14 |
| *** Apoorva has joined #openstack-infra | 01:14 | |
| clarkb | ianw: but maybe we ahould centralize that determination in devstack rather than have it in mulyiple places for dofferent neutron files | 01:14 |
| clarkb | I will defer to your thoughts on that now | 01:14 |
| ianw | clarkb: yes, i hadn't read it since it got abandoned, i think as explained it's sufficiently corner case | 01:14 |
| anteaya | only 2 neutron changes in zuul, one in check and one in the gate, they both started before the change merged | 01:17 |
| *** rossella_s has quit IRC | 01:18 | |
| fungi | should wait for zuul's last reconfigured time to update on the status page, then recheck a failed neutron change to see updated jobs | 01:18 |
| anteaya | k | 01:18 |
| *** rossella_s has joined #openstack-infra | 01:19 | |
| *** IlyaG has quit IRC | 01:21 | |
| *** spzala has joined #openstack-infra | 01:21 | |
| *** yanyanhu has joined #openstack-infra | 01:21 | |
| *** kaisers_ has joined #openstack-infra | 01:22 | |
| *** jamesdenton has quit IRC | 01:25 | |
| anteaya | fungi: feel like addressing the post to dev about the recaptcha on the wiki? | 01:25 |
| anteaya | seems malini has not been following the precarious state of the wiki | 01:25 |
| *** spzala has quit IRC | 01:26 | |
| *** thcipriani is now known as thcipriani|afk | 01:26 | |
| anteaya | fungi: you'd be more diplomatic than I, I do believe | 01:26 |
| *** kaisers_ has quit IRC | 01:27 | |
| kevinbenton | clarkb: for the vxlan tunnel setup by infra in multi-node, how is the VNI identifier selected? | 01:36 |
| anteaya | zuul last updated 5.5 hours ago and I'm getting tired of waiting | 01:36 |
| anteaya | hello kevinbenton | 01:36 |
| kevinbenton | anteaya: hello | 01:36 |
| kevinbenton | anteaya: long time, no see! | 01:37 |
| anteaya | kevinbenton: I know it, how've you been? | 01:37 |
| fungi | anteaya: diplomacy applied with a trowel | 01:37 |
| anteaya | fungi: like only you can | 01:37 |
| kevinbenton | anteaya: busy busy | 01:37 |
| kevinbenton | anteaya: you? | 01:37 |
| anteaya | less busy than you online, extremely busy offline for the past 4 weeks | 01:38 |
| anteaya | but happy as the effects of the offline things are marvelous | 01:38 |
| *** yamahata has joined #openstack-infra | 01:38 | |
| kevinbenton | that's good | 01:38 |
| anteaya | neutron seems to be in reasonable shape this round to my eyes | 01:38 |
| anteaya | thank you | 01:38 |
| kevinbenton | are you going to barcelona? | 01:38 |
| anteaya | do you feel the same or am I wearing rose coloured glasses | 01:39 |
| kevinbenton | that's good to hear :) | 01:39 |
| anteaya | thanks | 01:39 |
| anteaya | I won't be at summit, no | 01:39 |
| kevinbenton | yeah, been busy fixing bugs and making things stable | 01:39 |
| clarkb | kevinbenton: we use the last octet of the ipv4 address for the "core" of thr overlay iirc | 01:39 |
| anteaya | the things I am doing offline require me to stay home this fall | 01:39 |
| anteaya | yay for making this stable, thank you kevinbenton | 01:39 |
| clarkb | kevinbenton: probably there are better eays but that already is unique so we overloaded iirc | 01:39 |
| kevinbenton | anteaya: ah, makes sense | 01:39 |
| anteaya | kevinbenton: I'll miss you, that will make it a long time since I've seen you | 01:39 |
| anteaya | kevinbenton: so I'll just look forward to the next opportunity :) | 01:40 |
| kevinbenton | anteaya: yeah, will have to catch up in Atlanta I suppose | 01:40 |
| clarkb | I would have to double check by reading the code though | 01:40 |
| *** zshuo has joined #openstack-infra | 01:40 | |
| anteaya | that probably will be the next opportunity, yeah | 01:40 |
| kevinbenton | clarkb: so the VNI identifier just seems to be "2" in this case | 01:41 |
| kevinbenton | clarkb: http://logs.openstack.org/91/367191/2/check/gate-tempest-dsvm-neutron-dvr-multinode-full-ubuntu-xenial/7fd0e33/logs/worlddump-2016-09-08-142826.txt.gz | 01:41 |
| kevinbenton | clarkb: ctrl-f 'options: {key="2", local_ip="10.201.127.234",' | 01:41 |
| *** gongysh has quit IRC | 01:41 | |
| clarkb | ya rereading it its the ip addrs last octet for the remotes | 01:42 |
| clarkb | so I was almost right | 01:42 |
| clarkb | core gets .1 remote gets .2 thus vni is 2 | 01:43 |
| anteaya | I'm going offline now, zuul hasn't updated recently so I haven't rechecked any neutron patches | 01:43 |
| anteaya | g'night | 01:44 |
| kevinbenton | anteaya: night | 01:44 |
| kevinbenton | clarkb: so does that mean the other side is using a different VNI ? | 01:44 |
| clarkb | kevinbenton: no its both 2 | 01:44 |
| kevinbenton | clarkb: ok. so next question might be silly, but why aren't they just static? | 01:45 |
| *** Apoorva has quit IRC | 01:45 | |
| clarkb | why isnt the vni static? because we sometimes have more than one overlay or more than 2 hosts (though wr havnet implemented more than 2 hosts yet | 01:45 |
| clarkb | nova net requires two overlays | 01:46 |
| *** piet has joined #openstack-infra | 01:46 | |
| kevinbenton | clarkb: i see | 01:46 |
| clarkb | you cant share the l2 due to nova net ebtables | 01:46 |
| kevinbenton | clarkb: so does that mean the VNI will always be < 10 ? | 01:47 |
| kevinbenton | clarkb: in infra? | 01:47 |
| kevinbenton | clarkb: i need to submit a patch to make sure neutron isn't using the same VNIs | 01:48 |
| kevinbenton | clarkb: it's causing issues since br-ex is managed by the agent | 01:48 |
| clarkb | no I think that we use a /25 so the last octet on thr second overlays second host is.129 | 01:48 |
| clarkb | kevinbenton: I'm not sure I folloe | 01:48 |
| clarkb | br-ex is not managed which us ehy its such a pita | 01:48 |
| clarkb | if you didnt manually create one you get a dangling one | 01:49 |
| kevinbenton | clarkb: er, sorry. i mean the agent will wire things between it and br-int | 01:49 |
| clarkb | but that doesnt touch vxlan thats just a local veth pair right? | 01:50 |
| kevinbenton | clarkb: yeah, actually that doesn't matter. the issue seems to be that OVS has multiple tunnels using the same VNI | 01:51 |
| clarkb | oh fun. I recommend neutron not use existing IDs :) | 01:51 |
| *** jamesdenton has joined #openstack-infra | 01:51 | |
| kevinbenton | clarkb: yes | 01:51 |
| kevinbenton | clarkb: i'm seeing things that belong to a neutron network decapped onto br-ex because of it | 01:51 |
| clarkb | all yhis setup happens before neuyron starts so as long as ypu pick others in the 12bit ramge should be fine | 01:51 |
| kevinbenton | clarkb: yep, which brings me to one last question | 01:52 |
| kevinbenton | clarkb: where should i put the config change for this? | 01:52 |
| kevinbenton | clarkb: devstack-gate? | 01:52 |
| clarkb | no | 01:52 |
| clarkb | neutron | 01:52 |
| clarkb | neutron shouodnt reuse vnis | 01:52 |
| kevinbenton | clarkb: it's not reusing VNIs | 01:53 |
| kevinbenton | clarkb: the operator used a VNI and then told neutron it was safe to use one | 01:53 |
| clarkb | what? | 01:53 |
| clarkb | you mean ops have to explicitly set the range? | 01:53 |
| kevinbenton | clarkb: neutron is configured with a range of VNIs to safely allocate from for tenant networks | 01:53 |
| clarkb | ugh | 01:53 |
| clarkb | this is the myu stuff all over again | 01:53 |
| kevinbenton | clarkb: yes, because there is no way to know what is going to be on each agent | 01:53 |
| clarkb | why not? | 01:54 |
| clarkb | you cant ask the local ovs? | 01:54 |
| kevinbenton | clarkb: this is the neutron server | 01:54 |
| clarkb | yes but you have agents rumning eberywhere | 01:54 |
| kevinbenton | clarkb: do you propose we query every agent to ask what VNI is in use? | 01:54 |
| clarkb | well everywhere that matters | 01:54 |
| clarkb | its the same mtu issur all over again | 01:54 |
| kevinbenton | clarkb: everywhere that matters is any possible agent | 01:54 |
| clarkb | neutron has thr data and should figure it out | 01:54 |
| kevinbenton | clarkb: no it doesn't | 01:54 |
| kevinbenton | clarkb: a VNI is a resource of the datacenter | 01:55 |
| kevinbenton | clarkb: there can be VNIs in use in the datacenter that isn't in use on the agents | 01:55 |
| kevinbenton | clarkb: and it's still unsafe to use | 01:55 |
| clarkb | but that only matters if you have ips that overlap right? | 01:55 |
| clarkb | in ehich case the agemt would know? | 01:56 |
| clarkb | reusing ids in non overlapping tunnels shpuld be fine | 01:56 |
| kevinbenton | clarkb: VNIs can be carried in multicast traffic (linux bridge can do this for it's broadcast data) | 01:56 |
| kevinbenton | clarkb: if neutron arbitrarily grabs a VNI | 01:57 |
| kevinbenton | clarkb: it can step on VNIs in use by the datacenter | 01:57 |
| clarkb | I see (we dont use mulyicast because we cant) | 01:57 |
| fungi | well, assuming they reuse the same multicast groups i guess? | 01:57 |
| clarkb | right that too | 01:57 |
| kevinbenton | fungi: yep | 01:57 |
| fungi | i mean, reusing multicast groups on your network is also a bad idea if you don't know they won't carry conflicting traffic | 01:58 |
| clarkb | ok what is the default range used by drvstack/neutron? | 01:58 |
| kevinbenton | devstack looks to select 1:1000 | 01:58 |
| clarkb | kevinbenton: we can use 8192 + offset in devstack-gate then. | 01:59 |
| clarkb | or similar | 01:59 |
| *** kzaitsev_mb has joined #openstack-infra | 01:59 | |
| kevinbenton | clarkb: yep, anything that will be well out of your way | 01:59 |
| clarkb | would need to edit bottom of functions.sh | 01:59 |
| kevinbenton | clarkb: we could even do something exotic like 1234567 | 01:59 |
| kevinbenton | clarkb: nobody ever uses the high numbers | 01:59 |
| fungi | right, too big to remember ;) | 02:00 |
| clarkb | kevinbenton: I'll let ypu pick yhe offset :) | 02:00 |
| kevinbenton | clarkb: ok. where am i submitting this? | 02:00 |
| clarkb | devstack-gate/funcntions.sh | 02:00 |
| *** gongysh has joined #openstack-infra | 02:01 | |
| clarkb | set key = offset + additional offset | 02:01 |
| clarkb | where additional offset is whatever you pock | 02:01 |
| * clarkb wishes he could just go back to gre oh well | 02:01 | |
| kevinbenton | clarkb: ah, i see. changing the VNI on infra side | 02:02 |
| kevinbenton | clarkb: rather than the pool neutron uses | 02:02 |
| clarkb | kevinbenton: ya to avoid non backward compat with devstack | 02:02 |
| kevinbenton | clarkb: oh, for grenade jobs, right? | 02:02 |
| clarkb | not sure if changing devstack will break anything but devstack gate is uses in single use things | 02:02 |
| clarkb | kevinbenton: ya or people with local installs | 02:03 |
| fungi | while this is riveting, i think i need to retire for the evening. later, all | 02:03 |
| clarkb | fungi: goodnight | 02:03 |
| kevinbenton | clarkb: oh, well i meant the change would still be devstack-gate, just in setting the tunnel ranges for neutron | 02:03 |
| kevinbenton | clarkb: but this works as well | 02:03 |
| openstackgerrit | Rui Chen proposed openstack-infra/project-config: Create a new project python-nimbleclient https://review.openstack.org/367146 | 02:03 |
| clarkb | kevinbenton: no if we were to chamge neutrons range that should go in devstack so just changing the range multinode uses is best | 02:04 |
| kevinbenton | clarkb: ack | 02:04 |
| *** akshai has joined #openstack-infra | 02:04 | |
| *** adrian_otto has joined #openstack-infra | 02:05 | |
| clarkb | hrm zuul still hasnt reloaded its config | 02:05 |
| *** yamahata has quit IRC | 02:07 | |
| clarkb | kevinbenton: what changed so that wr are only noticing this now? | 02:08 |
| kevinbenton | clarkb: it's a pretty rare failure | 02:09 |
| kevinbenton | clarkb: IIRC the neutron allocation strategy for VNIs is to go through the pool before starting over | 02:10 |
| *** adrian_otto has quit IRC | 02:11 | |
| clarkb | looked like we had an old stuck ansible run from just before the fix for linuxbridge things merged so I went ahead and cleared those processes out hoping the next run through will be happier | 02:13 |
| *** adrian_otto has joined #openstack-infra | 02:13 | |
| openstackgerrit | Kevin Benton proposed openstack-infra/devstack-gate: Don't use 1:1000 VNI range for node-to-node tunnels https://review.openstack.org/367716 | 02:14 |
| *** roxanagh_ has joined #openstack-infra | 02:15 | |
| clarkb | and ansible is running again | 02:15 |
| clarkb | kevinbenton: is it expected that in normal operation neutron and the rest of your datacenter would overlap? Not sure how that would work since you can't specify the vni when you create a neutron entwork right? | 02:18 |
| clarkb | kevinbenton: basically this is a hard requirement from neutron today that you always set non overlapping ranges? | 02:19 |
| kevinbenton | clarkb: yeah, set non overlapping ranges | 02:19 |
| clarkb | ianw: maybe you can also review 367716? | 02:20 |
| kevinbenton | clarkb: as an admin you can specify the VNI for an individual network | 02:20 |
| clarkb | kevinbenton: ah but not as a normal user | 02:20 |
| *** sarob has joined #openstack-infra | 02:20 | |
| kevinbenton | clarkb: right, the ranges just control automatic allocation | 02:20 |
| kevinbenton | clarkb: as an admin you can even pick one outside of the range if you wanted | 02:20 |
| *** spzala has joined #openstack-infra | 02:21 | |
| ianw | clarkb: looking... | 02:22 |
| *** adrian_otto has quit IRC | 02:24 | |
| *** sarob has quit IRC | 02:24 | |
| *** chlong has joined #openstack-infra | 02:26 | |
| clarkb | there we go zuul is updated | 02:27 |
| clarkb | I rechecked https://review.openstack.org/#/c/367529/2 | 02:28 |
| clarkb | and it has queued the trusty linuxbrideg job so I think we are good for now | 02:28 |
| * clarkb returns to regularly scheduled evening | 02:28 | |
| *** salv-orl_ has joined #openstack-infra | 02:30 | |
| *** devananda|dinner has quit IRC | 02:32 | |
| *** clayton has quit IRC | 02:32 | |
| *** IlyaG has joined #openstack-infra | 02:32 | |
| *** clayton has joined #openstack-infra | 02:33 | |
| *** salv-orlando has quit IRC | 02:33 | |
| *** IlyaG has quit IRC | 02:34 | |
| *** fguillot has quit IRC | 02:34 | |
| amrith | AJaeger, at your convenience please take a look at https://review.openstack.org/#/c/367640/, it found some interesting problems :) | 02:42 |
| *** mriedem has quit IRC | 02:46 | |
| *** tonytan4ever has quit IRC | 02:46 | |
| *** spzala has quit IRC | 02:48 | |
| *** amotoki has quit IRC | 02:49 | |
| *** ilyashakhat_mobi has joined #openstack-infra | 02:49 | |
| *** piet has quit IRC | 02:51 | |
| *** ilyashakhat_mobi has quit IRC | 02:54 | |
| *** piet has joined #openstack-infra | 02:57 | |
| *** sflanigan is now known as sflanigan_afk | 02:58 | |
| *** devananda|dinner has joined #openstack-infra | 02:59 | |
| *** vinaypotluri has quit IRC | 03:02 | |
| *** roxanagh_ has quit IRC | 03:07 | |
| *** Jeffrey4l has joined #openstack-infra | 03:09 | |
| *** amitgandhinz has quit IRC | 03:11 | |
| *** kaisers_ has joined #openstack-infra | 03:11 | |
| *** tqtran has joined #openstack-infra | 03:15 | |
| *** kaisers_ has quit IRC | 03:16 | |
| *** tqtran has quit IRC | 03:19 | |
| *** sflanigan_afk is now known as sflanigan | 03:20 | |
| *** IlyaG has joined #openstack-infra | 03:20 | |
| *** Apoorva has joined #openstack-infra | 03:21 | |
| *** IlyaG has quit IRC | 03:22 | |
| *** david-lyle_ has joined #openstack-infra | 03:22 | |
| *** david-lyle_ has quit IRC | 03:22 | |
| *** roxanagh_ has joined #openstack-infra | 03:23 | |
| *** armax has quit IRC | 03:24 | |
| *** rfolco has quit IRC | 03:25 | |
| *** ociuhandu has joined #openstack-infra | 03:32 | |
| *** roxanagh_ has quit IRC | 03:33 | |
| *** jamesdenton has quit IRC | 03:34 | |
| *** ociuhandu has quit IRC | 03:36 | |
| *** tonytan4ever has joined #openstack-infra | 03:44 | |
| *** amotoki has joined #openstack-infra | 03:49 | |
| *** shu-mutou-AFK is now known as shu-mutou | 03:51 | |
| *** rhallisey has joined #openstack-infra | 03:54 | |
| *** roxanagh_ has joined #openstack-infra | 03:57 | |
| *** kaisers_ has joined #openstack-infra | 03:59 | |
| *** piet has quit IRC | 04:10 | |
| *** kzaitsev_mb has quit IRC | 04:14 | |
| *** amotoki has quit IRC | 04:16 | |
| *** tqtran has joined #openstack-infra | 04:16 | |
| *** spzala has joined #openstack-infra | 04:17 | |
| *** spzala has quit IRC | 04:17 | |
| *** rhallisey has quit IRC | 04:18 | |
| *** hamzy has joined #openstack-infra | 04:19 | |
| *** woodster_ has quit IRC | 04:19 | |
| *** tqtran has quit IRC | 04:21 | |
| *** harlowja_at_home has joined #openstack-infra | 04:23 | |
| *** zul has joined #openstack-infra | 04:27 | |
| *** rm_work has quit IRC | 04:31 | |
| *** akshai has quit IRC | 04:32 | |
| *** rm_work has joined #openstack-infra | 04:34 | |
| *** rm_work has quit IRC | 04:36 | |
| *** rm_work has joined #openstack-infra | 04:38 | |
| *** coolsvap has joined #openstack-infra | 04:43 | |
| *** psachin has joined #openstack-infra | 04:45 | |
| openstackgerrit | Merged openstack-infra/project-config: Add py3 jobs for fuel-ccp-entrypoint repo https://review.openstack.org/366412 | 04:47 |
| *** snarwade has joined #openstack-infra | 04:49 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder: Move elements & lib relative to diskimage_builder package https://review.openstack.org/344017 | 04:49 |
| *** roxanagh_ has quit IRC | 04:50 | |
| openstackgerrit | Anusha Ramineni proposed openstack-infra/project-config: Congress - Add replicated PE job https://review.openstack.org/365862 | 04:50 |
| *** ilyashakhat_mobi has joined #openstack-infra | 04:51 | |
| *** pgadiya has joined #openstack-infra | 04:52 | |
| *** thorst has quit IRC | 04:55 | |
| *** ilyashakhat_mobi has quit IRC | 04:56 | |
| *** kzaitsev_mb has joined #openstack-infra | 04:58 | |
| *** Apoorva has quit IRC | 04:58 | |
| *** roxanagh_ has joined #openstack-infra | 05:00 | |
| *** Jeffrey4l_ has joined #openstack-infra | 05:00 | |
| *** furlongm_ is now known as furlongm | 05:01 | |
| *** roxanagh_ has joined #openstack-infra | 05:02 | |
| *** thorst has joined #openstack-infra | 05:03 | |
| *** senk has joined #openstack-infra | 05:03 | |
| *** thorst has quit IRC | 05:03 | |
| *** thorst has joined #openstack-infra | 05:03 | |
| *** Jeffrey4l has quit IRC | 05:04 | |
| *** kzaitsev_mb has quit IRC | 05:04 | |
| *** roxanagh_ has quit IRC | 05:07 | |
| *** roxanagh_ has joined #openstack-infra | 05:10 | |
| *** dahoo has quit IRC | 05:13 | |
| *** zul has quit IRC | 05:13 | |
| *** ilyashakhat_mobi has joined #openstack-infra | 05:15 | |
| *** moravec has joined #openstack-infra | 05:16 | |
| *** moravec1 has quit IRC | 05:16 | |
| *** claudiub|2 has joined #openstack-infra | 05:19 | |
| *** jaosorior has joined #openstack-infra | 05:21 | |
| *** claudiub has joined #openstack-infra | 05:21 | |
| *** zul has joined #openstack-infra | 05:23 | |
| *** claudiub|2 has quit IRC | 05:25 | |
| *** burgerk has quit IRC | 05:28 | |
| *** roxanagh_ has quit IRC | 05:28 | |
| *** zshuo has quit IRC | 05:30 | |
| *** chlong_ has joined #openstack-infra | 05:31 | |
| *** Mary has joined #openstack-infra | 05:31 | |
| *** Mary is now known as Guest26802 | 05:31 | |
| *** ilyashakhat_mobi has quit IRC | 05:32 | |
| *** roxanagh_ has joined #openstack-infra | 05:33 | |
| *** harlowja_at_home has quit IRC | 05:33 | |
| *** Guest26802 has quit IRC | 05:35 | |
| *** zul has quit IRC | 05:38 | |
| *** Ravikiran_K has joined #openstack-infra | 05:41 | |
| *** amotoki has joined #openstack-infra | 05:43 | |
| *** ilyashakhat_mobi has joined #openstack-infra | 05:48 | |
| *** roxanagh_ has quit IRC | 05:49 | |
| openstackgerrit | zhurong proposed openstack-infra/project-config: Enable translation for solum https://review.openstack.org/367756 | 05:54 |
| *** adriant has quit IRC | 05:55 | |
| openstackgerrit | zhurong proposed openstack-infra/project-config: Enable translation for solum https://review.openstack.org/367756 | 05:59 |
| *** tonytan4ever has quit IRC | 06:00 | |
| *** kzaitsev_mb has joined #openstack-infra | 06:01 | |
| *** claudiub has quit IRC | 06:01 | |
| openstackgerrit | mathieu bultel proposed openstack-infra/tripleo-ci: WIP -- clean full upgrade review https://review.openstack.org/364859 | 06:04 |
| *** yamahata has joined #openstack-infra | 06:09 | |
| *** ilyashakhat_mobi has quit IRC | 06:12 | |
| *** kvcobb has quit IRC | 06:13 | |
| *** _nadya_ has joined #openstack-infra | 06:13 | |
| *** ccamacho has joined #openstack-infra | 06:13 | |
| *** _nadya_ has quit IRC | 06:17 | |
| *** berendt has quit IRC | 06:17 | |
| *** tqtran has joined #openstack-infra | 06:18 | |
| *** amotoki has quit IRC | 06:19 | |
| *** YorikSar_ has quit IRC | 06:21 | |
| *** tqtran has quit IRC | 06:23 | |
| *** e0ne has joined #openstack-infra | 06:23 | |
| *** YorikSar has joined #openstack-infra | 06:23 | |
| *** pcaruana has joined #openstack-infra | 06:23 | |
| *** amotoki has joined #openstack-infra | 06:28 | |
| openstackgerrit | yinxiulin proposed openstack-infra/project-config: tricircle: Enable neutron in tricircle tempest https://review.openstack.org/367220 | 06:29 |
| *** harlowja_at_home has joined #openstack-infra | 06:29 | |
| *** _nadya_ has joined #openstack-infra | 06:31 | |
| *** hferenc has joined #openstack-infra | 06:38 | |
| *** hashar has joined #openstack-infra | 06:38 | |
| *** Na3iL has joined #openstack-infra | 06:39 | |
| *** andreas_s has joined #openstack-infra | 06:46 | |
| *** senk has quit IRC | 06:46 | |
| *** harlowja_at_home has quit IRC | 06:47 | |
| *** harlowja_at_home has joined #openstack-infra | 06:48 | |
| *** amotoki has quit IRC | 06:48 | |
| *** ilyashakhat_mobi has joined #openstack-infra | 06:48 | |
| openstackgerrit | zhurong proposed openstack-infra/project-config: Enable translation for solum https://review.openstack.org/367756 | 06:50 |
| *** camunoz has quit IRC | 06:51 | |
| *** amitgandhinz has joined #openstack-infra | 06:52 | |
| openstackgerrit | Chaoyi Huang proposed openstack-infra/project-config: Add new project Trio2o https://review.openstack.org/367114 | 06:52 |
| *** amotoki has joined #openstack-infra | 06:52 | |
| *** harlowja_at_home has quit IRC | 06:57 | |
| *** pahuang has quit IRC | 06:57 | |
| *** mikelk has joined #openstack-infra | 06:58 | |
| *** florianf has joined #openstack-infra | 07:00 | |
| *** mikelk has quit IRC | 07:01 | |
| *** jlanoux has joined #openstack-infra | 07:02 | |
| *** amotoki has quit IRC | 07:02 | |
| *** jamielennox|away is now known as jamielennox | 07:03 | |
| *** tkelsey has joined #openstack-infra | 07:04 | |
| *** aeng has quit IRC | 07:05 | |
| *** tesseract- has joined #openstack-infra | 07:07 | |
| *** dtardivel has joined #openstack-infra | 07:07 | |
| *** flepied has quit IRC | 07:07 | |
| *** yaume has joined #openstack-infra | 07:07 | |
| *** jlanoux_ has joined #openstack-infra | 07:10 | |
| *** jlanoux has quit IRC | 07:10 | |
| *** matrohon has joined #openstack-infra | 07:12 | |
| *** claudiub has joined #openstack-infra | 07:13 | |
| *** amotoki has joined #openstack-infra | 07:14 | |
| *** Hal1 has joined #openstack-infra | 07:15 | |
| openstackgerrit | mathieu bultel proposed openstack-infra/tripleo-ci: WIP -- clean full upgrade review https://review.openstack.org/364859 | 07:17 |
| *** _nadya_ has quit IRC | 07:18 | |
| *** chlong has quit IRC | 07:19 | |
| *** oanson has joined #openstack-infra | 07:19 | |
| *** chlong_ has quit IRC | 07:19 | |
| *** ihrachys has joined #openstack-infra | 07:20 | |
| *** senk has joined #openstack-infra | 07:20 | |
| *** drifterza has joined #openstack-infra | 07:21 | |
| *** drifterza has quit IRC | 07:24 | |
| *** dizquierdo has joined #openstack-infra | 07:24 | |
| *** senk__ has joined #openstack-infra | 07:28 | |
| *** senk has quit IRC | 07:28 | |
| *** jpich has joined #openstack-infra | 07:29 | |
| *** esikachev has joined #openstack-infra | 07:32 | |
| *** gouthamr has quit IRC | 07:40 | |
| *** jlanoux_ has quit IRC | 07:40 | |
| *** berendt has joined #openstack-infra | 07:40 | |
| *** yanyanhu has quit IRC | 07:41 | |
| *** yanyanhu has joined #openstack-infra | 07:41 | |
| *** chem|off is now known as chem | 07:42 | |
| *** flepied has joined #openstack-infra | 07:42 | |
| yolanda | good morning | 07:46 |
| *** tonytan4ever has joined #openstack-infra | 07:46 | |
| *** kzaitsev_mb has quit IRC | 07:48 | |
| *** berendt has quit IRC | 07:49 | |
| jd__ | thanks AJaeger :) | 07:50 |
| *** tonytan4ever has quit IRC | 07:52 | |
| *** vsaienko has joined #openstack-infra | 07:54 | |
| *** matthewbodkin has joined #openstack-infra | 07:54 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack-infra/project-config: Promote OSA tests CentOS7 job to voting https://review.openstack.org/367799 | 07:55 |
| *** ilyashakhat_mobi has quit IRC | 07:55 | |
| *** gildub has quit IRC | 07:56 | |
| vsaienko | infra-core: could you please review https://review.openstack.org/#/c/340295/ | 07:56 |
| *** zz_dimtruck is now known as dimtruck | 07:57 | |
| *** samuelBartel has joined #openstack-infra | 07:57 | |
| *** thorst has quit IRC | 07:57 | |
| *** zzzeek has quit IRC | 08:00 | |
| *** e0ne has quit IRC | 08:00 | |
| *** thorst has joined #openstack-infra | 08:01 | |
| *** zzzeek has joined #openstack-infra | 08:02 | |
| *** terryw has quit IRC | 08:03 | |
| *** derekh has joined #openstack-infra | 08:05 | |
| *** thorst has quit IRC | 08:06 | |
| *** otherwiseguy has joined #openstack-infra | 08:07 | |
| *** cgross has quit IRC | 08:09 | |
| *** lmiccini_ has joined #openstack-infra | 08:09 | |
| *** thorst has joined #openstack-infra | 08:09 | |
| *** mkoderer has quit IRC | 08:10 | |
| *** lmiccini has quit IRC | 08:10 | |
| *** mkoderer has joined #openstack-infra | 08:11 | |
| openstackgerrit | Merged openstack-infra/project-config: tripleo/upgrades: move jobs to check https://review.openstack.org/367641 | 08:11 |
| *** cgross has joined #openstack-infra | 08:12 | |
| *** drifterza has joined #openstack-infra | 08:13 | |
| *** acoles_ is now known as acoles | 08:13 | |
| openstackgerrit | Merged openstack-infra/project-config: Add new worker-model job, and remove 2 legacy dvsm jobs https://review.openstack.org/360695 | 08:14 |
| openstackgerrit | Merged openstack-infra/project-config: Congress - Add replicated PE job https://review.openstack.org/365862 | 08:16 |
| openstackgerrit | Merged openstack-infra/project-config: New Project timmy - ansible-like tool https://review.openstack.org/359831 | 08:16 |
| *** thorst has quit IRC | 08:17 | |
| openstackgerrit | Merged openstack-infra/project-config: Grafana Neutron: use string substitution to track jobs running on different nodes https://review.openstack.org/367620 | 08:18 |
| openstackgerrit | Merged openstack-infra/project-config: Create a release notes job for glanceclient https://review.openstack.org/367473 | 08:19 |
| *** dizquierdo has quit IRC | 08:20 | |
| openstackgerrit | mathieu bultel proposed openstack-infra/tripleo-ci: WIP - Implement overcloud upgrade job - Mitaka -> Newton https://review.openstack.org/323750 | 08:21 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack-infra/project-config: Normalize projects.yaml https://review.openstack.org/367817 | 08:23 |
| *** lmiccini_ is now known as lmiccini | 08:23 | |
| *** thorst has joined #openstack-infra | 08:27 | |
| *** sflanigan has quit IRC | 08:28 | |
| openstackgerrit | Merged openstack-infra/project-config: setup post job for sahara-extra https://review.openstack.org/367271 | 08:28 |
| *** sanek has joined #openstack-infra | 08:29 | |
| *** salv-orlando has joined #openstack-infra | 08:30 | |
| *** yolanda has quit IRC | 08:31 | |
| *** Mary has joined #openstack-infra | 08:31 | |
| openstackgerrit | Merged openstack-infra/project-config: Reduce bluebox-sjc1 by 1 node https://review.openstack.org/367319 | 08:31 |
| *** Mary is now known as Guest65751 | 08:32 | |
| *** salv-orl_ has quit IRC | 08:32 | |
| *** ilyashakhat_mobi has joined #openstack-infra | 08:34 | |
| *** amitgandhinz has quit IRC | 08:34 | |
| *** YorikSar has quit IRC | 08:35 | |
| *** _nadya_ has joined #openstack-infra | 08:35 | |
| *** Guest65751 has quit IRC | 08:35 | |
| *** YorikSar has joined #openstack-infra | 08:37 | |
| *** yolanda has joined #openstack-infra | 08:37 | |
| jamespage | fungi, ah yes - (un)predictable network interface naming | 08:38 |
| jamespage | fungi, tricky | 08:38 |
| jamespage | there may be packaging bits that updated to accomodate this, but not in the openstack package set | 08:38 |
| *** dsutyagin has joined #openstack-infra | 08:38 | |
| jamespage | in terms of deployment tooling, we generally ignore interface naming now, and do things by mac address, which is less automagic | 08:39 |
| jamespage | I'd like to revisit this next cycle; make things a little less brittle | 08:39 |
| openstackgerrit | mathieu bultel proposed openstack-infra/tripleo-ci: WIP - Implement overcloud upgrade job - Mitaka -> Newton https://review.openstack.org/323750 | 08:39 |
| *** jaosorior is now known as jaosorior_lunch | 08:40 | |
| dsutyagin | Hello infra team! We have https://review.openstack.org/#/c/359831/ merged, what do we need to do to get a new repository created in openstack github? Doc says "coordinate with the Infra team". Also, how do we set up and manage the "-core" group for this project? Thanks! | 08:40 |
| *** Jeffrey4l_ has quit IRC | 08:42 | |
| *** asettle has joined #openstack-infra | 08:42 | |
| openstackgerrit | Merged openstack-infra/tripleo-ci: scenario002: deploy Swift https://review.openstack.org/367401 | 08:43 |
| *** mhickey__ has joined #openstack-infra | 08:45 | |
| *** jlanoux has joined #openstack-infra | 08:45 | |
| *** dizquierdo has joined #openstack-infra | 08:46 | |
| openstackgerrit | Merged openstack-infra/project-config: Add charm-manila https://review.openstack.org/360762 | 08:46 |
| *** dragonmaster has joined #openstack-infra | 08:47 | |
| *** bethwhite_ has joined #openstack-infra | 08:47 | |
| *** senk__ has quit IRC | 08:49 | |
| *** _degorenko|afk is now known as degorenko | 08:52 | |
| *** yar_ has quit IRC | 08:53 | |
| *** matthewbodkin has quit IRC | 08:53 | |
| AJaeger | vsaienko: please have ironic folks review this first | 08:55 |
| *** tosky has joined #openstack-infra | 08:55 | |
| vsaienko | AJaeger, there was +1 from Ironic PTL, I just resolved merge conflict | 08:55 |
| *** e0ne has joined #openstack-infra | 08:55 | |
| AJaeger | vsaienko: ah - will review later then. | 08:56 |
| *** vsaienko has left #openstack-infra | 08:56 | |
| AJaeger | dsutyagin: it takes some time until a new repo shows up. Check http://git.openstack.org/cgit/openstack/timmy | 08:56 |
| *** vsaienko has joined #openstack-infra | 08:56 | |
| vsaienko | AJaeger: thank you! | 08:56 |
| AJaeger | dsutyagin: github is just a mirror, it should sync automatically there. But first puppet needs to run its jobs everywhere | 08:57 |
| *** matthewbodkin has joined #openstack-infra | 08:57 | |
| AJaeger | dsutyagin: regarding the core group: An infra-root needs to add you to it and then you can add others to it. Hope somebody reads backscroll and will answer, otherwise ask again later giving the review. | 08:58 |
| *** sputnik13 has quit IRC | 08:58 | |
| AJaeger | dsutyagin: http://git.openstack.org/cgit/openstack/timmy has content now - you can clone and start sending in changes -but not approve until the ACL is setup | 08:59 |
| *** bauzas is now known as bauwser | 09:00 | |
| *** sdake has joined #openstack-infra | 09:02 | |
| *** amotoki has quit IRC | 09:04 | |
| *** thorst has quit IRC | 09:04 | |
| openstackgerrit | Waldemar Znoinski proposed openstack/diskimage-builder: don't configure 'lo' for dhcp https://review.openstack.org/367527 | 09:04 |
| wznoinsk | is it me or is review.o.o crawling ? | 09:05 |
| *** thorst has joined #openstack-infra | 09:05 | |
| *** nijaba has quit IRC | 09:05 | |
| openstackgerrit | mathieu bultel proposed openstack-infra/tripleo-ci: WIP - Implement overcloud upgrade job - Mitaka -> Newton https://review.openstack.org/323750 | 09:06 |
| *** nijaba has joined #openstack-infra | 09:06 | |
| *** nijaba has joined #openstack-infra | 09:06 | |
| openstackgerrit | mathieu bultel proposed openstack-infra/tripleo-ci: WIP - Implement overcloud upgrade job - Mitaka -> Newton https://review.openstack.org/323750 | 09:07 |
| *** Jeffrey4l has joined #openstack-infra | 09:08 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Complete retirement of keystoneauth-saml2 https://review.openstack.org/367844 | 09:09 |
| *** dimtruck is now known as zz_dimtruck | 09:10 | |
| *** thorst has quit IRC | 09:10 | |
| *** HeOS has joined #openstack-infra | 09:11 | |
| *** shardy has joined #openstack-infra | 09:11 | |
| *** sdake_ has joined #openstack-infra | 09:16 | |
| *** caowei has joined #openstack-infra | 09:18 | |
| *** sdake has quit IRC | 09:18 | |
| *** rossella_s has quit IRC | 09:18 | |
| *** rossella_s has joined #openstack-infra | 09:19 | |
| *** yanyanhu has quit IRC | 09:20 | |
| *** sarob has joined #openstack-infra | 09:20 | |
| *** ilyashakhat_mobi has quit IRC | 09:24 | |
| *** sarob has quit IRC | 09:25 | |
| *** thorst has joined #openstack-infra | 09:26 | |
| *** _nadya_ has quit IRC | 09:29 | |
| *** sambetts|afk is now known as sambetts | 09:30 | |
| *** thorst has quit IRC | 09:31 | |
| *** Na3iL has quit IRC | 09:32 | |
| *** r-mibu has quit IRC | 09:33 | |
| *** Hal1 has quit IRC | 09:33 | |
| *** r-mibu has joined #openstack-infra | 09:33 | |
| *** oanson has quit IRC | 09:35 | |
| *** Hal1 has joined #openstack-infra | 09:35 | |
| dtantsur | AJaeger, hi! can we please chat about this xenial thing? | 09:35 |
| dtantsur | AJaeger, I suggest we start switch with 2 jobs with "ipmitool" in there names, they won't be voting very soon, but we'll switch to them in the future. | 09:36 |
| dtantsur | I also think they're Newton-only, but I need to double-check | 09:36 |
| dtantsur | hmm, no, they run on mitaka too.. not sure how to handle that. | 09:38 |
| AJaeger | dtantsur: Let's discuss later with clarkb on what he proposes as best way going forward with ironic and xenial move, ok? | 09:38 |
| dtantsur | fine, as soon as we can move forward reasonable fast :) | 09:38 |
| dtantsur | I don't feel well making such a drastic switch with completely new jobs fwiw | 09:38 |
| AJaeger | dtantsur: I want to move reasonable fast with Xenail ;) | 09:38 |
| dtantsur | AJaeger, the fastest way will be to clone one of the existing jobs. this is the only non-contentious approach IMO | 09:39 |
| AJaeger | dtantsur: I do - but I'm happy to get convinced by clarkb to do it differently... | 09:39 |
| dtantsur | AJaeger, does it mean you offer your help debugging it, if it does not work for an unclear reason? | 09:40 |
| AJaeger | dtantsur: Let's postpone this whole discussion until clarkb is up, please. | 09:40 |
| dtantsur | ok, ok.. | 09:40 |
| AJaeger | dtantsur: infra helps for sure with this switch and can give advice. | 09:41 |
| *** asettle has quit IRC | 09:41 | |
| AJaeger | bbl | 09:41 |
| *** asettle has joined #openstack-infra | 09:41 | |
| *** kzaitsev_mb has joined #openstack-infra | 09:46 | |
| *** Na3iL has joined #openstack-infra | 09:51 | |
| *** sdake has joined #openstack-infra | 09:57 | |
| openstackgerrit | Adam Coldrick proposed openstack-infra/storyboard-webclient: Rework the task list layout https://review.openstack.org/357306 | 09:57 |
| openstackgerrit | Adam Coldrick proposed openstack-infra/storyboard-webclient: Add a way to add tasks to worklists from the story view https://review.openstack.org/366899 | 09:57 |
| openstackgerrit | Adam Coldrick proposed openstack-infra/storyboard-webclient: Complex priorities UI in stories https://review.openstack.org/312666 | 09:58 |
| *** yamahata has quit IRC | 09:59 | |
| *** sdake_ has quit IRC | 10:00 | |
| *** zhurong has quit IRC | 10:03 | |
| openstackgerrit | Vladimir Kuklin proposed openstack-infra/project-config: Add a Fuel extension to support Serializers Conversion https://review.openstack.org/367345 | 10:07 |
| openstackgerrit | Adam Coldrick proposed openstack-infra/storyboard-webclient: Add a way to add tasks to worklists from the story view https://review.openstack.org/366899 | 10:10 |
| openstackgerrit | Adam Coldrick proposed openstack-infra/storyboard-webclient: Complex priorities UI in stories https://review.openstack.org/312666 | 10:10 |
| *** thorst has joined #openstack-infra | 10:11 | |
| zynzel | anybody from infra online? :) | 10:15 |
| zynzel | can anybody create 'glare-support' branch in openstack-infra/puppet-apps_site? | 10:16 |
| zynzel | this was discussed in http://lists.openstack.org/pipermail/openstack-infra/2016-September/004707.html | 10:16 |
| openstackgerrit | Maciej Relewicz proposed openstack-infra/project-config: Added new repos for Murano chef cookbook https://review.openstack.org/366729 | 10:19 |
| *** shardy has quit IRC | 10:21 | |
| openstackgerrit | Adam Coldrick proposed openstack-infra/storyboard-webclient: Add recent events templates for worklist notifications https://review.openstack.org/356441 | 10:22 |
| sanek | fungi: could you please add adobdin@mirantis.com to timmy-core group (https://review.openstack.org/#/c/359831/) | 10:24 |
| *** sshnaidm|afk is now known as sshnaidm|pto | 10:28 | |
| openstackgerrit | Vitaly Gridnev proposed openstack-infra/project-config: fix bad substitution issue https://review.openstack.org/367931 | 10:31 |
| *** sdake_ has joined #openstack-infra | 10:31 | |
| vgridnev | AJaeger, could you please review ^^ | 10:31 |
| *** ramishra has quit IRC | 10:32 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Drbd enables service itself https://review.openstack.org/367932 | 10:32 |
| *** sdake has quit IRC | 10:33 | |
| *** ramishra has joined #openstack-infra | 10:33 | |
| *** Guest90983 is now known as rook | 10:36 | |
| AJaeger | zynzel: best wait for fungi | 10:38 |
| *** rhallisey has joined #openstack-infra | 10:39 | |
| AJaeger | vgridnev: Argh. Yes, templates need two {{. Let's wait for XML output and then I'll fast approve... | 10:39 |
| *** jaosorior_lunch is now known as jaosorior | 10:41 | |
| yolanda | hi sanek , i can do it | 10:42 |
| yolanda | done | 10:43 |
| sanek | yolanda: thanks a lot | 10:44 |
| AJaeger | sanek: I made you the first patch for the repo: https://review.openstack.org/367939 ;) | 10:46 |
| sanek | AJaeger: thanks :) | 10:48 |
| AJaeger | sanek: you'rew welcome. Happy hacking! | 10:49 |
| AJaeger | sanek: and mirroring at github is now setup as well - thanks to yolanda! | 10:49 |
| * yolanda is a github fixer :) | 10:49 | |
| sanek | hehe | 10:49 |
| *** gildub has joined #openstack-infra | 10:50 | |
| AJaeger | sanek: you first need to fix the jobs before my change can pass - they all fail... | 10:50 |
| AJaeger | sanek: feel free to add my change as part of those job fix changes and tell me to abandon mine | 10:50 |
| AJaeger | yolanda: could you +2A https://review.openstack.org/#/c/367931/ , please? | 10:52 |
| yolanda | yes, done | 10:52 |
| openstackgerrit | Vladimir Kuklin proposed openstack-infra/project-config: Add a Fuel extension to support Serializers Conversion https://review.openstack.org/367345 | 10:59 |
| openstackgerrit | Merged openstack-infra/project-config: fix bad substitution issue https://review.openstack.org/367931 | 11:00 |
| *** rtheis has joined #openstack-infra | 11:03 | |
| *** yaume has quit IRC | 11:03 | |
| *** jkilpatr has quit IRC | 11:04 | |
| *** matbu is now known as matbu|lunch | 11:05 | |
| pabelanger | fungi: re: provider booting. Yes, not sure if the right solution but would make nodepool easier on clouds | 11:06 |
| *** jordanP has joined #openstack-infra | 11:07 | |
| *** akshai has joined #openstack-infra | 11:11 | |
| *** senk has joined #openstack-infra | 11:11 | |
| AJaeger | sanek: timmy should pass tests now ;) | 11:13 |
| AJaeger | morning, pabelanger ! | 11:13 |
| sanek | ok, thanks) | 11:13 |
| *** stewie925 has quit IRC | 11:16 | |
| *** caowei has quit IRC | 11:17 | |
| *** sdague has joined #openstack-infra | 11:18 | |
| *** dingyichen has quit IRC | 11:19 | |
| openstackgerrit | yolanda.robla proposed openstack-infra/puppet-infracloud: Add execution perms to post-install in bridge https://review.openstack.org/367950 | 11:20 |
| *** stewie925 has joined #openstack-infra | 11:21 | |
| *** tqtran has joined #openstack-infra | 11:23 | |
| *** baoli has joined #openstack-infra | 11:24 | |
| openstackgerrit | Shu Muto proposed openstack-infra/project-config: Add nodejs4-jobs for magnum-ui https://review.openstack.org/367951 | 11:24 |
| *** tkelsey has quit IRC | 11:26 | |
| *** baoli_ has joined #openstack-infra | 11:27 | |
| *** matthewbodkin has quit IRC | 11:28 | |
| *** tqtran has quit IRC | 11:28 | |
| *** baoli has quit IRC | 11:30 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/nodepool: Disable connecting to the SSH agent for ssh_connect https://review.openstack.org/367952 | 11:31 |
| *** tkelsey has joined #openstack-infra | 11:31 | |
| pabelanger | ^ Is to help narrow down the SSHException: No existing session errors we are still getting in nodepool.o.o | 11:31 |
| pabelanger | I believe the other code paths in paramiko are actually hiding what the real exception is | 11:32 |
| pabelanger | next step will be to enable SSH debugs in nodepool | 11:32 |
| *** baoli_ has quit IRC | 11:34 | |
| *** ldnunes has joined #openstack-infra | 11:36 | |
| *** baoli has joined #openstack-infra | 11:38 | |
| *** jkilpatr has joined #openstack-infra | 11:38 | |
| *** ilyashakhat has quit IRC | 11:38 | |
| openstackgerrit | Brad P. Crochet proposed openstack-infra/tripleo-ci: Add Zaqar to scenario002 https://review.openstack.org/365026 | 11:39 |
| *** berendt has joined #openstack-infra | 11:41 | |
| *** Mary has joined #openstack-infra | 11:42 | |
| *** Mary is now known as Guest4027 | 11:42 | |
| *** _nadya_ has joined #openstack-infra | 11:43 | |
| *** baoli has quit IRC | 11:44 | |
| *** abregman has joined #openstack-infra | 11:47 | |
| *** tonytan4ever has joined #openstack-infra | 11:48 | |
| *** Guest4027 has quit IRC | 11:49 | |
| SergK | Hi, question regarding infra, should we add "jeepyb" user to Administrators group in gerrit, or better just to provide createProject and createGroup capability to it? | 11:51 |
| *** tonytan4ever has quit IRC | 11:53 | |
| openstackgerrit | Jim Rollenhagen proposed openstack-infra/elastic-recheck: Add bug #1621791 https://review.openstack.org/367969 | 11:54 |
| openstack | bug 1621791 in Ironic Inspector "Inspector grenade fails in test_network_basic_ops" [Critical,In progress] https://launchpad.net/bugs/1621791 - Assigned to Dmitry Tantsur (divius) | 11:54 |
| *** lucasagomes is now known as lucas-hungry | 11:55 | |
| pabelanger | SergK: I believe both is needed still: http://docs.openstack.org/infra/system-config/gerrit.html#users | 11:55 |
| *** jed56 has quit IRC | 11:55 | |
| SergK | pabelanger, thank you | 11:58 |
| *** psilvad has joined #openstack-infra | 11:58 | |
| *** baoli has joined #openstack-infra | 11:59 | |
| *** gongysh has quit IRC | 12:00 | |
| *** trown|outtypewww is now known as trown | 12:01 | |
| openstackgerrit | mathieu bultel proposed openstack-infra/tripleo-ci: Implement overcloud upgrade job - Mitaka -> Newton https://review.openstack.org/323750 | 12:04 |
| *** matthewbodkin has joined #openstack-infra | 12:05 | |
| *** kaisers__ has joined #openstack-infra | 12:06 | |
| openstackgerrit | Merged openstack-infra/elastic-recheck: Add bug #1621791 https://review.openstack.org/367969 | 12:07 |
| openstack | bug 1621791 in Ironic Inspector "Inspector grenade fails in test_network_basic_ops" [Critical,In progress] https://launchpad.net/bugs/1621791 - Assigned to Dmitry Tantsur (divius) | 12:07 |
| zigo | pabelanger: Hi there, how's everything? Do you have time to review this? https://review.openstack.org/367343 | 12:07 |
| *** kaisers_ has quit IRC | 12:08 | |
| *** MaryM has joined #openstack-infra | 12:08 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: scenarios: set Debug to True https://review.openstack.org/366896 | 12:08 |
| *** kaisers__ has quit IRC | 12:10 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: scenario001: deploy Ceph https://review.openstack.org/366810 | 12:10 |
| pabelanger | zigo: I haven't been following along the discussions, for this to be honest. So not the best person to be reviewing right now | 12:11 |
| zigo | pabelanger: The idea is that we now have merge commit rights to merge new upstream releases inside the debian/newton packaging branch. If we want a commit merge to be effectively a single reviewable commit on Gerrit, then Gerrit needs to have the upstream branches. | 12:12 |
| zigo | pabelanger: Meaning we need the track-upstream option so that Gerrit non-packaging branches are up-to-date. | 12:13 |
| zigo | Then "git merge -X theirs <TAG-NAME>" will produce a merge commit as expected. | 12:14 |
| *** asettle has quit IRC | 12:14 | |
| zigo | In fact, that's more or less what I wrote in the commit message. | 12:14 |
| *** matbu|lunch is now known as matbu | 12:15 | |
| openstackgerrit | Adam Coldrick proposed openstack-infra/storyboard-webclient: Rework the task list layout https://review.openstack.org/357306 | 12:21 |
| openstackgerrit | Adam Coldrick proposed openstack-infra/storyboard-webclient: Add a way to add tasks to worklists from the story view https://review.openstack.org/366899 | 12:21 |
| *** asettle has joined #openstack-infra | 12:23 | |
| AJaeger | EmilienM: a recheck will not help at all with depends-on | 12:23 |
| openstackgerrit | Adam Coldrick proposed openstack-infra/storyboard-webclient: Complex priorities UI in stories https://review.openstack.org/312666 | 12:24 |
| EmilienM | AJaeger: yeah, I need another +A | 12:24 |
| EmilienM | AJaeger: thanks :-) | 12:24 |
| AJaeger | EmilienM: done. | 12:24 |
| EmilienM | AJaeger: merci | 12:25 |
| *** dkehn has quit IRC | 12:25 | |
| *** dkehn_ has quit IRC | 12:25 | |
| *** dkehn__ has quit IRC | 12:25 | |
| pabelanger | zigo: Yup, understood. I'm just not sure what the long term plan is. I'm sure fungi will be online shortly to review | 12:25 |
| *** esberglu has joined #openstack-infra | 12:25 | |
| openstackgerrit | mathieu bultel proposed openstack-infra/tripleo-ci: Implement overcloud upgrade job - Mitaka -> Newton https://review.openstack.org/323750 | 12:25 |
| *** mriedem has joined #openstack-infra | 12:26 | |
| pabelanger | Time to deal with glean issues an upstart | 12:26 |
| *** pradk has joined #openstack-infra | 12:28 | |
| *** drybjed has left #openstack-infra | 12:28 | |
| *** gordc has joined #openstack-infra | 12:30 | |
| openstackgerrit | Merged openstack-infra/project-config: tripleo/scenario002: add swift https://review.openstack.org/367404 | 12:31 |
| zigo | pabelanger: May I help with that? | 12:31 |
| zigo | pabelanger: I mean, glean and upstream, can I help? | 12:33 |
| *** rfolco has joined #openstack-infra | 12:33 | |
| pabelanger | zigo: thanks, just making sure our upstart order is current for glean. It needs to run before openssh and networking otherwise, early SSH connections get dropped | 12:35 |
| *** dkehn has joined #openstack-infra | 12:37 | |
| *** dkehn__ has joined #openstack-infra | 12:38 | |
| *** dkehn_ has joined #openstack-infra | 12:38 | |
| *** wznoinsk has quit IRC | 12:40 | |
| yolanda | hi, is there some issue with puppet-lint right now? my job failed a pair of times with http://logs.openstack.org/50/367950/1/check/gate-puppet-infracloud-puppet-lint/4ddde21/console.html#_2016-09-09_12_32_56_822839 | 12:41 |
| *** gildub has quit IRC | 12:43 | |
| pabelanger | yolanda: might want to ping EmilienM to confirm | 12:43 |
| *** salv-orlando has quit IRC | 12:46 | |
| *** dizquierdo has quit IRC | 12:46 | |
| *** salv-orlando has joined #openstack-infra | 12:47 | |
| *** wznoinsk has joined #openstack-infra | 12:47 | |
| *** sdake_ is now known as sdake | 12:47 | |
| *** salv-orlando has quit IRC | 12:48 | |
| *** salv-orlando has joined #openstack-infra | 12:48 | |
| EmilienM | mhh weird | 12:49 |
| EmilienM | Puppet CI doesn't fail AFIK | 12:49 |
| EmilienM | yolanda: let me 5 min | 12:49 |
| *** tonytan4ever has joined #openstack-infra | 12:49 | |
| yolanda | EmilienM, thanks | 12:50 |
| *** lucas-hungry is now known as lucasagomes | 12:51 | |
| openstackgerrit | Mathieu Mitchell proposed openstack-infra/system-config: Introduce TinyCoreLinux mirror https://review.openstack.org/325981 | 12:52 |
| *** jaosorior has quit IRC | 12:52 | |
| *** salv-orlando has quit IRC | 12:52 | |
| *** jaosorior has joined #openstack-infra | 12:52 | |
| *** salv-orlando has joined #openstack-infra | 12:53 | |
| *** tonytan4ever has quit IRC | 12:54 | |
| *** rlandy has joined #openstack-infra | 12:54 | |
| *** Goneri has joined #openstack-infra | 12:54 | |
| *** psilvad has quit IRC | 12:54 | |
| EmilienM | yolanda: it sounds like related to puppet-openstack_infra_spec_helper | 12:54 |
| *** psilvad has joined #openstack-infra | 12:57 | |
| yolanda | it started to fail today as far as i know, but i need to find more time to debug | 12:57 |
| *** jcoufal has joined #openstack-infra | 12:59 | |
| *** drifterza has quit IRC | 12:59 | |
| EmilienM | yolanda: so I double checked and we don't have it in PuppetOpenStack | 12:59 |
| EmilienM | so yes, it might be something in the gemspec | 12:59 |
| *** pgadiya has quit IRC | 13:01 | |
| *** thorst has quit IRC | 13:02 | |
| openstackgerrit | Merged openstack-infra/system-config: Update infracloud documentation to remove old geo information https://review.openstack.org/365018 | 13:02 |
| *** andreas_s has quit IRC | 13:02 | |
| *** jamesdenton has joined #openstack-infra | 13:04 | |
| *** woodster_ has joined #openstack-infra | 13:05 | |
| *** sdake has quit IRC | 13:05 | |
| *** psachin has quit IRC | 13:07 | |
| *** psilvad has quit IRC | 13:08 | |
| *** mdrabe has joined #openstack-infra | 13:09 | |
| *** claudiub has quit IRC | 13:10 | |
| *** psilvad has joined #openstack-infra | 13:11 | |
| *** gongysh has joined #openstack-infra | 13:11 | |
| yolanda | so it looks like a conflict , puppet-lint-unquoted_string depending on puppet-lint (~> 1.0), and puppetlabs_spec_helper on puppet-lint (~> 2.0) | 13:13 |
| *** matt-borland has joined #openstack-infra | 13:15 | |
| *** andreas_s has joined #openstack-infra | 13:19 | |
| *** fguillot has joined #openstack-infra | 13:20 | |
| *** mriedem has quit IRC | 13:20 | |
| *** mriedem has joined #openstack-infra | 13:22 | |
| jtomasek | krotscheck: Hi, we're hitting issues with phantomjs-prebuilt on jenkins https://review.openstack.org/#/c/366615/, the relevant github issue https://github.com/Medium/phantomjs/issues/519 indicates broken npm cache, any idea how to fix this, please? | 13:22 |
| *** andreas_s has quit IRC | 13:23 | |
| jtomasek | apetrich: ^ | 13:25 |
| *** xyang1 has joined #openstack-infra | 13:25 | |
| *** kgiusti has joined #openstack-infra | 13:26 | |
| *** mriedem has quit IRC | 13:26 | |
| apetrich | jtomasek, give me a sec | 13:26 |
| *** dsutyagin has quit IRC | 13:27 | |
| *** sdake has joined #openstack-infra | 13:30 | |
| *** amitgandhinz has joined #openstack-infra | 13:31 | |
| *** pradk has quit IRC | 13:32 | |
| *** pradk has joined #openstack-infra | 13:32 | |
| *** fguillot has quit IRC | 13:33 | |
| *** tonytan4ever has joined #openstack-infra | 13:35 | |
| apetrich | jtomasek, so from what I know every run of the ci there is on a clean machine so you should not have a npm cache. last time we had a problem like that was two steps that were installing the same package or something | 13:38 |
| *** berendt has quit IRC | 13:39 | |
| *** berendt has joined #openstack-infra | 13:39 | |
| *** berendt has quit IRC | 13:39 | |
| openstackgerrit | Ihar Hrachyshka proposed openstack-infra/project-config: Remove ACLs to push signed tags from neutron-release https://review.openstack.org/368012 | 13:39 |
| jtomasek | apetrich: yeah, I think that should not be the case this time. Also I am not able to reproduce the issue locally. Updating to latest version of phantomjs-prebuilt did not help either | 13:40 |
| apetrich | jtomasek, the github issue looks reasonable | 13:41 |
| *** ddieterly has joined #openstack-infra | 13:42 | |
| *** fguillot has joined #openstack-infra | 13:43 | |
| *** rhallisey has quit IRC | 13:43 | |
| *** rhallisey has joined #openstack-infra | 13:44 | |
| *** ddieterly has quit IRC | 13:44 | |
| *** ddieterly has joined #openstack-infra | 13:45 | |
| *** jed56 has joined #openstack-infra | 13:45 | |
| *** cardeois has joined #openstack-infra | 13:46 | |
| *** senk has quit IRC | 13:48 | |
| *** sdague has quit IRC | 13:50 | |
| *** dansmith is now known as superdan | 13:50 | |
| *** ociuhandu has joined #openstack-infra | 13:51 | |
| openstackgerrit | Vasyl Saienko proposed openstack-infra/project-config: Add ironic-dsvm-multitest experimental job. https://review.openstack.org/340295 | 13:52 |
| *** e0ne has quit IRC | 13:53 | |
| *** e0ne has joined #openstack-infra | 13:53 | |
| *** alaski is now known as lascii | 13:54 | |
| *** ddieterly is now known as ddieterly[away] | 13:55 | |
| *** kaisers_ has joined #openstack-infra | 13:56 | |
| cloudnull | mornings | 13:56 |
| *** kaisers_ has quit IRC | 13:57 | |
| *** mriedem has joined #openstack-infra | 13:59 | |
| *** rbrndt has joined #openstack-infra | 14:02 | |
| *** bin_ has joined #openstack-infra | 14:02 | |
| *** thorst has joined #openstack-infra | 14:02 | |
| *** mriedem1 has joined #openstack-infra | 14:03 | |
| clarkb | AJaeger: dtantsur not sure what the question is but I have been trying my best to test locally then switch. And this has worled so far for everything but the interface name thing we ran into with neutron | 14:04 |
| *** ddieterly[away] is now known as ddieterly | 14:04 | |
| *** mriedem has quit IRC | 14:04 | |
| clarkb | because really most things have just worked | 14:04 |
| *** inc0 has joined #openstack-infra | 14:04 | |
| dtantsur | clarkb, have you tried ironic on xenial? | 14:05 |
| dtantsur | cause if nobody tried that, I'm more worried to move a completely new job straight to it | 14:06 |
| *** mriedem1 is now known as mriedem | 14:06 | |
| *** thorst has quit IRC | 14:07 | |
| clarkb | dtantsur: no because I asked ironic a while back to get their job config under control.before trying yo even attempt making things run on xenial | 14:07 |
| dtantsur | well, I guess it's time? at least according to AJaeger :) | 14:07 |
| clarkb | well xenial needs to happen sooner than later | 14:07 |
| dtantsur | depending on your definition of "under control" of course | 14:08 |
| *** akshai_ has joined #openstack-infra | 14:08 | |
| clarkb | dtantsur: ironic had >100 integration jobs when I last counted | 14:08 |
| dtantsur | clarkb, fully agree. I still don't get why not start moving *existing* jobs that are known to work already. | 14:08 |
| clarkb | not all of these are used but they are configured. So was asking for ironic to address that and get the number to more accurately reflect ehat is wanted/used | 14:09 |
| dtantsur | clarkb, well, I don't think the situation has changed much re number of jobs.. we do have them better organized now | 14:09 |
| clarkb | and I dont want to make matters worse by yhrowing xenial into the mix without first addressing the other issue | 14:10 |
| *** akshai has quit IRC | 14:11 | |
| anteaya | clarkb: thanks for getting zuul updating again last night and rechecking that neutron patch, it passed jenkins | 14:11 |
| dtantsur | clarkb, well, we're improving, but that does take time, especially when number of features and configurations grows in parallel... | 14:11 |
| *** Jeffrey4l has quit IRC | 14:11 | |
| dtantsur | clarkb, however AJaeger wants us to start switching now | 14:12 |
| clarkb | yes so I recommend we first fix thr list of jobs | 14:12 |
| clarkb | step one is remove all unused jobs | 14:12 |
| *** thorst has joined #openstack-infra | 14:12 | |
| clarkb | then go from there | 14:12 |
| *** Jeffrey4l has joined #openstack-infra | 14:12 | |
| *** kaisers_ has joined #openstack-infra | 14:13 | |
| clarkb | otherwise we will spend effort testing those on xenial that is wasted | 14:13 |
| fungi | zynzel: for consistency, the branch should actually be called "feature/glare-support" and yes i can create it now... do you just want it branched from the tip of master (389e331)? | 14:13 |
| *** notnownikki has joined #openstack-infra | 14:15 | |
| *** edmondsw has joined #openstack-infra | 14:15 | |
| *** spzala has joined #openstack-infra | 14:15 | |
| *** zul has joined #openstack-infra | 14:16 | |
| *** Jeffrey4l has quit IRC | 14:17 | |
| *** Jeffrey4l has joined #openstack-infra | 14:19 | |
| AJaeger | dtantsur: before even adding more jobs - and there're at least two reviews open for that - let's come up with a plan to move forward with xenial. I don't want to make it more complex with adding them... | 14:20 |
| dtantsur | AJaeger, note that my patch is for replacing jobs eventually... | 14:20 |
| *** zhurong has joined #openstack-infra | 14:20 | |
| dtantsur | AJaeger, also such plan should be built on ML IMO and with PTL contributing (hey jroll!) | 14:21 |
| *** vgridnev has quit IRC | 14:22 | |
| AJaeger | dtantsur: my understanding was that clarkb reached out already to ironic team as mentioned above. | 14:22 |
| dtantsur | let's make sure we understand the goals: to have minimum amount of jobs or to have the best coverage? | 14:22 |
| dtantsur | because what I hear now is essentially to remove jobs, even if it makes harder to cover features | 14:22 |
| clarkb | dtantsur: no immediate goal is just remove unused tests from jjb | 14:22 |
| clarkb | since I am fairly certain you are not using all of them | 14:22 |
| AJaeger | dtantsur: our goal is to have Newton tested on Xenial. | 14:23 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack-infra/project-config: Skip scenario tests for OSA liberty/mitaka jobs https://review.openstack.org/368033 | 14:23 |
| AJaeger | And the integrated test runs already on Xenial | 14:23 |
| dtantsur | clarkb, well, we're having two transitions in place: from SSH to ipmitool jobs, and from inspector shell-based jobs to tempest jobs | 14:23 |
| clarkb | then we know what needs testing on xenial and we can do that work | 14:23 |
| *** zz_dimtruck is now known as dimtruck | 14:23 | |
| dtantsur | clarkb, such transition leaves some duplication, later it will leave jobs only running on liberty/mitaka, etc | 14:23 |
| *** tqtran has joined #openstack-infra | 14:25 | |
| * jroll reads things | 14:25 | |
| * jroll in a meeting so will be slow as heck | 14:25 | |
| clarkb | dtantsur: can you at least take a look and see if we can trim things? we were fairly certain there was a lot of trimming that could be done a few weeks ago just in removing unused stuff | 14:25 |
| *** askb has quit IRC | 14:25 | |
| clarkb | jobs that are in jjb but not zuul | 14:25 |
| * dtantsur looks for devananda|dinner's etherpad with our jobs | 14:26 | |
| jroll | clarkb: dtantsur: we had JayF and devananda|dinner looking into that but not sure where it went :/ | 14:26 |
| dtantsur | clarkb, what would help is a tox target to check for that | 14:26 |
| dtantsur | meh, of course FF undigs anything, but not this etherpad... | 14:27 |
| AJaeger | dtantsur: I have a script to generate a list of all unused jobs, let me start that and grep for ironic and add to paste - expect results after my meeting (in 30+ mins) | 14:27 |
| *** kzaitsev_mb has quit IRC | 14:27 | |
| zynzel | fungi: yeah, if possible please create it from master | 14:28 |
| dtantsur | AJaeger, thanks! would be extremely helpful | 14:28 |
| zynzel | moreover, can you create this branch also on openstack/app-catalog, openstack-infra/apps_site and openstack-infra/system-config? | 14:28 |
| zynzel | guys from app-catalog dosent have permission to create new branches in app-catalog repo :) | 14:28 |
| zynzel | and system-config for future work, not to ping you next week | 14:29 |
| *** tqtran has quit IRC | 14:29 | |
| fungi | zynzel: we can't branch system-config, and thinking through this i don' think a branch in puppet-apps_site makes sense either. you just need the changes to it to be backwards-compatible until we move off he old site | 14:30 |
| *** andreas_s has joined #openstack-infra | 14:30 | |
| *** asselin_ has quit IRC | 14:30 | |
| fungi | but a feature branch in app-catalog should be fine | 14:30 |
| zynzel | ok, so i will do some kind of 'if' for backward compatibility in apps_site | 14:31 |
| *** burgerk has joined #openstack-infra | 14:31 | |
| fungi | we can make the puppet module configurable as to which branch of apps-site is used for different servers (conrolled through class parameters) | 14:31 |
| jroll | dtantsur: AJaeger: seems a bit late to be making a plan to get on xenial in newton, no? :/ | 14:31 |
| zynzel | so, please create only new branch in openstack/app-catalog | 14:31 |
| AJaeger | jroll: this is ongoing for some time already, see for example http://lists.openstack.org/pipermail/openstack-dev/2016-July/099818.html | 14:32 |
| jroll | AJaeger: I understand, I'm saying "hi ironic, please make a plan to do this" is a bit late.... I wasn't aware we needed to get involved in this | 14:32 |
| clarkb | jroll: I brought this up with ironic way back when | 14:33 |
| clarkb | it was the entire motivation for that conversation about job cleanup | 14:33 |
| *** burgerk_ has joined #openstack-infra | 14:33 | |
| mat128 | AJaeger, yolanda: the depends-on change has merged for this change: https://review.openstack.org/#/c/366290/ | 14:33 |
| clarkb | basically we eant to port jobs but need a good list first" | 14:33 |
| jroll | clarkb: oh, I understood that as "help clean this up" (and also thought folks were working on that) | 14:33 |
| jroll | fair enough | 14:33 |
| fungi | zynzel: i can do it right after the release team meeting wraps up, no problem | 14:34 |
| jroll | I will bug people and eagerly await Andreas' list since he has a script | 14:34 |
| zynzel | fungi: thanks | 14:34 |
| AJaeger | mat128: great, +A | 14:34 |
| mat128 | AJaeger: thank you :) | 14:35 |
| openstackgerrit | Merged openstack-infra/storyboard-webclient: Rework the task list layout https://review.openstack.org/357306 | 14:35 |
| pabelanger | clarkb: working on an upstart issue now with glean and ubuntu-trusty. Question I have, to we want to force a dependency on ssh server from init scripts or not? | 14:36 |
| *** psilvad has quit IRC | 14:36 | |
| *** burgerk has quit IRC | 14:37 | |
| clarkb | pabelanger: can we? this was aleays the hardest part of using upstart. if its a sysv compat init script they just run at a certain time and otherwise there are some weird dep tree issues. Generally I think we should have ssh come up.after we have working network though | 14:37 |
| *** rajinir has joined #openstack-infra | 14:38 | |
| pabelanger | clarkb: with upstart, we can. And I have it working. | 14:38 |
| pabelanger | clarkb: the issues, default ssh init scripts don't actually depend on networking, just runlevel [12345] | 14:38 |
| clarkb | huh is that causing problems or can we just accept it? | 14:39 |
| pabelanger | so, we can force the dependency with our glean service scripts or just kick openssh reload from glean.sh script | 14:39 |
| AJaeger | dtantsur, jroll, clarkb: FYI, we currently have a total of 12049 jobs, 1612 of these are unused - grep for ironic returns 44 | 14:39 |
| *** jheroux has joined #openstack-infra | 14:39 | |
| dtantsur | wow, that's a lot | 14:39 |
| pabelanger | clarkb: I think there is a race condition that is cause some issues with ubuntu-trusty to launch in rax | 14:39 |
| openstackgerrit | amrith proposed openstack-infra/project-config: add newton gate jobs for trove https://review.openstack.org/368067 | 14:40 |
| AJaeger | dtantsur: not all of those 44 are part of ironic team, it matches ansible ironic stuff as well. | 14:40 |
| AJaeger | jroll, dtantsur, clarkb : full list http://paste.openstack.org/show/570215/ | 14:40 |
| jroll | AJaeger: thank you | 14:41 |
| dtantsur | thanks AJaeger, I'll look into it and propose a patch removing stuff | 14:41 |
| * AJaeger just grepped for ironic in the name of the job, so filter out yours | 14:41 | |
| *** andymaier has joined #openstack-infra | 14:42 | |
| openstackgerrit | Merged openstack-infra/project-config: Publish install guide for Ironic https://review.openstack.org/366290 | 14:42 |
| *** andymaier has quit IRC | 14:42 | |
| fungi | pabelanger: clarkb: i think opensshd has listened on [::]:22/tcp for ages, so just automatically receives connections on any new interfaces that are brought up later | 14:43 |
| fungi | unless you explicitly bind it to particular addresses/interfaces | 14:44 |
| pabelanger | fungi: need to check what the default in trusty is, we don't set ListenAddress right now | 14:45 |
| *** asettle has quit IRC | 14:47 | |
| openstackgerrit | Bartosz Kupidura proposed openstack-infra/puppet-apps_site: [wip] Glare support for app-catalog https://review.openstack.org/359029 | 14:48 |
| pabelanger | SSHException: Error reading SSH protocol banner[Errno 104] Connection reset by peer | 14:48 |
| pabelanger | that's the error we get from ubuntu-trusty | 14:48 |
| *** zhurong has quit IRC | 14:48 | |
| pabelanger | which leads me to thing ssh server is restarted | 14:48 |
| pabelanger | glean, will dpkg-reconfigure openssh-server if ssh keys have already been generated | 14:49 |
| fungi | pabelanger: glean _does_ restart sshd though, right? (something to do with regenerating host keys?) | 14:49 |
| fungi | yeah, that | 14:49 |
| pabelanger | Right | 14:49 |
| clarkb | it should generate them before ssh ever starts or at least thats thr goal | 14:50 |
| *** kaisers_ has quit IRC | 14:50 | |
| *** senk has joined #openstack-infra | 14:50 | |
| AJaeger | clarkb: for new jobs that target master/newton, I currently ask everybody to run them on xenial | 14:50 |
| pabelanger | test -f /etc/ssh/ssh_host_rsa_key || dpkg-reconfigure openssh-server || true | 14:50 |
| pabelanger | thats the command | 14:50 |
| AJaeger | clarkb: so, that should give us less jobs to migrate in the end... | 14:50 |
| *** zul has quit IRC | 14:51 | |
| pabelanger | I suspect glean and openssh-server are racing, nodepool establishes a connection, then glean dpkg-reconfigure openssh-server, which drops the connect | 14:52 |
| pabelanger | we had this issue with systemd and fixed it | 14:52 |
| clarkb | AJaeger: ya thats probably a good idea. | 14:52 |
| *** jcoufal has quit IRC | 14:53 | |
| pabelanger | however, with systemd we could depend on network-pre.target. That doesn't exist for upstart | 14:53 |
| clarkb | AJaeger: so if this is just new jobs we can ptobably move ahead lets just make sure they are used and we dont forget to clean out the old ones | 14:54 |
| zigo | pabelanger: Just for fun, I tried building all of the python-xstatic* packages at once (needed anyway for building or running horizon). That's 29 packages build 3 times... (so, 87 package build). It looks like it's doing to be done in 45 minutes! :) | 14:54 |
| zigo | I tried hard, but couldn't get to use more than 20 nodes at once... :P | 14:55 |
| * zigo wont do that again... | 14:55 | |
| krotscheck | jtomasek: Do you still need help with phantom? | 14:55 |
| jtomasek | krotscheck: yes, please | 14:55 |
| jtomasek | krotscheck: strange thing is that it happens only with that patch which adds 2 new dependencies | 14:56 |
| zigo | pabelanger: My goal was also to check if concurrent POST job would work, and it looks like it is very much ok. | 14:56 |
| pabelanger | clarkb: fungi: this is what I have been testing: https://review.openstack.org/#/c/368072/ | 14:56 |
| jtomasek | krotscheck: It is hard to debug as I am not able to reproduce it locally | 14:56 |
| *** jcoufal has joined #openstack-infra | 14:56 | |
| pabelanger | zigo: good to hear | 14:56 |
| *** Hal1 has quit IRC | 14:57 | |
| pabelanger | zigo: do you know is anybody is actually testing those builds yet? | 14:57 |
| jtomasek | krotscheck: my node version is 4.2.6 and npm is 3.5.2 | 14:57 |
| krotscheck | jtomasek: So, just an FYI, I'm no longer paid to work on OpenStack, so this counts against what volunteer time I permit myself. | 14:57 |
| krotscheck | jtomasek: I don't remember if they've changed this, but in the past the issue with phantomjs is that it's nto actually hosted on npm. | 14:57 |
| jtomasek | krotscheck: ack | 14:58 |
| pabelanger | clarkb: fungi: another option is for us to install a new init.d script for ssh on ubuntu-trusty | 14:58 |
| pabelanger | clarkb: fungi: which adds the dependency on glean | 14:58 |
| krotscheck | jtomasek: In reality, the npm package just loads the most recent version from bitbucket, and the relevant account is rate limited. | 14:58 |
| *** annegentle has joined #openstack-infra | 14:58 | |
| krotscheck | jtomasek: From my converstaions with the owner, their response was: "We have a mirror on the other side of the great chinese firewall, please use that" | 14:58 |
| clarkb | pabelanger: how is sshd starting the first time around without ssh host keys? maybe we can manipulate that to drlay it? | 14:59 |
| krotscheck | jtomasek: Which is not really a thing we can do. | 14:59 |
| AJaeger | clarkb: I tried already removing some old jobs but it's difficult. Biggest offenders are ansible and charm - with 220 unused jobs each. But those need some template reworking... | 14:59 |
| krotscheck | jtomasek: So what was happening at that time was that at the beginning of every day, the quota would reset, a bunch of builds would pass, and then they'd all start to fail. | 14:59 |
| *** matrohon has quit IRC | 14:59 | |
| *** roxanagh_ has joined #openstack-infra | 14:59 | |
| *** mdrabe has quit IRC | 15:00 | |
| clarkb | AJaeger: fun fun thpugh at least thr ansible team has done their own xenialing so we dont have to think about them much | 15:00 |
| AJaeger | ;) | 15:00 |
| mordred | krotscheck: that sounds like a thing we might want to figure out how to pre-cache somehow | 15:00 |
| zigo | pabelanger: Nobody, because I haven't gone up to having services built. And I'm currently stuck, because of blocker project-config patches. IE: 3 patches for adding missing Gerrit repo: https://review.openstack.org/366352 https://review.openstack.org/366734 and https://review.openstack.org/366960 and the track-upstream patch: https://review.openstack.org/367343 | 15:00 |
| krotscheck | mordred: Well, lemme finish | 15:00 |
| AJaeger | clarkb: I'm putting the full list up on a server, waiting for next sync to share the link | 15:00 |
| jtomasek | krotscheck: I see, so you're saying that it might start to succeed on subsequent runs in next days? | 15:00 |
| zigo | fungi: Could you review https://review.openstack.org/367343 BTW ? | 15:00 |
| krotscheck | mordred, jtomasek: It turns out that no actual human uses phantom to browse ui's. | 15:00 |
| dtantsur | AJaeger, oh wonderful, it seems like now we're missing quite a few real jobs which are supposed to run Oo | 15:00 |
| krotscheck | mordred, jtomasek: It's just a developer tool, and it lags singificantly behind the node runtime. | 15:01 |
| krotscheck | mordred, jtomasek: The best option we came up with is "Don't use phantom". No user uses it, and we provide both firefox and chrome running in Xvfb's for karma to grab. | 15:01 |
| *** armax has joined #openstack-infra | 15:02 | |
| *** ddieterly is now known as ddieterly[away] | 15:02 | |
| jtomasek | krotscheck: I see, yeah, we've been considering switching to jsdom solution for unit testing | 15:02 |
| *** asettle has joined #openstack-infra | 15:02 | |
| mordred | krotscheck: ah. cool | 15:02 |
| AJaeger | dtantsur: fun ;( | 15:03 |
| krotscheck | jtomasek: Also, and I hate to bring this up-> ReactJS's licensing has a patent rider with a very aggressive retaliation clause, which may make it non-OSI-compliant. | 15:03 |
| openstackgerrit | Mike Fedosin proposed openstack-infra/project-config: Add py35 and pypi jobs to Glare and its client https://review.openstack.org/366281 | 15:03 |
| jtomasek | krotscheck: yep, I am aware of this | 15:03 |
| dtantsur | AJaeger, e.g. ironic-inspector-client config is just a mess... it misses python-jobs, and defines all jobs (except for forgotten docs) manually.. I dunno how we ended up like that | 15:03 |
| *** sflanigan has joined #openstack-infra | 15:03 | |
| krotscheck | Nobody's respodned to http://lists.openstack.org/pipermail/legal-discuss/2016-September/000418.html yet | 15:03 |
| pabelanger | clarkb: openssh-server generates them if missing I think. Glean will only create the root keys | 15:04 |
| clarkb | pabelanger: glean creates them because sshd doesnt iirc | 15:04 |
| pabelanger | clarkb: Hmm, let me check that | 15:05 |
| *** mdrabe has joined #openstack-infra | 15:05 | |
| clarkb | at least on debuntu. rhel/centos/fedora generate them in the init for sshd if they dont exist | 15:05 |
| AJaeger | dtantsur: if there's anything I can help with, feel free to ask | 15:06 |
| dtantsur | AJaeger, can I get my hands on your script, so that I can check the results? | 15:06 |
| pabelanger | clarkb: Oh, is that because we delete the original keys from the DIB? | 15:07 |
| *** jaosorior has quit IRC | 15:07 | |
| AJaeger | dtantsur: let me walk you through it: http://paste.openstack.org/show/570224/ | 15:07 |
| *** esikachev has quit IRC | 15:07 | |
| *** ddieterly[away] is now known as ddieterly | 15:07 | |
| dtantsur | thnx | 15:07 |
| *** pcaruana has quit IRC | 15:08 | |
| AJaeger | It cleans up the old results, runs tox to get list of all jobs defined, checks zuul output to look for jobs defined - and then creates list of used/unused jobs | 15:08 |
| *** ihrachys has quit IRC | 15:08 | |
| dtantsur | AJaeger, interesting, I don't see some of the jobs mentioned there defined e.g. gate-tempest-dsvm-ironic-inspector (without -nv) | 15:08 |
| AJaeger | dtantsur: So, that's the high level view. Just run it locally and double check my regex foo ;) | 15:08 |
| *** hashar is now known as hasharAway | 15:09 | |
| dtantsur | sure, maybe it's my fault, lemme check again | 15:09 |
| *** matthewbodkin has quit IRC | 15:09 | |
| *** kzaitsev_mb has joined #openstack-infra | 15:09 | |
| clarkb | pabelanger: yes | 15:09 |
| AJaeger | dtantsur: git grep '{pipeline}-tempest-dsvm-ironic-inspector{job-suffix}' in project-config | 15:10 |
| *** abregman has quit IRC | 15:10 | |
| dtantsur | AJaeger, what's the group "devstack-jobs" for? | 15:10 |
| dtantsur | in devstack-gate.yaml | 15:10 |
| AJaeger | dtantsur: a nice way to instantiate those - see how it's used in jenkins/jobs/projects.yaml. | 15:11 |
| mordred | krotscheck: thanks for reminding me about react | 15:11 |
| AJaeger | dtantsur: we can easily remove it from devstack-jobs... | 15:11 |
| AJaeger | dtantsur: that all comes from the big integrated gate AFAIK | 15:11 |
| dtantsur | AJaeger, yeah, it might be the cause of unused jobs | 15:11 |
| AJaeger | dtantsur: So, if you find an unused job, check the config - ./.test/jenkins-job-builder/.test/new/out/gate-tempest-dsvm-ironic-inspector in this case, read it to get the template used, and then grep for the template in jenkins/jobs to find where it's used | 15:12 |
| pabelanger | mrhillsman: clarkb: fungi: raddaoui: So, I am seeing instability from osic-cloud8 APIs, which is causing some issues with nodepool. http://grafana.openstack.org/dashboard/db/nodepool-osic shows a pictures of the issues. I'm think we should turn off osic-cloud8 for the moment until we can do some more manually testing. | 15:12 |
| *** asettle has quit IRC | 15:13 | |
| krotscheck | mordred: YOu're... welcome? I'm guessing you can throw lawyers at the problem? | 15:13 |
| *** amitgandhinz has quit IRC | 15:13 | |
| AJaeger | dtantsur: so, let me cleanup that list for you... | 15:13 |
| *** amitgandhinz has joined #openstack-infra | 15:13 | |
| *** gongysh has quit IRC | 15:13 | |
| zigo | mordred: Could you please approve https://review.openstack.org/366352 so I could build all python-*client packages? | 15:13 |
| openstackgerrit | yolanda.robla proposed openstack-infra/puppet-infracloud: Only spin up bridge on RHEL case https://review.openstack.org/368075 | 15:14 |
| *** Benj_ has joined #openstack-infra | 15:14 | |
| *** thcipriani|afk is now known as thcipriani | 15:14 | |
| mordred | krotscheck: well, I know at least one ... | 15:14 |
| krotscheck | mordred: Is that lawyer compact enough to throw? | 15:15 |
| *** dizquierdo has joined #openstack-infra | 15:15 | |
| mordred | krotscheck: oh yeah | 15:15 |
| pabelanger | clarkb: going to put trusty failures on hold for now, since it is the lowest failure rate atm | 15:15 |
| *** gouthamr has joined #openstack-infra | 15:15 | |
| dtantsur | AJaeger, will https://github.com/openstack-infra/project-config/blob/master/jenkins/jobs/projects.yaml#L5206-L5237 generate all possible combinations? | 15:15 |
| dtantsur | if so, we might need moar excludes :) | 15:16 |
| *** gouthamr_ has joined #openstack-infra | 15:16 | |
| *** vinaypotluri has joined #openstack-infra | 15:17 | |
| *** _nadya_ has quit IRC | 15:17 | |
| pabelanger | corvus: clarkb: would appreciate your feedback on https://review.openstack.org/#/c/367381 and https://review.openstack.org/#/c/367952 however. Changes to paramiko ssh_connect settings for nodepool and private keys | 15:17 |
| zigo | mordred: Thanks! | 15:17 |
| zigo | AJaeger: Can you add your +2A to the one of Monty? https://review.openstack.org/366352 | 15:18 |
| fungi | mordred: krotscheck: it does seem at least superficially weird that if what they wanted was a bsd-like license with explicit patent protections, they would have been more likely to just use the apache license... makes it seem like there's something else going on there | 15:19 |
| mrhillsman | pabelanger if cloud8 is more of an issue than a help right now please go ahead and disable | 15:20 |
| *** gouthamr has quit IRC | 15:20 | |
| mrhillsman | raddaoui can correct me if i am wrong but we have identified the issue at the least | 15:20 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Remove unused jobs from devstack-gate https://review.openstack.org/368078 | 15:21 |
| AJaeger | clarkb, dtantsur ^ | 15:21 |
| AJaeger | dtantsur: yes, that generates all possible combinations | 15:22 |
| dtantsur | AJaeger, thanks! I'll base my further work on top of it | 15:22 |
| *** jcoufal has quit IRC | 15:22 | |
| openstackgerrit | Dmitry Tantsur proposed openstack-infra/project-config: Use standard python jobs for python-ironic-inspector-client https://review.openstack.org/368080 | 15:23 |
| dtantsur | AJaeger, 1st part ^^ (haven't run tests yet) | 15:23 |
| AJaeger | zigo, done | 15:24 |
| dtantsur | AJaeger, what is preferred, to generate all combinations, then add exceptions, or explicitly list all expected variants? | 15:24 |
| AJaeger | dtantsur: I don't see a docs directory in python-ironic-inspector-client, so hope you didn't add docs jobs? | 15:24 |
| dtantsur | AJaeger, ouch, it seems like I derped this one | 15:25 |
| dtantsur | yeah, there are no docs there, probably that's why it was so messy | 15:25 |
| AJaeger | dtantsur: Often generating all combinations is less error prone. clarkb, what do you think in this case? | 15:25 |
| AJaeger | clarkb: see also dtantsur's gist from 10 mins ago | 15:26 |
| pabelanger | mrhillsman: right now, we're only launching a few nodes on osic-cloud8, up to 10. I think we could keep it running for a bit, if it helps trace down issues on your side. However, if not, ya, we'd likely disable it for now as not to impact test jobs | 15:27 |
| *** amotoki has joined #openstack-infra | 15:27 | |
| pabelanger | mrhillsman: it looks like once a node is launched, test will be run, but we are seeing some issues working with the openstack api | 15:27 |
| dtantsur | AJaeger, I'll probably continue with a fresh head on Monday, after your patch lands and the list shrinks a bit | 15:28 |
| AJaeger | project-config cores, here're three cleanup changes, reviews are welcome: https://review.openstack.org/367844 and https://review.openstack.org/367844 https://review.openstack.org/365361 | 15:29 |
| AJaeger | dtantsur: ok | 15:29 |
| AJaeger | dtantsur: does the script work for you? | 15:29 |
| *** senk has quit IRC | 15:29 | |
| dtantsur | haven't tried yet, lemme do | 15:29 |
| *** gongysh has joined #openstack-infra | 15:30 | |
| fungi | mordred: i'm refreshing my memory of how our upstream tracking implementation works... it basically updates copies of what's in the upstream remote each time manage-projects is run? | 15:30 |
| *** Benj_ has quit IRC | 15:30 | |
| anteaya | AJaeger: you have 844 linked twice | 15:30 |
| fungi | mordred: if so, i wonder if that choice was made at a time when we fired manage-projects on every puppet pulse rather than only when there are changes to projects.yaml | 15:31 |
| openstackgerrit | Merged openstack-infra/project-config: Add missing deb repo 3/5 https://review.openstack.org/366352 | 15:31 |
| AJaeger | anteaya: ;( Thanks for noticing. Try https://review.openstack.org/367932 instead | 15:31 |
| fungi | mordred: and whether we should move that functionality to a separate cron instead of having it rely on frequency of projects.yaml changes | 15:32 |
| *** mhickey__ has quit IRC | 15:32 | |
| zigo | AJaeger: Awesome, thanks. | 15:32 |
| clarkb | AJaeger: dtantsur I am looking at that github link and its just python jobs for instack things? | 15:32 |
| clarkb | AJaeger: dtantsur I feel like I am missing context on the generating all combinations question | 15:33 |
| dtantsur | clarkb, gah, something merged in between, and the link is no longer right | 15:34 |
| clarkb | dtantsur: oh ha | 15:34 |
| dtantsur | clarkb, take it while it's still valid: https://github.com/openstack-infra/project-config/blob/master/jenkins/jobs/projects.yaml#L5248-L5268 :) | 15:34 |
| *** ZZelle has quit IRC | 15:34 | |
| anteaya | AJaeger: keystoneauth-saml2 still has dot files: http://git.openstack.org/cgit/openstack/keystoneauth-saml2/tree/ | 15:34 |
| *** mdrabe has quit IRC | 15:34 | |
| anteaya | stevemar: we need the dot files removed from keystoneauth-saml2 ^^ | 15:35 |
| AJaeger | anteaya: we removed all jobs already from it ;( | 15:35 |
| *** mdrabe has joined #openstack-infra | 15:35 | |
| anteaya | AJaeger: well the dot files remain | 15:35 |
| stevemar | aww man, i failed in deleting :facepalm: | 15:35 |
| stevemar | https://github.com/openstack/keystoneauth-saml2 | 15:35 |
| *** jistr is now known as jistr|biab | 15:35 | |
| AJaeger | anteaya: yes, so that would need new jobs added first, then removed again. | 15:35 |
| fungi | stevemar: `git ls-files` is your friend | 15:36 |
| stevemar | AJaeger: anteaya remove all the files, including the README? | 15:36 |
| anteaya | stevemar: leave the readme | 15:36 |
| anteaya | stevemar: all other files are removed, including the dot files | 15:36 |
| AJaeger | stevemar: leave the README in - and only the README. But if you add another commit, you need to update the README | 15:36 |
| *** ZZelle has joined #openstack-infra | 15:36 | |
| stevemar | AJaeger: why do i need to update it? | 15:37 |
| clarkb | AJaeger: dtantsur I am a fan of having more matrices but applyting them as necessary to more specific templates. looks like we already do that below in the jobs list too | 15:37 |
| anteaya | AJaeger: why change the readme? | 15:37 |
| AJaeger | stevemar: I noticed that we didn't retire it properly, ACLs still exist, see https://review.openstack.org/367932 | 15:37 |
| *** sflanigan has quit IRC | 15:37 | |
| fungi | could do two commits... a revert of the incomplete deletion followed by a rework that deletes the right files | 15:37 |
| clarkb | AJaeger: dtantsur basically instead of having a giant global matrix have the matrix for job foo and matrix for job bar and or group foo and group bar then you can have a bit more control over how things multiply | 15:37 |
| AJaeger | anteaya: it says checkout "HEAD^1" - this would be wrong if you add another change | 15:37 |
| stevemar | ah | 15:37 |
| *** jcoufal has joined #openstack-infra | 15:37 | |
| anteaya | AJaeger: oh okay | 15:37 |
| stevemar | AJaeger: you sent me the wrong link? | 15:37 |
| openstackgerrit | Thomas Goirand proposed openstack-infra/project-config: upstream: and options: track-upstream for deb-* https://review.openstack.org/367343 | 15:38 |
| AJaeger | stevemar: https://review.openstack.org/367844 - I'm screwing up links to day ;( | 15:38 |
| zigo | fungi: Merging the addition of "missing deb 3/5" created a merge conflict, please review again https://review.openstack.org/367343 | 15:38 |
| dtantsur | clarkb, I wonder how it would look in our case though.. | 15:39 |
| stevemar | AJaeger: ahhh | 15:39 |
| clarkb | AJaeger: 368078 didn't pass the layout job | 15:39 |
| clarkb | dtantsur: I think you would maybe have a group for ipmi, a group for ipa, a group for ssh (or whatever just examples not necessarily sure these are the specific values) | 15:39 |
| stevemar | AJaeger: so.. what do y'all need me to do? remove the dotfiles? or do i wait until the jobs are back? | 15:39 |
| AJaeger | clarkb, will investigate... | 15:40 |
| anteaya | stevemar: well fungi gave a suggestion above | 15:40 |
| clarkb | dtantsur: then your driver remains fixed while you control other things like whther or not you need voting or non voting, the ramdisk, partitioning, etc for each of those groups | 15:40 |
| anteaya | stevemar: would be simplest I think | 15:40 |
| krotscheck | fungi, mordred: IANAL and have no clue about such things. | 15:40 |
| dtantsur | clarkb, yeah, makes sense | 15:40 |
| AJaeger | stevemar: add jobs back, follow fungi's suggestion, remove the jobs and merge my change ;) | 15:40 |
| clarkb | dtantsur: it might make more sense to group my ramdisk or by partition but I think thats the general idea I would start with. Find the flag that ends up grouping things logically given what you want to test then group on that | 15:41 |
| zigo | mordred: Can you also review https://review.openstack.org/367343 so that I can do upgrades to new upstream releases? | 15:41 |
| openstackgerrit | Matt Riedemann proposed openstack-infra/project-config: Create gate-grenade-dsvm-neutron-nova-next-ubuntu-xenial-nv https://review.openstack.org/368087 | 15:41 |
| zigo | fungi: How does the track-upstream option work? Is this a cron job? How often does it run? | 15:41 |
| AJaeger | clarkb, layout failure is "ImportError: cannot import name monkey". What's up? | 15:42 |
| *** esikachev has joined #openstack-infra | 15:42 | |
| AJaeger | "ould not import setuptools which is required to install from a source distribution." | 15:42 |
| zigo | fungi: mordred: Outch, my bad, fixing the layout... | 15:42 |
| AJaeger | http://logs.openstack.org/78/368078/1/check/gate-project-config-layout/fcd0b64/console.html#_2016-09-09_15_32_48_810665 | 15:42 |
| mtreinish | AJaeger: hmm, I just hit that locally trying to build the governance repo docs | 15:42 |
| AJaeger | mtreinish: any new broken release out? | 15:43 |
| clarkb | AJaeger: huh that is new to me | 15:43 |
| mtreinish | yep setuptools released today | 15:43 |
| mtreinish | https://pypi.python.org/pypi/setuptools | 15:43 |
| *** vhosakot has joined #openstack-infra | 15:43 | |
| mtreinish | dstufft: ^^^ | 15:43 |
| clarkb | that is a circular import | 15:43 |
| clarkb | goes monkey -> msvc -> monkey | 15:43 |
| anteaya | mtreinish: can you +1 https://review.openstack.org/#/c/368087 which adds an experimental job to grenade? | 15:43 |
| openstackgerrit | Steve Martinelli proposed openstack-infra/project-config: Revert "retire keystoneauth-saml2" https://review.openstack.org/368089 | 15:43 |
| zigo | fungi: Isn't this a bug in the check script? http://logs.openstack.org/43/367343/5/check/gate-project-config-layout/43eaa5a/console.html#_2016-09-09_15_41_54_662747 | 15:44 |
| anteaya | stevemar: the patch is in merge conflict, also can you say in the commit message that this is so you can remove the forgotten dot files? | 15:45 |
| mtreinish | anteaya: done | 15:45 |
| AJaeger | zigo, read backscrool for 10 lines ;) | 15:45 |
| anteaya | mtreinish: thank you | 15:45 |
| AJaeger | Team, seems layout gate is broken now ;( | 15:45 |
| stevemar | anteaya: yeah, i'm rebasing locally, i'm in over my head, give me a few | 15:45 |
| dstufft | Talking to jason | 15:45 |
| mtreinish | dstufft: thanks | 15:45 |
| dtantsur | seems like many gates are, I just saw this import problem in ironic-inspector grenade | 15:45 |
| anteaya | dtantsur: jobs | 15:46 |
| raissa | just got that import problem in sahara-tests | 15:46 |
| AJaeger | infra-root, what do you think of a #status notice? | 15:46 |
| mriedem | is this a known failure in project-config? http://logs.openstack.org/87/368087/1/check/gate-project-config-layout/60d11e8/console.html#_2016-09-09_15_43_58_947548 | 15:46 |
| *** esikachev has quit IRC | 15:46 | |
| anteaya | mriedem: the monkey thing? | 15:47 |
| clarkb | AJaeger: probably a good idea. "New setuptools release appears to have a circular import which is breaking many jobs" ? | 15:47 |
| *** IlyaG has joined #openstack-infra | 15:47 | |
| *** Jeffrey4l has quit IRC | 15:47 | |
| anteaya | mriedem: yeah we just found it all over the place, new setuptools release | 15:47 |
| AJaeger | clarkb: Let's mention monkey in it. | 15:47 |
| anteaya | AJaeger: agreed | 15:47 |
| clarkb | "New setuptools release includes a circular import of "monkey" which is breaking many jobs." | 15:47 |
| AJaeger | Suggestion: "New setuptools release appears to have a circular import which is breaking many jobs - check for ImportError: cannot import name monkey." | 15:48 |
| anteaya | either mentions monkey so works for me | 15:48 |
| clarkb | that is nice and specific | 15:48 |
| anteaya | either/both | 15:48 |
| clarkb | ok now to remember which of the commands I want | 15:48 |
| AJaeger | clarkb: #status notice "text" | 15:48 |
| clarkb | do we want to update the topics in channels? I think that is "alert" | 15:48 |
| clarkb | and notice won't update the topics | 15:48 |
| anteaya | I think update topics | 15:48 |
| anteaya | this is hitting everyone and we can unalert once it clears | 15:49 |
| *** openstackgerrit has quit IRC | 15:49 | |
| AJaeger | Lots of red in zuul - alert | 15:49 |
| *** tesseract- has quit IRC | 15:49 | |
| anteaya | I too vote alert | 15:49 |
| *** openstackgerrit has joined #openstack-infra | 15:49 | |
| fungi | is there an upstream issue open for it? | 15:49 |
| anteaya | the whole gate pipeline is affected | 15:49 |
| fungi | can link https://github.com/pypa/setuptools/issues/780 looks like | 15:50 |
| anteaya | fungi: dunno. so far dstufft is aware and talking to somone named Jason | 15:50 |
| openstackgerrit | amrith proposed openstack-infra/project-config: add newton gate jobs for trove https://review.openstack.org/368067 | 15:50 |
| fungi | i expect there will be a brown-bag release forthwith | 15:50 |
| clarkb | #status alert New setuptools release appears to have a circular import which is breaking many jobs - check for ImportError: cannot import name monkey. | 15:50 |
| openstackstatus | clarkb: sending alert | 15:50 |
| dstufft | Jason is @jaraco, the setuptools maintainer | 15:50 |
| mtreinish | fungi: https://github.com/pypa/setuptools/issues/780 | 15:50 |
| anteaya | dstufft: ah thank you | 15:50 |
| AJaeger | thanks, clarkb | 15:50 |
| mtreinish | fungi: you win :) | 15:50 |
| fungi | mtreinish: the setuptools issue tracker is in my auto-complete ;) | 15:51 |
| *** andreas_s has quit IRC | 15:51 | |
| mtreinish | hah | 15:51 |
| mriedem | i'll get an e-r up for https://bugs.launchpad.net/openstack-gate/+bug/1621916 | 15:51 |
| openstack | Launchpad bug 1621916 in OpenStack-Gate "setuptools 27.1.0 has a circular import error" [Undecided,New] | 15:51 |
| fungi | (i'm not sure if that says something about me, or about setuptools, but whatever) | 15:51 |
| anteaya | mriedem: thanks | 15:51 |
| dstufft | fungi: why not both! | 15:51 |
| dstufft | https://github.com/pypa/setuptools/pull/781 should fix it I think | 15:51 |
| zigo | So, to sum-up: the monkey broke setuptools... :P | 15:52 |
| *** sambetts is now known as sambetts|afk | 15:52 | |
| fungi | <mr_burns>exxxxxxellent</mr_burns> | 15:52 |
| *** zul has joined #openstack-infra | 15:52 | |
| anteaya | zigo: and dstufft has a fix up | 15:52 |
| fungi | chaos monkey at work! | 15:52 |
| -openstackstatus- NOTICE: New setuptools release appears to have a circular import which is breaking many jobs - check for ImportError: cannot import name monkey. | 15:52 | |
| *** ChanServ changes topic to "New setuptools release appears to have a circular import which is breaking many jobs - check for ImportError: cannot import name monkey." | 15:52 | |
| *** slashme has joined #openstack-infra | 15:52 | |
| openstackgerrit | Matt Riedemann proposed openstack-infra/elastic-recheck: Add query for setuptools 27.1.0 bug 1621916 https://review.openstack.org/368094 | 15:53 |
| mriedem | mtreinish: ^ | 15:53 |
| openstack | bug 1621916 in OpenStack-Gate "setuptools 27.1.0 has a circular import error" [Undecided,New] https://launchpad.net/bugs/1621916 | 15:53 |
| zigo | anteaya: dstufft killed the money... :) | 15:53 |
| anteaya | or moved it | 15:53 |
| anteaya | sometimes monkeys just need a better place to be | 15:54 |
| * AJaeger will be back later | 15:54 | |
| mtreinish | mriedem: +A | 15:54 |
| anteaya | AJaeger: thank you | 15:54 |
| mtreinish | mriedem: hmm, I wonder if we should look into adding other tracker support into e-r | 15:54 |
| openstackgerrit | Steve Martinelli proposed openstack-infra/project-config: Revert "retire keystoneauth-saml2" https://review.openstack.org/368089 | 15:55 |
| stevemar | AJaeger: ummm ^ ? | 15:55 |
| fungi | mtreinish: you can just open a reference lp bug against openstack-gate for now. that's how we've normally handled stuff like this | 15:55 |
| stevemar | AJaeger: i didn't want to add back *all* the jobs, but i may have made a booboo | 15:55 |
| *** psilvad has joined #openstack-infra | 15:55 | |
| *** e0ne has quit IRC | 15:55 | |
| mriedem | mtreinish: asselin added jira support | 15:55 |
| mriedem | or support to make it configurable | 15:56 |
| mriedem | for other bug trackers | 15:56 |
| openstackstatus | clarkb: finished sending alert | 15:56 |
| *** vhosakot has quit IRC | 15:56 | |
| mtreinish | fungi: it is, I realize that. But I was jsut thinking that it wouldn't be too difficult to for tracking based on github directly | 15:56 |
| *** jcoufal has quit IRC | 15:56 | |
| anteaya | stevemar: AJaeger is out of a bit whilst we address the monkey issue | 15:56 |
| stevemar | anteaya: coolio | 15:56 |
| anteaya | stevemar: thanks :) | 15:56 |
| fungi | mtreinish: agreed, if it's something you want to add, it certainly makes sense in these situations | 15:56 |
| *** jcoufal has joined #openstack-infra | 15:57 | |
| mtreinish | mriedem: oh right | 15:57 |
| stevemar | anteaya: i'll get all my reviews queued up, so it's ready for you all when you've caught the monkey | 15:57 |
| *** dimtruck is now known as zz_dimtruck | 15:57 | |
| mtreinish | mriedem: so I guess it's just a matter of leveraging that | 15:57 |
| anteaya | stevemar: thank you | 15:57 |
| *** dtantsur is now known as dtantsur|afk | 15:59 | |
| *** Jeffrey4l has joined #openstack-infra | 15:59 | |
| *** dragonmaster has quit IRC | 15:59 | |
| jlvillal | I hope this monkey thing when fixed will have a funny #success entry :) | 15:59 |
| *** mrtenio has joined #openstack-infra | 16:00 | |
| *** amotoki_ has joined #openstack-infra | 16:02 | |
| *** zz_dimtruck is now known as dimtruck | 16:02 | |
| *** jordanP has quit IRC | 16:03 | |
| *** gongysh has quit IRC | 16:03 | |
| *** amotoki_ has quit IRC | 16:03 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/glean: Append to interface definition for multi-address https://review.openstack.org/366862 | 16:03 |
| openstackgerrit | Paul Belanger proposed openstack-infra/glean: Add ipv6 support to gentoo https://review.openstack.org/367487 | 16:03 |
| openstackgerrit | Paul Belanger proposed openstack-infra/glean: Index networks using id, not links https://review.openstack.org/366937 | 16:03 |
| *** ddieterly is now known as ddieterly[away] | 16:04 | |
| pabelanger | mordred: clarkb: ^ our glean stack to fix rax-iad | 16:04 |
| *** tphummel has joined #openstack-infra | 16:04 | |
| openstackgerrit | Steve Martinelli proposed openstack-infra/project-config: bring back keystoneauth-saml2 jobs temporarily https://review.openstack.org/368089 | 16:05 |
| *** amotoki has quit IRC | 16:05 | |
| openstackgerrit | Merged openstack-infra/elastic-recheck: Add query for setuptools 27.1.0 bug 1621916 https://review.openstack.org/368094 | 16:05 |
| openstack | bug 1621916 in OpenStack-Gate "setuptools 27.1.0 has a circular import error" [Undecided,New] https://launchpad.net/bugs/1621916 | 16:05 |
| *** _nadya_ has joined #openstack-infra | 16:05 | |
| openstackgerrit | Tom Fifield proposed openstack-infra/puppet-mediawiki: Skip CAPTCHA for autopatrol users https://review.openstack.org/368114 | 16:06 |
| *** yamahata has joined #openstack-infra | 16:06 | |
| dstufft | 27.1.2 should fix I think | 16:07 |
| *** ddieterly[away] is now known as ddieterly | 16:08 | |
| anteaya | dstufft: thank you | 16:09 |
| anteaya | clarkb: ^^ | 16:09 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/elastic-recheck: Make Elastic Recheck Watch more reusable https://review.openstack.org/251515 | 16:10 |
| *** thorst has quit IRC | 16:11 | |
| clarkb | once we see it on our mirror we can status ok | 16:11 |
| anteaya | clarkb: awesome, thank you | 16:11 |
| anteaya | any thoughts on how long it should take for the mirror to see it? | 16:12 |
| *** akshai_ has quit IRC | 16:12 | |
| *** thorst has joined #openstack-infra | 16:12 | |
| openstackgerrit | Steve Martinelli proposed openstack-infra/project-config: bring back keystoneauth-saml2 jobs temporarily https://review.openstack.org/368089 | 16:12 |
| clarkb | we build the mirror every 5 minutes so ~5minutes or so after the release happens | 16:12 |
| anteaya | great | 16:12 |
| clarkb | https://review.openstack.org/#/c/367699/ has the votes it needs to address the linuxbridge xenial issues. So once setuptools is working again we should merge that then the project-config change | 16:14 |
| anteaya | great | 16:14 |
| openstackgerrit | Steve Martinelli proposed openstack-infra/project-config: Revert "bring back keystoneauth-saml2 jobs temporarily" https://review.openstack.org/368135 | 16:15 |
| *** thorst_ has joined #openstack-infra | 16:15 | |
| openstackgerrit | Steve Martinelli proposed openstack-infra/project-config: Complete retirement of keystoneauth-saml2 https://review.openstack.org/367844 | 16:16 |
| *** thorst_ has quit IRC | 16:16 | |
| *** thorst has quit IRC | 16:16 | |
| stevemar | AJaeger: i compiled a hilarious chain of patches, you can see them here: https://review.openstack.org/#/q/topic:retire-keystoneauth-saml2-again -- all the reverts -- | 16:17 |
| stevemar | AJaeger: please review when you've fixed your infra monkies | 16:17 |
| *** vhosakot has joined #openstack-infra | 16:18 | |
| *** thorst has joined #openstack-infra | 16:19 | |
| *** vhosakot has quit IRC | 16:19 | |
| openstackgerrit | Tom Fifield proposed openstack-infra/puppet-mediawiki: Skip CAPTCHA for autopatrol users https://review.openstack.org/368114 | 16:19 |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/puppet-mediawiki: Skip captcha enforcement for autopatrolled users https://review.openstack.org/368141 | 16:20 |
| anteaya | fungi: I think fifield wins | 16:20 |
| anteaya | 114 vs 141 | 16:21 |
| *** mkoderer has quit IRC | 16:21 | |
| *** lmiccini has quit IRC | 16:21 | |
| *** florianf has quit IRC | 16:21 | |
| *** spzala has quit IRC | 16:21 | |
| fungi | oh! his e-mail didn't mention having submitted a change for that | 16:22 |
| anteaya | this third one did | 16:22 |
| fungi | anyway, i had to apply it manually in production and test it | 16:22 |
| anteaya | ah and how does it test? | 16:22 |
| *** spzala has joined #openstack-infra | 16:22 | |
| *** jcoufal has quit IRC | 16:22 | |
| fungi | works great | 16:22 |
| *** cgross has quit IRC | 16:22 | |
| anteaya | awesome | 16:22 |
| *** senk has joined #openstack-infra | 16:22 | |
| anteaya | thanks for testing that | 16:22 |
| *** xarses has quit IRC | 16:23 | |
| *** jlanoux has quit IRC | 16:23 | |
| *** jcoufal has joined #openstack-infra | 16:24 | |
| openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci: Use current-tripleo for undercloud upgrades https://review.openstack.org/368142 | 16:24 |
| *** thorst_ has joined #openstack-infra | 16:24 | |
| *** thorst has quit IRC | 16:24 | |
| *** akshai has joined #openstack-infra | 16:25 | |
| *** jistr|biab is now known as jistr | 16:25 | |
| *** sdake has quit IRC | 16:28 | |
| *** thorst_ has quit IRC | 16:29 | |
| inc0 | pabelanger, sooo...we resolved issue with osic cloud8 | 16:30 |
| *** thorst has joined #openstack-infra | 16:31 | |
| *** vhosakot has joined #openstack-infra | 16:31 | |
| *** thorst has quit IRC | 16:32 | |
| pabelanger | inc0: oh, nice. let me check nodepool quickly | 16:33 |
| *** dimtruck is now known as zz_dimtruck | 16:33 | |
| *** zz_dimtruck is now known as dimtruck | 16:34 | |
| pabelanger | inc0: okay, I see 3 stuck deleting nodes from before: http://paste.openstack.org/show/570270/ | 16:34 |
| clarkb | pabelanger: in other nodepool news we didn't finish the ovh bhs1 uploads last night :/ we will want to keep an eye on that and maybe tralk to them about how we can make that work better | 16:34 |
| inc0 | yeah mrhillsman solved this one too | 16:35 |
| pabelanger | clarkb: Ya, that might be a good idea. | 16:35 |
| anteaya | clarkb: I like the word you coined, tralk | 16:35 |
| *** raildo has joined #openstack-infra | 16:35 | |
| anteaya | it is a good word | 16:35 |
| mrhillsman | pabelanger a few more need to fix up | 16:36 |
| mrhillsman | give me moments | 16:36 |
| *** tkelsey has quit IRC | 16:36 | |
| clarkb | there is a setuptools 27.1.2 in our mirror now | 16:37 |
| anteaya | yay | 16:37 |
| * clarkb looks at jobs to see if they are happy again | 16:37 | |
| pabelanger | mrhillsman: inc0: ack | 16:37 |
| *** samuelBartel has quit IRC | 16:37 | |
| inc0 | pabelanger, hold on, it seems it wasnt fixed after all - btw issue was pretty ... interesting | 16:37 |
| inc0 | we simply got hacked | 16:37 |
| pabelanger | eep | 16:38 |
| mordred | inc0: you got hacked and the hackers solved problems for you? that seems friendly | 16:38 |
| inc0 | no, I bet we're digging someones bitcoins | 16:39 |
| inc0 | or host a cs server | 16:39 |
| inc0 | I hope it's nothing porn related, but whatever, it seems we'll need to repave env :( | 16:40 |
| dmsimard | Is that setuptools thing fixed? Looks like the bug mentioned a new release being cut with the fix | 16:40 |
| anteaya | dmsimard: clarkb is just checking our jobs now | 16:40 |
| clarkb | I am watiching the tests for https://review.openstack.org/367468 | 16:41 |
| clarkb | since it just recently started, looks happy so far | 16:41 |
| anteaya | awesome | 16:41 |
| dmsimard | ack, ty | 16:41 |
| *** kzaitsev_mb has quit IRC | 16:41 | |
| openstackgerrit | sebastian marcet proposed openstack-infra/openstackid-resources: Smart Conference https://review.openstack.org/368148 | 16:43 |
| fungi | inc0: any idea how they gained a foothold? | 16:43 |
| fungi | it would be a great lesson learned for infra-cloud too if you have a "don't do <this>" anecdote | 16:44 |
| *** lucasagomes is now known as lucas-afk | 16:44 | |
| *** mkoderer has joined #openstack-infra | 16:44 | |
| *** cgross has joined #openstack-infra | 16:44 | |
| *** notnownikki has quit IRC | 16:45 | |
| inc0 | I have good idea | 16:46 |
| inc0 | don't keep password login on public-facing server | 16:47 |
| inc0 | also...don't use 6 letter all lowercase password | 16:47 |
| *** lmiccini has joined #openstack-infra | 16:47 | |
| inc0 | ;) | 16:47 |
| inc0 | we were sloppy, that's all | 16:47 |
| inc0 | we carefully read all the security good practices and decided to ignore all of them | 16:48 |
| clarkb | I have rechecked the devstack fix for linuxbridge jobs | 16:48 |
| *** pilgrimstack has quit IRC | 16:48 | |
| clarkb | once that is in we can approve the change to remove eth0 from being hardcoded in the configs then we can revert the change that made those tests not run on xenial in check/gate | 16:48 |
| anteaya | clarkb: awesome | 16:49 |
| clarkb | armax: ^ fyi | 16:49 |
| clarkb | and then I can review glean things | 16:49 |
| openstackgerrit | Merged openstack-infra/openstackid-resources: Smart Conference https://review.openstack.org/368148 | 16:51 |
| *** xarses has joined #openstack-infra | 16:51 | |
| zigo | Is Xenial using BIOS names for ifaces, even in KVM VMs? | 16:51 |
| clarkb | zigo: it is | 16:52 |
| fungi | inc0: fair enough! so restated, i guess sshd was configured to allow password auth and you had a too-weak system account password on one or more servers and 22/tcp exposed to the internet? | 16:52 |
| *** tosky has quit IRC | 16:52 | |
| *** jpich has quit IRC | 16:52 | |
| inc0 | yup | 16:53 |
| *** gouthamr_ is now known as gouthamr | 16:53 | |
| fungi | we mainly mitigate it through disallowing password auth for sshd on all our systems. key auth only | 16:53 |
| inc0 | yeah, that's what we're going to do | 16:53 |
| fungi | even my personal servers are configured to only allow ssh key auth | 16:53 |
| inc0 | also I'd just listen on mgmt network | 16:53 |
| clarkb | #status ok setuptools 27.1.2 addresses the circular import | 16:54 |
| openstackstatus | clarkb: sending ok | 16:54 |
| anteaya | clarkb: thank you | 16:54 |
| anteaya | dstufft: thank you | 16:54 |
| inc0 | my personal fav is to not use 0.0.0.0 ever | 16:54 |
| *** derekh has quit IRC | 16:54 | |
| inc0 | but yeah...all our knowledge | 16:54 |
| anteaya | dstufft: and thanks to Jason | 16:54 |
| inc0 | we were just sloppy | 16:54 |
| anteaya | dmsimard: ^^ | 16:54 |
| dmsimard | \o/ | 16:54 |
| *** ilyashakhat_mobi has joined #openstack-infra | 16:54 | |
| *** Swami has joined #openstack-infra | 16:55 | |
| dmsimard | So can we put the channel topics back on now? :p | 16:56 |
| anteaya | smcginnis: it is happening | 16:56 |
| clarkb | dmsimard: it should do it automatically | 16:56 |
| anteaya | dmsimard: that was for you | 16:56 |
| dmsimard | ah, okay | 16:56 |
| smcginnis | :) | 16:56 |
| anteaya | smcginnis: sorry about that, meant as reply to dmsimard | 16:56 |
| clarkb | dmsimard: it throttles itself to avoid getting kicked for flooding | 16:56 |
| dmsimard | clarkb: makes sense | 16:56 |
| anteaya | smcginnis: and happy friday to you | 16:56 |
| *** ChanServ changes topic to "Discussion of OpenStack Developer and Community Infrastructure | docs http://docs.openstack.org/infra/ | bugs https://storyboard.openstack.org/ | source https://git.openstack.org/cgit/openstack-infra/ | channel logs http://eavesdrop.openstack.org/irclogs/%23openstack-infra/" | 16:56 | |
| -openstackstatus- NOTICE: setuptools 27.1.2 addresses the circular import | 16:56 | |
| *** trown is now known as trown|lunch | 16:56 | |
| fungi | inc0: yep, in our case we need to allow ssh from everywhere, but key auth alone is enough to have kept us from worrying about brute-force login attacks (we just have to worry about someone compromising a trusted key, or pre-authentication vulnerabilities in the sshd) | 16:57 |
| inc0 | yeah ofc, but you know what I mean, in Kolla for example wherever we can remove 0.0.0.0, we remove it | 16:57 |
| openstackgerrit | Andrey Nikitin proposed openstack-infra/jeepyb: Update mechanism of creating groups in database and managing access rights with jeepyb https://review.openstack.org/284843 | 16:58 |
| fungi | inc0: also i hope you'd have used :: instead of 0.0.0.0 anyway (dual-stack friendly syntax for all addresses!) | 16:59 |
| openstackstatus | clarkb: finished sending ok | 16:59 |
| *** dizquierdo has quit IRC | 17:00 | |
| inc0 | good point fungi, no idea.;) | 17:00 |
| clarkb | btw I am going to weekend early today in order to do doctor appointments this afternoon. (seems like they always come in clumps) | 17:00 |
| clarkb | at least the pediatrician will see the girls back to back instead of requiring two seaprate appointments | 17:01 |
| *** degorenko is now known as _degorenko|afk | 17:01 | |
| fungi | clarkb: have a good afternoon | 17:02 |
| fungi | i'm hoping i get freed up enough to get back to wiki testing for the rest of the day | 17:02 |
| *** rodrigods has quit IRC | 17:03 | |
| anteaya | clarkb: I hope you have good doctor's appointments, yay for back to back appointments | 17:03 |
| *** rodrigods has joined #openstack-infra | 17:03 | |
| anteaya | clarkb: and enjoy the weekend | 17:03 |
| *** tongli has joined #openstack-infra | 17:03 | |
| clarkb | (I'm not leaving yet just making sure you don't all expect me to be around in ~4-5 hours) | 17:03 |
| *** kzaitsev_mb has joined #openstack-infra | 17:04 | |
| *** tqtran has joined #openstack-infra | 17:05 | |
| *** vhosakot has quit IRC | 17:05 | |
| fungi | i actually need to pop out for a few minutes. back in ~1 hour | 17:05 |
| *** yamahata has quit IRC | 17:06 | |
| fungi | oh, did the puppet lint job issues get worked out yet? looks like we were still seeing failures as of 30 minutes ago. any fix for that i can help fast-track? | 17:07 |
| anteaya | clarkb: ah | 17:07 |
| anteaya | fungi: happy errands | 17:07 |
| *** chem has quit IRC | 17:10 | |
| *** matt-borland has quit IRC | 17:10 | |
| *** Na3iL has quit IRC | 17:11 | |
| *** JerryOpenix has joined #openstack-infra | 17:11 | |
| *** JerryOpenix has quit IRC | 17:11 | |
| *** ddieterly is now known as ddieterly[away] | 17:11 | |
| *** sarob has joined #openstack-infra | 17:12 | |
| *** sarob has quit IRC | 17:13 | |
| *** sarob has joined #openstack-infra | 17:13 | |
| *** senk has quit IRC | 17:14 | |
| *** zul has quit IRC | 17:15 | |
| *** senk has joined #openstack-infra | 17:15 | |
| *** zul has joined #openstack-infra | 17:16 | |
| *** tonytan4ever has quit IRC | 17:16 | |
| *** kzaitsev_mb has quit IRC | 17:17 | |
| *** Hal1 has joined #openstack-infra | 17:17 | |
| *** asselin_ has joined #openstack-infra | 17:17 | |
| *** rossella_s has quit IRC | 17:18 | |
| *** bethwhite_ has quit IRC | 17:19 | |
| *** rossella_s has joined #openstack-infra | 17:19 | |
| clarkb | fungi: was that the setuptools thing? | 17:19 |
| *** e0ne has joined #openstack-infra | 17:20 | |
| clarkb | hrm shouldn't be for puppet lint | 17:20 |
| *** vhosakot has joined #openstack-infra | 17:24 | |
| clarkb | pabelanger: on https://review.openstack.org/#/c/366937/4/glean/tests/test_glean.py shoudl we add a test where multiple networks share the same link? | 17:26 |
| raddaoui | pabelanger: clarkb , we are repaving env, should be up again in few hours | 17:26 |
| clarkb | raddaoui: ok | 17:26 |
| pabelanger | clarkb: let me check if we have a test for that | 17:27 |
| *** bnemec is now known as beekneemech | 17:28 | |
| *** asselin__ has joined #openstack-infra | 17:30 | |
| *** jcoufal has quit IRC | 17:30 | |
| *** flepied has quit IRC | 17:30 | |
| *** asselin_ has quit IRC | 17:31 | |
| openstackgerrit | Sumit Naiksatam proposed openstack-infra/project-config: Add non-voting job for GBP https://review.openstack.org/367591 | 17:32 |
| *** e0ne has quit IRC | 17:32 | |
| *** devananda|dinner is now known as devananda | 17:32 | |
| *** thorst_ has joined #openstack-infra | 17:33 | |
| *** vhosakot has quit IRC | 17:34 | |
| *** tonytan4ever has joined #openstack-infra | 17:37 | |
| clarkb | pabelanger: looks like its added in the next change | 17:37 |
| *** thorst_ has quit IRC | 17:38 | |
| pabelanger | clarkb: Ya, that's right. I moved it up a patch set because we didn't have test for it | 17:38 |
| *** tonytan_brb has joined #openstack-infra | 17:38 | |
| pabelanger | but made sure we did test in the follow up | 17:38 |
| *** e0ne has joined #openstack-infra | 17:38 | |
| clarkb | pabelanger: and have we done any boot testing of real VMs yet? | 17:40 |
| pabelanger | clarkb: no, that is the last step to do. I was going to ask you to create a minimal image again :) Since I had bad luck last time around | 17:40 |
| clarkb | pabelanger: ah ok, I can give you what I ran if you want to try too | 17:41 |
| * clarkb boots dib builder VM | 17:41 | |
| pabelanger | clarkb: please | 17:41 |
| pabelanger | I'm doing something wrong locally | 17:42 |
| *** tonytan4ever has quit IRC | 17:42 | |
| clarkb | pabelanger: `DIB_INSTALLTYPE_simple_init=repo DIB_RELEASE=xenial DIB_DEV_USER_PWDLESS_SUDO=yes DIB_DEV_USER_AUTHORIZED_KEYS=/home/ubuntu/.ssh/authorized_keys DIB_DEV_USER_PASSWORD=foobar DIB_REPOLOCATION_glean=/home/ubuntu/glean DIB_REPOREF_glean=clarkb time tools/build-image.sh` is the command then I edit build-image.sh so that the elements list is minimal | 17:42 |
| *** gyee has joined #openstack-infra | 17:43 | |
| anteaya | can I get another project-config core reviewer to review this change to bring back keystoneauth-saml2 jobs so that stevemar can remove the dot files on the repo, please? https://review.openstack.org/#/c/368089 | 17:43 |
| *** acoles is now known as acoles_ | 17:43 | |
| *** _nadya_ has quit IRC | 17:43 | |
| clarkb | pabelanger: http://paste.openstack.org/show/570294/ | 17:44 |
| clarkb | pabelanger: should I build xenial again? | 17:44 |
| anteaya | they were left behind on the first retirement attempt | 17:44 |
| pabelanger | clarkb: sure, I'll get trusty | 17:44 |
| clarkb | pabelanger: oh ya you have to add haveged to infra package needs too | 17:44 |
| *** yamahata has joined #openstack-infra | 17:44 | |
| pabelanger | right | 17:44 |
| pabelanger | need to fix our element for that | 17:44 |
| clarkb | I should probably fix that properly | 17:45 |
| clarkb | ya | 17:45 |
| clarkb | pabelanger: if I fetch the gentoo chagne will that include the other two? they are all stacked ya? | 17:45 |
| pabelanger | clarkb: yes | 17:46 |
| *** _nadya_ has joined #openstack-infra | 17:46 | |
| clarkb | ok I have xenial running locally with that glean stack | 17:47 |
| * clarkb is going to trust that prometheanfire's +1 means gentoo works | 17:48 | |
| prometheanfire | yes | 17:48 |
| prometheanfire | :D | 17:48 |
| *** salv-orlando has quit IRC | 17:49 | |
| *** Na3iL has joined #openstack-infra | 17:49 | |
| *** salv-orlando has joined #openstack-infra | 17:49 | |
| clarkb | pabelanger: you'll want to edit the authorized keys file and the dib repolocation and reporefs for glean | 17:51 |
| clarkb | so they match whatever your local setup is using | 17:51 |
| pabelanger | clarkb: ack | 17:52 |
| *** senk has quit IRC | 17:53 | |
| clarkb | and yes I create branches with my name in them :P (habit formed when using shared resources for testing) | 17:53 |
| *** ilyashakhat_mobi has quit IRC | 17:55 | |
| *** trown|lunch is now known as trown | 17:55 | |
| clarkb | pabelanger: once my build is done I will boot it locally to see that that at least works then if so I can put it on nodepool | 18:00 |
| *** burgerk_ has quit IRC | 18:01 | |
| *** mfisch has joined #openstack-infra | 18:01 | |
| mfisch | /usr/local/lib/ruby/gems/2.1.0/gems/bundler-1.10.6/lib/bundler/rubygems_integration.rb:292:in `block in replace_gem': rake is not part of the bundle. Add it to Gemfile. (Gem::LoadError) | 18:02 |
| mfisch | from /usr/local/bin/rake:22:in `<main>' | 18:02 |
| mfisch | sorry about that ^ | 18:02 |
| *** burgerk has joined #openstack-infra | 18:02 | |
| pabelanger | clarkb: sure, build still running for trusty here | 18:03 |
| openstackgerrit | Sumit Naiksatam proposed openstack-infra/project-config: Add non-voting job for GBP https://review.openstack.org/367591 | 18:03 |
| AJaeger | clarkb: https://review.openstack.org/#/c/368078/ passes now | 18:04 |
| AJaeger | stevemar: all looks fine, thanks | 18:04 |
| *** vhosakot has joined #openstack-infra | 18:04 | |
| *** vhosakot has quit IRC | 18:05 | |
| clarkb | pabelanger: it works for me local one nic ipv4 setup | 18:05 |
| *** cardeois_ has joined #openstack-infra | 18:05 | |
| *** akshai has quit IRC | 18:05 | |
| pabelanger | clarkb: great! our unit testing is valid | 18:06 |
| pabelanger | just converting image now | 18:06 |
| stevemar | anteaya: thanks for the reviews | 18:06 |
| pabelanger | clarkb: rax-iad should be the region we want first | 18:06 |
| *** Hal1 has quit IRC | 18:06 | |
| anteaya | stevemar: thanks for taking the trouble to clean this up | 18:06 |
| pabelanger | maybe ord too | 18:06 |
| stevemar | anteaya: np | 18:07 |
| clarkb | pabelanger: ok you didn't happen to write down the magical shade stuff did you? I should do that if not (after I figure it out again) | 18:07 |
| *** cardeois has quit IRC | 18:09 | |
| anteaya | stevemar: so let me know once that repo has zero dot files and a corrected readme | 18:09 |
| stevemar | anteaya: should be quick, it'll be only no-op jobs in the gate | 18:10 |
| stevemar | anteaya: the whole chain is here: https://review.openstack.org/#/q/topic:retire-keystoneauth-saml2-again | 18:10 |
| AJaeger | armax: you gave your +2 on the dependent change, do you want to +1 https://review.openstack.org/#/c/368012/ as well? | 18:10 |
| anteaya | stevemar: great thank you | 18:11 |
| armax | AJaeger: dougwig should have a look too | 18:11 |
| mfisch | any known issues with gerrit atm? I can't rebase | 18:12 |
| mfisch | https://review.openstack.org/#/c/344464/ | 18:12 |
| *** burgerk has quit IRC | 18:12 | |
| mfisch | "Unable to create new object..." | 18:12 |
| clarkb | mfisch: usually that means you hae a conflict | 18:12 |
| *** akshai has joined #openstack-infra | 18:12 | |
| mfisch | it does say merge confluct | 18:13 |
| clarkb | yup there is a red cannot merge string on that change | 18:13 |
| mfisch | wasnt familiar with that error | 18:13 |
| pabelanger | clarkb: http://eavesdrop.openstack.org/irclogs/%23openstack-infra/%23openstack-infra.2016-09-07.log.html#t2016-09-07T14:36:02 | 18:13 |
| mfisch | ok I'll go do it by hand | 18:13 |
| clarkb | mfisch: means that git cannot resolve the rebase/merge on its own so you have to do it | 18:13 |
| clarkb | pabelanger: ty | 18:13 |
| mfisch | clarkb: yep thanks | 18:13 |
| openstackgerrit | Merged openstack-infra/project-config: bring back keystoneauth-saml2 jobs temporarily https://review.openstack.org/368089 | 18:13 |
| *** sarob has quit IRC | 18:14 | |
| clarkb | pabelanger: that is now in the readme in that dir | 18:14 |
| *** sarob has joined #openstack-infra | 18:14 | |
| openstackgerrit | Sumit Naiksatam proposed openstack-infra/project-config: Add non-voting job for GBP https://review.openstack.org/367591 | 18:15 |
| pabelanger | clarkb: | 18:15 |
| pabelanger | err | 18:15 |
| pabelanger | k | 18:15 |
| AJaeger | stevemar: before you recheck the saml2 changes, wait that zuul is reconfigured - see "Last reconfigured" at the bottom of http://status.openstack.org/zuul/ | 18:16 |
| AJaeger | otherwise the job change is not life | 18:16 |
| clarkb | pabelanger: uploading now will be called clarkb-multi-addr-glean | 18:16 |
| stevemar | AJaeger: ah, i figured there was some lag, but assumed the recheck was harmless | 18:16 |
| clarkb | pabelanger: in the nodepool account rax iad | 18:16 |
| mfisch | clarkb: manual rebase works but now I see "Conflicts with N/A: 500 Internal Server Error" | 18:17 |
| AJaeger | stevemar: yes, it's harmless - just premature ;) | 18:17 |
| clarkb | mfisch: did you update to latest master before rebasing? | 18:17 |
| mfisch | y | 18:17 |
| pabelanger | clarkb: thanks | 18:18 |
| mfisch | rebase went ok from the cli here | 18:18 |
| openstackgerrit | Merged openstack-infra/project-config: Publish salt specs https://review.openstack.org/365361 | 18:18 |
| clarkb | mfisch: I think thats saying it doesn't conflcit with anything anymore | 18:18 |
| clarkb | mfisch: may be a bug in gerrit to keep showing that panel after you fix the conflict | 18:18 |
| mfisch | clarkb: agreed, but I didnt expect to see an internal server error | 18:18 |
| AJaeger | clarkb, dtantsur|afk : http://users.suse.com/~aj/jobs-unused.txt and http://users.suse.com/~aj/jobs-used.txt are the list of unused and used jobs that I generated. In case you want to do some more analysis... | 18:18 |
| mfisch | clarkb: I'd agree with that | 18:18 |
| clarkb | mfisch: I think it thinks it needs to show you that due to caching or something and its just a bug in the UX | 18:18 |
| clarkb | AJaeger: ty | 18:18 |
| openstackgerrit | Merged openstack-infra/storyboard-webclient: Add a way to add tasks to worklists from the story view https://review.openstack.org/366899 | 18:20 |
| openstackgerrit | Merged openstack-infra/project-config: Create gate-grenade-dsvm-neutron-nova-next-ubuntu-xenial-nv https://review.openstack.org/368087 | 18:20 |
| *** eranrom has quit IRC | 18:21 | |
| *** Na3iL has quit IRC | 18:21 | |
| *** esberglu has quit IRC | 18:22 | |
| pabelanger | clarkb: were getting better with nodepool, almost 20mins without a launch node failure | 18:22 |
| pabelanger | 18mins that time | 18:22 |
| clarkb | pabelanger: slow but steady progress. Image upload is done if you want to take over from there | 18:23 |
| pabelanger | clarkb: sure | 18:23 |
| *** sarob has quit IRC | 18:24 | |
| *** sarob has joined #openstack-infra | 18:24 | |
| *** sbalukoff has joined #openstack-infra | 18:26 | |
| *** dtardivel has quit IRC | 18:27 | |
| *** sarob has quit IRC | 18:29 | |
| *** flepied has joined #openstack-infra | 18:30 | |
| *** dkehn has quit IRC | 18:31 | |
| *** _nadya_ has quit IRC | 18:33 | |
| *** dkehn has joined #openstack-infra | 18:33 | |
| *** dkehn_ has quit IRC | 18:33 | |
| *** dkehn__ has quit IRC | 18:33 | |
| *** jcoufal has joined #openstack-infra | 18:34 | |
| *** esberglu has joined #openstack-infra | 18:34 | |
| *** tkelsey has joined #openstack-infra | 18:34 | |
| openstackgerrit | Zara proposed openstack-infra/storyboard-webclient: Hide edit buttons for logged-out users https://review.openstack.org/368176 | 18:34 |
| *** jed56 has quit IRC | 18:35 | |
| openstackgerrit | Merged openstack-infra/project-config: Normalize projects.yaml https://review.openstack.org/367817 | 18:36 |
| *** esberglu has quit IRC | 18:37 | |
| AJaeger | stevemar: you might need to toggle the +W for the saml2 changes | 18:37 |
| *** ilyashakhat_mobi has joined #openstack-infra | 18:37 | |
| AJaeger | stevemar: oh wait, zuul is queing right now... | 18:37 |
| clarkb | armax: the devstack change for the interface thing is ~40 minutes away from merging according to zuul. You good with me merging your job updates to remove the eth0 hardcode then revert the move to experimental once that is in? | 18:38 |
| *** ilyashakhat_mobi has quit IRC | 18:39 | |
| armax | clarkb: yes | 18:39 |
| AJaeger | stevemar, anteaya : saml2 is clean now, we can merge https://review.openstack.org/368135 and https://review.openstack.org/367844 | 18:39 |
| *** rbrndt has quit IRC | 18:39 | |
| anteaya | clean saml2, yay | 18:40 |
| anteaya | AJaeger: is the readme correct? http://git.openstack.org/cgit/openstack/keystoneauth-saml2/tree/README.rst | 18:40 |
| *** tkelsey has quit IRC | 18:40 | |
| pabelanger | clarkb: Ya, ipv4 works | 18:40 |
| pabelanger | trying to find out why ipv6 doesn't | 18:41 |
| AJaeger | anteaya: yes, it is. Thanks for double checking | 18:41 |
| anteaya | AJaeger: wonderful | 18:42 |
| anteaya | stevemar: thank you | 18:42 |
| anteaya | AJaeger: this one next? https://review.openstack.org/#/c/368135/ | 18:42 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Remove unused charm jobs https://review.openstack.org/368179 | 18:43 |
| AJaeger | anteaya: they're stacked ;) But yes | 18:43 |
| *** dkehn__ has joined #openstack-infra | 18:43 | |
| AJaeger | anteaya, clarkb: 368179 removes around 200 unused charm jobs - a tiny cleanup | 18:43 |
| *** dkehn_ has joined #openstack-infra | 18:43 | |
| *** esberglu has joined #openstack-infra | 18:44 | |
| *** senk has joined #openstack-infra | 18:44 | |
| AJaeger | clarkb: would be nice if you could review https://review.openstack.org/368078 please to get rid of a few ironic unused jobs to make it easier for them. | 18:44 |
| *** zul has quit IRC | 18:44 | |
| clarkb | AJaeger: ya will queue those up | 18:44 |
| * AJaeger wishes everybody a great weekend | 18:44 | |
| AJaeger | thanks, clarkb | 18:44 |
| anteaya | AJaeger: thank you, you too | 18:46 |
| *** Sukhdev has joined #openstack-infra | 18:48 | |
| pabelanger | clarkb: yay, ipv6 working too | 18:50 |
| pabelanger | but we have a bug to fix | 18:50 |
| clarkb | pabelanger: ok | 18:50 |
| *** mfisch has left #openstack-infra | 18:51 | |
| pabelanger | clarkb: are we okay with stacking it on the end? Or do the fix in 366862 | 18:51 |
| clarkb | pabelanger: I think we should fix the appropriate change | 18:52 |
| pabelanger | I'm good with that | 18:52 |
| openstackgerrit | Merged openstack-infra/tripleo-ci: scenarios: set Debug to True https://review.openstack.org/366896 | 18:52 |
| *** esberglu has quit IRC | 18:52 | |
| openstackgerrit | Merged openstack-infra/project-config: Revert "bring back keystoneauth-saml2 jobs temporarily" https://review.openstack.org/368135 | 18:55 |
| openstackgerrit | Merged openstack-infra/project-config: Remove unused jobs from devstack-gate https://review.openstack.org/368078 | 18:56 |
| *** sdague has joined #openstack-infra | 18:57 | |
| *** ddieterly[away] is now known as ddieterly | 18:57 | |
| fungi | just to confirm, nobody has more details on the current state of the puppet lint issue yolanda was looking into (in scrollback around 12:41-13:13 utc)? if no, i'll start digging deeper in a bit | 18:57 |
| clarkb | I do not | 18:57 |
| fungi | i think it's currently blocking all our puppet module changes | 18:57 |
| *** esikachev has joined #openstack-infra | 18:58 | |
| anteaya | i do not | 18:58 |
| *** zul has joined #openstack-infra | 19:00 | |
| *** MaryM has quit IRC | 19:00 | |
| *** Mary has joined #openstack-infra | 19:03 | |
| *** Mary is now known as Guest5783 | 19:03 | |
| stevemar | AJaeger: anteaya thanks for the reviews! it was a nice sight to see after lunch :) | 19:05 |
| *** rbrndt has joined #openstack-infra | 19:06 | |
| stevemar | clarkb / fungi want to punt this one through? https://review.openstack.org/#/c/367844/ | 19:06 |
| anteaya | stevemar: thanks for the cleanup | 19:09 |
| anteaya | good to get it all tidied up | 19:09 |
| stevemar | anteaya: ma pleasure! | 19:09 |
| anteaya | :) | 19:09 |
| *** ilyashakhat_mobi has joined #openstack-infra | 19:10 | |
| anteaya | fungi: thank you | 19:10 |
| *** e0ne has quit IRC | 19:11 | |
| *** senk has quit IRC | 19:13 | |
| *** ldnunes has quit IRC | 19:13 | |
| *** kzaitsev_mb has joined #openstack-infra | 19:16 | |
| openstackgerrit | Merged openstack-infra/project-config: Complete retirement of keystoneauth-saml2 https://review.openstack.org/367844 | 19:17 |
| *** isq has quit IRC | 19:17 | |
| *** spzala has quit IRC | 19:17 | |
| openstackgerrit | Doug Hellmann proposed openstack-infra/release-tools: teach latest-deliverable-versions to diff against the dashboard https://review.openstack.org/368186 | 19:20 |
| *** IlyaG has quit IRC | 19:20 | |
| openstackgerrit | Zara proposed openstack-infra/storyboard-webclient: Fix behaviour for logged-out users https://review.openstack.org/368176 | 19:22 |
| *** cgross has quit IRC | 19:22 | |
| *** lmiccini has quit IRC | 19:22 | |
| *** mkoderer has quit IRC | 19:23 | |
| *** ddieterly is now known as ddieterly[away] | 19:24 | |
| *** cardeois has joined #openstack-infra | 19:25 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/glean: Append to interface definition for multi-address https://review.openstack.org/366862 | 19:25 |
| openstackgerrit | Paul Belanger proposed openstack-infra/glean: Add ipv6 support to gentoo https://review.openstack.org/367487 | 19:25 |
| *** jheroux has quit IRC | 19:26 | |
| *** cardeoi__ has joined #openstack-infra | 19:26 | |
| *** cardeois_ has quit IRC | 19:28 | |
| *** esberglu has joined #openstack-infra | 19:29 | |
| *** ddieterly[away] is now known as ddieterly | 19:29 | |
| *** cardeois has quit IRC | 19:30 | |
| *** roxanagh_ has quit IRC | 19:31 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/glean: Append to interface definition for multi-address https://review.openstack.org/366862 | 19:31 |
| openstackgerrit | Paul Belanger proposed openstack-infra/glean: Add ipv6 support to gentoo https://review.openstack.org/367487 | 19:31 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Cleanup: Specs repos are branchless https://review.openstack.org/368191 | 19:32 |
| AJaeger | clarkb: we added a few jobs too many - specs have no stable branches ^ | 19:32 |
| jesusaur | AJaeger: thanks for the reviews on 363969 and fixing the underlying issues | 19:32 |
| openstackgerrit | Paul Belanger proposed openstack-infra/glean: Append to interface definition for multi-address https://review.openstack.org/366862 | 19:33 |
| openstackgerrit | Paul Belanger proposed openstack-infra/glean: Add ipv6 support to gentoo https://review.openstack.org/367487 | 19:33 |
| openstackgerrit | Shamail Tahir proposed openstack-infra/irc-meetings: Add Interop Challenge Weekly IRC Meeting https://review.openstack.org/366851 | 19:33 |
| pabelanger | okay | 19:33 |
| pabelanger | I think that is it | 19:33 |
| AJaeger | jesusaur: thanks for the change - it helped ;) Will +2 once it passes... | 19:33 |
| clarkb | AJaeger: ok | 19:33 |
| *** _nadya_ has joined #openstack-infra | 19:33 | |
| jesusaur | yay | 19:34 |
| * jesusaur is helping | 19:34 | |
| pabelanger | fungi: do you have a moment to look at some ipv6 configuration? https://review.openstack.org/#/c/366862/8/glean/tests/fixtures/test/rax-iad.debian.network.out | 19:34 |
| *** cardeoi__ has quit IRC | 19:34 | |
| pabelanger | fungi: If you can confirm the settings on eth0 for static routes | 19:35 |
| anteaya | AJaeger: why are you getting rid of gate-api-wg-python27-ubuntu-trusty in 368191 | 19:35 |
| *** isq has joined #openstack-infra | 19:35 | |
| *** cardeois has joined #openstack-infra | 19:36 | |
| AJaeger | anteaya: check the branches the project has - and when the job runs. You will notice that the repo has only master and that all ubuntu-trusty jobs are only run on older stable branches. | 19:36 |
| AJaeger | So, gate-api-wg-python27-ubuntu-trusty will never be used | 19:36 |
| pabelanger | clarkb: glean stack ready | 19:36 |
| anteaya | okay the commit message just says specs jobs though | 19:36 |
| AJaeger | anteaya: And yes, I check the repo for branches ;) | 19:36 |
| AJaeger | anteaya: that repo uses the specs template | 19:36 |
| anteaya | oh | 19:36 |
| AJaeger | "openstack-specs-jobs" is used by it - and I searched for repos that use "specs-jobs" | 19:37 |
| AJaeger | and for most - especially these odd ones - I run "git branch -a". | 19:37 |
| AJaeger | still, I might have missed one, so I appreciate double checking ;) | 19:38 |
| *** sdake has joined #openstack-infra | 19:38 | |
| *** _nadya_ has quit IRC | 19:38 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/project-config: Remove hardcoding to eth0 in LinuxBridge job configuration https://review.openstack.org/367704 | 19:38 |
| openstackgerrit | Clark Boylan proposed openstack-infra/project-config: Revert "Fix linuxbridge job breakage on xenial" https://review.openstack.org/368195 | 19:38 |
| clarkb | anteaya: armax AJaeger fungi ^ thats the reorganizing of thinsg so that we can test linuxbridge on xenial as soon as the devstack change merges | 19:39 |
| anteaya | sure | 19:39 |
| *** dimtruck is now known as zz_dimtruck | 19:39 | |
| clarkb | pabelanger: have you rebuilt an image using those chagnes and booted it in rax? | 19:39 |
| fungi | clarkb: thanks. in the meantime, check experimental should still be exercising it | 19:39 |
| clarkb | fungi: yup | 19:40 |
| clarkb | fungi: everything should just work at this point in time, its just a matter of merging things one at a time and double checking | 19:40 |
| AJaeger | clarkb: please self-approve once the dependend job merged | 19:40 |
| clarkb | AJaeger: can do | 19:40 |
| fungi | awesome. that's some quick work | 19:40 |
| clarkb | (armax already gave his blessing too) | 19:40 |
| pabelanger | clarkb: not yet, I'd like to get confirmation the static ipv6 routes is correct first. | 19:41 |
| pabelanger | I am testing them now | 19:41 |
| *** mbacchi has quit IRC | 19:41 | |
| openstackgerrit | Shamail Tahir proposed openstack-infra/irc-meetings: Add Interop Challenge Weekly IRC Meeting https://review.openstack.org/366851 | 19:42 |
| *** sdake has quit IRC | 19:42 | |
| clarkb | AJaeger: on https://review.openstack.org/#/c/368191/1/zuul/layout.yaml is the reason more of them don't use the zuul jobs template because they don't want or we don't want them publishing the specs to the spec publishing location? | 19:42 |
| AJaeger | zigo, somehow you screwed up with openstack/deb-nodepool - that one is in zuul/layout but not in gerrit/projects.yaml ;( | 19:42 |
| AJaeger | clarkb: the template is the minimal set of jobs, but many use python27 jobs in addition. | 19:43 |
| clarkb | oh I see that now, thanks | 19:43 |
| AJaeger | or pep8 in addition | 19:44 |
| clarkb | looks like also pep8 ya | 19:44 |
| *** sdake has joined #openstack-infra | 19:44 | |
| *** zz_dimtruck is now known as dimtruck | 19:47 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Remove wrongly set up deb-nodepool https://review.openstack.org/368214 | 19:47 |
| AJaeger | deb-nodepool set up is broken, let's remove it - and readd it later ^ | 19:47 |
| *** mbacchi has joined #openstack-infra | 19:48 | |
| openstackgerrit | Merged openstack-infra/project-config: change post_test_hook location for osc functional tests https://review.openstack.org/365865 | 19:48 |
| *** cardeois has quit IRC | 19:49 | |
| * AJaeger leaves again... | 19:49 | |
| clarkb | AJaeger: enjoy your weekend | 19:49 |
| *** ccamacho has quit IRC | 19:49 | |
| *** cardeois has joined #openstack-infra | 19:50 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/glean: Append to interface definition for multi-address https://review.openstack.org/366862 | 19:52 |
| openstackgerrit | Paul Belanger proposed openstack-infra/glean: Add ipv6 support to gentoo https://review.openstack.org/367487 | 19:52 |
| pabelanger | route -6 add fd30::/48 gw fe80::f001 should be valid syntax | 19:52 |
| *** esikache1 has joined #openstack-infra | 19:53 | |
| pabelanger | Oh, maybe not | 19:53 |
| pabelanger | hmm | 19:53 |
| clarkb | pabelanger: -A inet6 | 19:54 |
| clarkb | pabelanger: or maybe use ip route | 19:54 |
| pabelanger | ya, going to do ip route I think | 19:54 |
| pabelanger | most of the examples are using it | 19:54 |
| *** ddieterly is now known as ddieterly[away] | 19:54 | |
| *** esikachev has quit IRC | 19:56 | |
| pabelanger | up ip -6 route add fd30::/48 via fe80::f001 dev eth0 || true | 20:01 |
| pabelanger | think that is right | 20:01 |
| *** _nadya_ has joined #openstack-infra | 20:02 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/glean: Append to interface definition for multi-address https://review.openstack.org/366862 | 20:02 |
| *** cardeois has quit IRC | 20:02 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/glean: Add ipv6 support to gentoo https://review.openstack.org/367487 | 20:02 |
| *** gyee has quit IRC | 20:02 | |
| pabelanger | clarkb: okay, building new image now | 20:02 |
| *** _nadya_ has quit IRC | 20:03 | |
| clarkb | pabelanger: ya that looks roughly correct. However, do you need a default route too? | 20:03 |
| pabelanger | clarkb: that is already working | 20:03 |
| *** tongli has quit IRC | 20:03 | |
| pabelanger | well, the syntax is working | 20:03 |
| clarkb | gotcha | 20:03 |
| pabelanger | I was only having issues with static routes | 20:03 |
| pabelanger | okay, build started | 20:04 |
| * clarkb is worried the devstack fix for linuxbridge may timeout one of its jobs | 20:05 | |
| clarkb | I think we have a few more minutes to finish up tempest and report bcak | 20:05 |
| clarkb | where is the turbo button | 20:05 |
| *** mbacchi has quit IRC | 20:06 | |
| *** esberglu has quit IRC | 20:06 | |
| * anteaya looks around | 20:06 | |
| anteaya | you don't hold the turbo down, it's for quick boosts | 20:06 |
| anteaya | yeah like you know | 20:06 |
| openstackgerrit | Merged openstack-infra/project-config: Cleanup: Specs repos are branchless https://review.openstack.org/368191 | 20:06 |
| *** cardeois has joined #openstack-infra | 20:07 | |
| *** zul has quit IRC | 20:10 | |
| *** zul has joined #openstack-infra | 20:10 | |
| *** esikache1 has quit IRC | 20:11 | |
| *** cardeois has quit IRC | 20:12 | |
| *** drawsmcgraw1 has quit IRC | 20:13 | |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci: Limit Heat engine workers to 4 in rh1 https://review.openstack.org/360650 | 20:13 |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci: Limit heat, neutron, and nova workers https://review.openstack.org/368220 | 20:13 |
| *** drawsmcgraw has joined #openstack-infra | 20:13 | |
| mordred | clarkb, pabelanger: heya! sorry - today has eaten my lunch - are there any glean or other things I should be looking at? | 20:15 |
| clarkb | mordred: yes the stack around https://review.openstack.org/367487 pabelanger is doing real world testing of it now | 20:16 |
| *** ilyashakhat_mobi has quit IRC | 20:16 | |
| harlowja | can not import monkey? | 20:16 |
| clarkb | mordred: it was mostly working except for ipv6 route addtions so fix for that is being tested now | 20:16 |
| harlowja | whattttt | 20:16 |
| anteaya | harlowja: recheck | 20:16 |
| harlowja | (no i'm just wondering what monkey is, lol) | 20:16 |
| mordred | clarkb: sweet | 20:16 |
| anteaya | harlowja: the first of two setuptools releases today couldn't find monkey | 20:17 |
| harlowja | lol | 20:17 |
| harlowja | that just sounds funny | 20:17 |
| harlowja | :) | 20:17 |
| fungi | harlowja: luckily the monkey has been found and returned to its native habitat | 20:17 |
| harlowja | safe? | 20:17 |
| harlowja | hopefully no poo thrown? | 20:17 |
| fungi | not in here at least | 20:17 |
| clarkb | probably on twitter | 20:17 |
| fungi | most of us are more evolved primates | 20:18 |
| harlowja | ha | 20:18 |
| *** tonytan_brb has quit IRC | 20:18 | |
| harlowja | so u think fungi so u think... | 20:18 |
| *** ddieterly[away] is now known as ddieterly | 20:18 | |
| fungi | yeah, no doubt the "bug reports" submitted via twitter were rife with poo flinging | 20:18 |
| dims | LOL | 20:18 |
| * mordred really should finish the statusbot twitter publisher | 20:18 | |
| dtroyer | ++ | 20:19 |
| dtroyer | I need a reason to read more twitter | 20:19 |
| fungi | harlowja: i _did_ say "most" | 20:19 |
| harlowja | :) | 20:19 |
| * fungi is, on the other hand, a long-standing member of the devolutionary army | 20:19 | |
| anteaya | fungi: does your army have turbo buttons? | 20:20 |
| fungi | no, but we have flowerpots | 20:20 |
| anteaya | nice | 20:20 |
| *** thorst_ has joined #openstack-infra | 20:20 | |
| mordred | harlowja: OH! the twitter support merged | 20:20 |
| harlowja | i've seen movies about this, doesn't seem to end well for humans | 20:20 |
| clarkb | my army just demands food all the time | 20:20 |
| mordred | harlowja: I guess I just need to make an account and get an API key and configure it | 20:20 |
| mordred | harlowja: https://review.openstack.org/#/c/92677/ | 20:21 |
| anteaya | clarkb: a two person army | 20:21 |
| *** kgiusti has left #openstack-infra | 20:21 | |
| pabelanger | clarkb: just converting to vhd now | 20:21 |
| clarkb | zomg this might actually pass and merge | 20:21 |
| *** superdan is now known as dansmith | 20:27 | |
| clarkb | I have approved https://review.openstack.org/#/c/367704/ which will stop hardcoding eth0 | 20:27 |
| clarkb | and am going to approve its child now | 20:27 |
| clarkb | we will want to watch neutron changes once zuul and jjb things update | 20:28 |
| openstackgerrit | Merged openstack-infra/statusbot: Explicitly catch Exception https://review.openstack.org/233781 | 20:28 |
| *** cardeois has joined #openstack-infra | 20:29 | |
| *** pabelanger is now known as pall | 20:29 | |
| *** ddieterly is now known as ddieterly[away] | 20:29 | |
| *** salv-orl_ has joined #openstack-infra | 20:30 | |
| *** ddieterly[away] is now known as ddieterly | 20:31 | |
| *** ddieterly has quit IRC | 20:31 | |
| *** xarses_ has joined #openstack-infra | 20:32 | |
| *** salv-orlando has quit IRC | 20:32 | |
| *** baoli has quit IRC | 20:33 | |
| *** cardeois has quit IRC | 20:33 | |
| *** salv-orl_ has quit IRC | 20:33 | |
| *** jkilpatr has quit IRC | 20:33 | |
| *** Guest5783 has quit IRC | 20:33 | |
| *** baoli has joined #openstack-infra | 20:33 | |
| *** xarses has quit IRC | 20:34 | |
| *** salv-orlando has joined #openstack-infra | 20:34 | |
| *** roxanagh_ has joined #openstack-infra | 20:35 | |
| *** mkoderer has joined #openstack-infra | 20:38 | |
| openstackgerrit | Merged openstack-infra/statusbot: Explicitly handle ImportError for pid_file_module https://review.openstack.org/233780 | 20:39 |
| *** lmiccini has joined #openstack-infra | 20:39 | |
| * clarkb makes note to contact ovh about bhs1 image uploads | 20:39 | |
| *** cgross has joined #openstack-infra | 20:40 | |
| *** gyee has joined #openstack-infra | 20:41 | |
| anteaya | 8 point buck in my dooryard | 20:41 |
| anteaya | he is so beautiful | 20:41 |
| openstackgerrit | Merged openstack-infra/project-config: Remove hardcoding to eth0 in LinuxBridge job configuration https://review.openstack.org/367704 | 20:43 |
| *** jcoufal has quit IRC | 20:43 | |
| openstackgerrit | Merged openstack-infra/project-config: Revert "Fix linuxbridge job breakage on xenial" https://review.openstack.org/368195 | 20:43 |
| *** askb has joined #openstack-infra | 20:44 | |
| *** rfolco_ has joined #openstack-infra | 20:45 | |
| *** rfolco has quit IRC | 20:46 | |
| *** mbacchi has joined #openstack-infra | 20:50 | |
| *** tonytan4ever has joined #openstack-infra | 20:50 | |
| *** fguillot has quit IRC | 20:51 | |
| *** raildo has quit IRC | 20:53 | |
| *** e0ne has joined #openstack-infra | 20:54 | |
| *** jkilpatr has joined #openstack-infra | 20:56 | |
| *** Mary has joined #openstack-infra | 20:56 | |
| *** bin_ has quit IRC | 20:57 | |
| *** Mary is now known as Guest46819 | 20:57 | |
| *** openstackstatus has quit IRC | 20:57 | |
| pall | Yay | 20:58 |
| pall | mordred: clarkb: ssh devuser@2001:4802:7801:102:be76:4eff:fe20:387b | 20:58 |
| pall | ssh devuser@162.242.226.100 | 20:58 |
| pall | that is rax-iad | 20:58 |
| *** openstackstatus has joined #openstack-infra | 20:58 | |
| *** ChanServ sets mode: +v openstackstatus | 20:58 | |
| clarkb | pall: and xenial? | 20:59 |
| pall | that is trusty | 20:59 |
| pall | I can do xenial now | 20:59 |
| clarkb | ah ok | 20:59 |
| clarkb | cool | 20:59 |
| clarkb | (should check since systemd has been special enough lately) | 20:59 |
| anteaya | pall: are you also pabelanger? | 20:59 |
| mordred | I was just about to ask the same thing | 20:59 |
| clarkb | anteaya: ya I had to unfilter my smartfilters to get that context | 21:00 |
| anteaya | clarkb: thanks | 21:00 |
| pall | Sorry, ya | 21:00 |
| *** pall is now known as pabelanger | 21:00 | |
| pabelanger | there we go | 21:00 |
| anteaya | my brother's name is paul | 21:00 |
| anteaya | that was his nick in high school | 21:00 |
| anteaya | thought he was in channel | 21:00 |
| clarkb | telnet://2001:4800:1ae1:18:f816:3eff:fedb:122e:19885 is a linuxbridge job on xenial that just started | 21:00 |
| * clarkb telenets | 21:00 | |
| *** Guest46819 has quit IRC | 21:01 | |
| *** baoli has quit IRC | 21:02 | |
| *** baoli has joined #openstack-infra | 21:02 | |
| *** _nadya_ has joined #openstack-infra | 21:03 | |
| *** lascii is now known as alaski | 21:04 | |
| *** mbacchi has quit IRC | 21:04 | |
| *** mbacchi has joined #openstack-infra | 21:04 | |
| *** ilyashakhat_mobi has joined #openstack-infra | 21:05 | |
| *** notnownikki has joined #openstack-infra | 21:07 | |
| *** baoli has quit IRC | 21:07 | |
| *** _nadya_ has quit IRC | 21:08 | |
| *** e0ne has quit IRC | 21:09 | |
| *** gyee has quit IRC | 21:12 | |
| *** notnownikki has quit IRC | 21:14 | |
| *** gyee has joined #openstack-infra | 21:15 | |
| *** inc0 has quit IRC | 21:16 | |
| fungi | somebody set us up the telenets | 21:20 |
| * mordred hands fungi a box of telenets he found | 21:21 | |
| *** ddieterly has joined #openstack-infra | 21:21 | |
| *** trown is now known as trown|outtypewww | 21:21 | |
| clarkb | I am a tyop machine | 21:21 |
| mordred | clarkb: you pretty much just computer on your phone now don't you? | 21:25 |
| pabelanger | clarkb: mordred: both xenial and trusty with the glean updates for rax-iad | 21:25 |
| pabelanger | ipv4 / ipv6 on eth0 | 21:26 |
| fungi | so to rehash the current broken with all our puppet-lint and beaker-rspec jobs, it looks like we're unable to install the puppet-lint gem due to dependency conflicts: http://logs.openstack.org/14/368114/2/check/gate-puppet-mediawiki-puppet-beaker-rspec-centos-7/010af66/console.html#_2016-09-09_21_08_29_268753 | 21:26 |
| *** thorst_ has quit IRC | 21:28 | |
| anteaya | :( | 21:29 |
| *** ddieterly has quit IRC | 21:29 | |
| anteaya | well the gem is there: https://rubygems.org/gems/puppet-lint | 21:29 |
| anteaya | what version is it after? | 21:30 |
| mordred | pabelanger: WOOT! | 21:30 |
| mordred | pabelanger: zomg. it's almost like we have working internet. in a CLOUD | 21:30 |
| *** HeOS has quit IRC | 21:31 | |
| *** xarses has joined #openstack-infra | 21:31 | |
| *** xarses_ has quit IRC | 21:31 | |
| *** xarses has quit IRC | 21:31 | |
| anteaya | fungi: I can't decipher what version it is looking for | 21:32 |
| *** mwhahaha has quit IRC | 21:32 | |
| *** xarses has joined #openstack-infra | 21:32 | |
| *** akshai has quit IRC | 21:32 | |
| anteaya | fungi: we may have specified a range of versions that can't be fulfilled | 21:32 |
| fungi | anteaya: yep, that's what i'm trying to suss out now | 21:33 |
| fungi | puppet-lint-absolute_template_path wants puppet-lint >=1.1,< 3.0 | 21:34 |
| *** abregman has joined #openstack-infra | 21:34 | |
| *** annegentle has quit IRC | 21:34 | |
| fungi | most of the others there want puppet-lint >=1.0,< 3.0 | 21:34 |
| fungi | except | 21:34 |
| *** mdrabe has quit IRC | 21:35 | |
| fungi | puppet-lint-unquoted_string-check wants puppet-lint (~> 1.0) | 21:35 |
| *** mdrabe has joined #openstack-infra | 21:35 | |
| anteaya | ah | 21:35 |
| anteaya | sigh | 21:35 |
| anteaya | that would make it impossible to fulfill | 21:35 |
| fungi | so my guess is we were previously getting puppet-lint 1.0 and then puppet-lint-absolute_template_path decided today that it needs at least puppet-lint 1.1 | 21:35 |
| anteaya | also https://github.com/puppetlabs/puppetlabs_spec_helper | 21:36 |
| anteaya | 1.2.2 wants 2.0 | 21:36 |
| anteaya | but my read is that change happened 2 weeks ago | 21:37 |
| fungi | yeah, i wonder if that's an artifact of a new puppetlabs_spec_helper release or a new puppet-lint-absolute_template_path release | 21:37 |
| anteaya | well the puppetlabs_spec_helper release was 2 weeks ago | 21:37 |
| *** rfolco_ has quit IRC | 21:38 | |
| anteaya | how long have we been affected by this? | 21:38 |
| fungi | just today | 21:38 |
| clarkb | ok I am now semi afk will keep an eye on those linuxbridge jobs though | 21:38 |
| fungi | puppet-lint-absolute_template_path 1.0.1 was released a couple of weeks ago as well, according to https://rubygems.org/gems/puppet-lint-absolute_template_path | 21:39 |
| anteaya | clarkb: thanks, have a good weekend | 21:39 |
| anteaya | wonder what happened today then | 21:39 |
| anteaya | I'm at a loss, but the gems definitely are fighting | 21:40 |
| anteaya | fungi: puppet-lint-unquoted_string-check is loaded twice | 21:42 |
| anteaya | first with puppet-lint (< 3.0, >= 1.0) | 21:43 |
| *** rtheis has quit IRC | 21:43 | |
| *** jamesdenton has quit IRC | 21:43 | |
| anteaya | then with puppet-lint (~> 1.0) | 21:43 |
| openstackgerrit | Elizabeth K. Joseph proposed openstack-infra/puppet-statusbot: Add Twitter support https://review.openstack.org/368238 | 21:43 |
| anteaya | sorry 7 times | 21:43 |
| anteaya | why is puppet-lint-unquoted_string-check loaded 7 times | 21:44 |
| *** drawsmcgraw has quit IRC | 21:44 | |
| fungi | i read that as it trying to resolve to a version which would be compatible with the other dependencies | 21:44 |
| *** igormarnat has quit IRC | 21:44 | |
| *** drawsmcgraw has joined #openstack-infra | 21:44 | |
| openstackgerrit | Elizabeth K. Joseph proposed openstack-infra/system-config: Add Twitter account details to our statusbot https://review.openstack.org/368239 | 21:44 |
| fungi | so, yeah, it's more likely puppetlabs_spec_helper that's at fault here | 21:44 |
| *** asilenkov has quit IRC | 21:44 | |
| *** rakhmerov has quit IRC | 21:44 | |
| pleia2 | mordred: those should get us there ^^ | 21:44 |
| fungi | i'm just not piecing together eth timeline, but am digging for a recent change which passed these jobs to compare | 21:45 |
| pleia2 | modulo actual reviews in case I missed something ;) | 21:45 |
| *** katyafervent_awa has quit IRC | 21:45 | |
| mordred | pleia2: woot! | 21:45 |
| mordred | pleia2: did you put the keys into the hieras already? | 21:45 |
| anteaya | fungi: cool | 21:45 |
| pleia2 | mordred: working on that now | 21:45 |
| mordred | woot | 21:45 |
| *** kzaitsev_ws has quit IRC | 21:45 | |
| fungi | anteaya: http://logs.openstack.org/64/366964/1/gate/gate-puppet-mediawiki-puppet-lint/ab41f88/console.html#_2016-09-08_19_12_58_121900 | 21:46 |
| *** gordc has quit IRC | 21:46 | |
| fungi | there's a working run from yesterday and the versions of things it resolved to install | 21:46 |
| *** mbacchi has quit IRC | 21:46 | |
| *** asilenkov has joined #openstack-infra | 21:46 | |
| fungi | "Installing puppetlabs_spec_helper 1.1.1 | 21:46 |
| fungi | " | 21:46 |
| pleia2 | g/ 114 | 21:47 |
| pleia2 | derp | 21:47 |
| *** kzaitsev_ws has joined #openstack-infra | 21:47 | |
| *** igormarnat has joined #openstack-infra | 21:47 | |
| *** rakhmerov has joined #openstack-infra | 21:47 | |
| anteaya | fungi: also puppet-lint-unquoted_string-check 0.3.0 | 21:48 |
| fungi | yep | 21:48 |
| *** katyafervent_awa has joined #openstack-infra | 21:49 | |
| fungi | which hasn't had a newer release according to https://rubygems.org/gems/puppet-lint-unquoted_string-check | 21:49 |
| clarkb | also we have success for linuxbridge ubuntu xenial | 21:49 |
| anteaya | yay success | 21:49 |
| anteaya | yeah so yesterday were had been 0.3.0 of unquoted_string-check and today we want an older version? | 21:51 |
| anteaya | 0.1.0 | 21:51 |
| anteaya | s/were had been/we had been using | 21:52 |
| *** mwhahaha has joined #openstack-infra | 21:52 | |
| anteaya | fungi: should we pin the puppetlabs_spec_helper version in our spec_helper? | 21:53 |
| *** dimtruck is now known as zz_dimtruck | 21:54 | |
| *** ilyashakhat_mobi has quit IRC | 21:54 | |
| fungi | well, we pinned the spec helper a few weeks ago https://review.openstack.org/359421 and then unpinned it the next day https://review.openstack.org/359539 | 21:55 |
| fungi | but that seems to have been related to changes in ruby interpreter version requirements | 21:55 |
| anteaya | I think we pin it again with a new comment | 21:56 |
| anteaya | since a new reason has reared its head | 21:56 |
| anteaya | unless there is a better way I am missing | 21:56 |
| kevinbenton | clarkb: was looking at Ihar's comment on https://review.openstack.org/#/c/367716/ . do you think i should try to parse the TUNNEL_RANGES config instead? | 21:56 |
| anteaya | moar kevinbenton, how can I be so lucky? | 21:57 |
| fungi | crinkle: nibalizer: no idea if either of you are around again yet, but trying to figure out why this just cropped up today http://logs.openstack.org/14/368114/2/check/gate-puppet-mediawiki-puppet-lint/a456a93/console.html#_2016-09-09_21_02_31_430459 (the release dates for the versions of things involved seem to be weeks old at best) | 21:57 |
| kevinbenton | anteaya: when people say that i can never tell if it's a good thing or a bad thing :) | 21:57 |
| anteaya | kevinbenton: it is nice to see you | 21:57 |
| anteaya | kevinbenton: we both know I know far too little to be lying in wait with some bug to surprise you with :) | 21:58 |
| clarkb | kevinbenton: I dont know that there is much need unless devstack decides to change it later to overlap | 21:58 |
| clarkb | not sure how often it would change | 21:58 |
| kevinbenton | anteaya: can never tell. Canadians are shifty characters ;) | 21:58 |
| anteaya | we are so | 21:58 |
| *** fguillot has joined #openstack-infra | 21:58 | |
| anteaya | have to keep both eyes on us :) | 21:58 |
| kevinbenton | clarkb: yeah, i don't expect it to change realistically | 21:59 |
| kevinbenton | clarkb: also it would be a non-trivial thing to parse correctly because it can be a comma-separated list of ranges | 21:59 |
| clarkb | why not start with ehat we have now then deal with parsing later | 21:59 |
| clarkb | this addresses the immediate concern/issue | 21:59 |
| fungi | anteaya: oh! look just above the gem installation on the broken vs working runs | 22:00 |
| * anteaya looks | 22:00 | |
| kevinbenton | clarkb: sounds good. just wanted to make sure nobody was waiting on parsing | 22:00 |
| fungi | anteaya: hint... https://rubygems.org/gems/bundler | 22:00 |
| anteaya | fungi: new bundler | 22:00 |
| clarkb | kevinbenton: I'm off to doctor viaits butaybe fungi mordred or pabelanger can review | 22:00 |
| anteaya | fungi: good eye | 22:00 |
| anteaya | clarkb phone is the best | 22:01 |
| kevinbenton | clarkb: ack. thanks again | 22:01 |
| openstackgerrit | Elizabeth K. Joseph proposed openstack-infra/system-config: Add Twitter account details to our statusbot https://review.openstack.org/368239 | 22:01 |
| fungi | crinkle: nibalizer: nevermind--looks like we're bitten by changes in dependency resolution brought on by today's bundler 1.13.0 release | 22:01 |
| clarkb | we did the thing where we had kids while living in city and now dont live in city but like their doctors so have to drive too far for these things | 22:02 |
| mordred | pleia2: I has question on that change | 22:03 |
| kevinbenton | clarkb: just take the company helicopter. that's a perk now, right? | 22:03 |
| *** mbacchi has joined #openstack-infra | 22:03 | |
| pleia2 | mordred: sure | 22:03 |
| anteaya | fungi: doesn't look like anyone has filed an issue against bundler 1.13 that looks like our situation | 22:03 |
| fungi | anteaya: agreed, i just finished skimming open and closed issues and pull requests for it | 22:04 |
| pleia2 | mordred: ah yeah, since true/false, on it | 22:04 |
| anteaya | good doctors are worth driving for, mind you I'm not in a van with twins in traffic | 22:04 |
| mordred | pleia2: looks like you did the right thing on the puppet module itself | 22:04 |
| * pleia2 nods | 22:05 | |
| openstackgerrit | Elizabeth K. Joseph proposed openstack-infra/system-config: Add Twitter account details to our statusbot https://review.openstack.org/368239 | 22:05 |
| *** tonytan_brb has joined #openstack-infra | 22:06 | |
| mordred | pleia2: woot \o/ | 22:07 |
| *** tonytan4ever has quit IRC | 22:07 | |
| pleia2 | I haven't paid attention, are we having rspec problems today? (failed job on the puppet patch) | 22:07 |
| fungi | EmilienM: yolanda: new details on the mysterious gem resolution issue from earlier today. looks like bundler 1.13.0 may have changed dependency resolver behavior. i'm working on pinning it to 1.12.5 for infra's puppet jobs until we can work out more details to file a bug upstream | 22:08 |
| pleia2 | how fortuitous, thanks fungi | 22:08 |
| fungi | indeed, good timing pleia2 ;) | 22:09 |
| *** mriedem has quit IRC | 22:11 | |
| openstackgerrit | sebastian marcet proposed openstack-infra/openstackid-resources: Notifications API https://review.openstack.org/368245 | 22:13 |
| *** yee379 has joined #openstack-infra | 22:16 | |
| *** vhosakot has joined #openstack-infra | 22:16 | |
| fungi | EmilienM: hrm, actually i don't think we can easily pin beaker just for infra's jobs... it looks like at some point we decided to have puppet-openstack, fuel and infra collectively use the same set of jobs? | 22:20 |
| anteaya | oh my | 22:20 |
| sdake | maybe i'm just dense - but did the post button in gerrit go away | 22:20 |
| mordred | sdake: what button? | 22:21 |
| fungi | i'm more inclined to stop running lint and rspec jobs temporarily on our puppet modules unless puppet-openstack and fuel agree we can pin beaker for all of us collectively | 22:21 |
| mordred | fungi: I'm fine with that | 22:21 |
| sdake | nm i'm dense | 22:21 |
| sdake | mordred its reply->post | 22:21 |
| mordred | sdake: :) | 22:21 |
| anteaya | yeah if you are not on the most current patchset the button disappears | 22:21 |
| sdake | and was showoing oddy because of my brokne laptop sren | 22:21 |
| sdake | anteaya that wasn't the problem but yes I am ware of that | 22:22 |
| anteaya | fungi: I'm also fine with that | 22:22 |
| anteaya | sdake: okay | 22:22 |
| sdake | anteaya operator erorr on my aprt :) | 22:22 |
| *** yamahata has quit IRC | 22:22 | |
| anteaya | okay | 22:22 |
| anteaya | fungi: I don't think our modules will get wildly out of control as a result of a few days without linters | 22:23 |
| *** inc0 has joined #openstack-infra | 22:24 | |
| fungi | on the other hand, agreeing that we'll all run a common set of jobs comes with the price that when those jobs break for one team's repos, the jobs may get adjusted for all | 22:25 |
| *** ddieterly has joined #openstack-infra | 22:25 | |
| anteaya | are those jobs not broken for them too? | 22:25 |
| fungi | apparently they aren't hitting this, or so said EmilienM earlier in channel (about 10 hours ago) | 22:26 |
| anteaya | I wonder how | 22:26 |
| fungi | it's possible their spec helper dependency set is not as strange as ours | 22:26 |
| fungi | we have separate spec helper repos, with different dependencies | 22:27 |
| *** ddieterly has quit IRC | 22:27 | |
| *** mdrabe has quit IRC | 22:27 | |
| anteaya | ah | 22:27 |
| anteaya | it is past 6pm on a Friday, at this point I am mostly nodding in agreement | 22:28 |
| *** thorst has joined #openstack-infra | 22:28 | |
| *** abregman has quit IRC | 22:28 | |
| anteaya | and standing by to review a patch indicating what you would like to do, should one materialize | 22:28 |
| *** ddieterly has joined #openstack-infra | 22:31 | |
| fungi | yeah, i'm working on putting together an upstream bug report first | 22:33 |
| anteaya | ah | 22:33 |
| *** rlandy has quit IRC | 22:33 | |
| *** thorst has quit IRC | 22:33 | |
| *** vhosakot has quit IRC | 22:35 | |
| *** xyang1 has quit IRC | 22:36 | |
| *** vhosakot has joined #openstack-infra | 22:36 | |
| mordred | fungi: yah - I think when we get acceptance from puppet and fuel teams on changing beaker - we should extra-verify the theory that if we need to pin something like that for one of the users of the jobs, we'll pin it for all of them | 22:37 |
| *** yamahata has joined #openstack-infra | 22:37 | |
| *** sdague has quit IRC | 22:38 | |
| fungi | er, i said pin beaker above. i obviously meant bundler | 22:38 |
| fungi | EmilienM: pin bundler, sorry, not beaker | 22:39 |
| *** sdake has quit IRC | 22:40 | |
| mordred | yah | 22:41 |
| mordred | and I also knew that and continued the game | 22:41 |
| *** vinaypotluri has quit IRC | 22:42 | |
| *** vhosakot has quit IRC | 22:42 | |
| *** zhurong has joined #openstack-infra | 22:46 | |
| mwhahaha | what's the bundler problem? | 22:46 |
| fungi | mwhahaha: 1.13.0 release today changed its dependency resolution behavior and can't install https://git.openstack.org/openstack-infra/puppet-openstack_infra_spec_helper now | 22:47 |
| fungi | i'm about done composing a bug report for them | 22:47 |
| mwhahaha | that's odd since we use a similar method | 22:49 |
| fungi | mwhahaha: i'm assuming it's something specific to the set of deps in ours | 22:50 |
| *** inc0 has quit IRC | 22:50 | |
| fungi | that tickles a behavior change in the resolver | 22:50 |
| mwhahaha | oh we do pin puppet-lint 1.1.0 | 22:50 |
| mwhahaha | and you don't | 22:50 |
| mwhahaha | trying to remember where we do this tho | 22:52 |
| fungi | https://github.com/bundler/bundler/issues/4961 | 22:53 |
| *** zhurong has quit IRC | 22:54 | |
| zigo | pabelanger: Jeremy & Monty already +2 that patch, can you workflow it now? https://review.openstack.org/367343 | 22:54 |
| fungi | mwhahaha: i wonder if it's related to https://github.com/bundler/bundler/issues/4960 which was also just opened moments ago | 22:54 |
| zigo | fungi: Or maybe you can? | 22:55 |
| pleia2 | zigo: the file is a bit picky about syntax, I recommend cleaning up the trailing whitespace on like 1820 | 22:55 |
| pleia2 | s/like/line | 22:56 |
| zigo | Ok. | 22:56 |
| *** ociuhandu has quit IRC | 22:56 | |
| fungi | oh, yep, gertty doesn't highlight trailing whitespace for me | 22:56 |
| *** ddieterly has quit IRC | 22:57 | |
| fungi | zigo: anyway, i'm happy to approve asap if you want to do one more that strips out excess trailing whitespace | 22:57 |
| zigo | Will this break something? | 22:57 |
| *** rbrndt has quit IRC | 22:57 | |
| zigo | fungi: Doing it right now | 22:57 |
| zigo | "most" puts trailing space in red, it's easy to see. | 22:58 |
| *** hongbin has quit IRC | 22:58 | |
| openstackgerrit | Thomas Goirand proposed openstack-infra/project-config: upstream: and options: track-upstream for deb-* https://review.openstack.org/367343 | 22:59 |
| zigo | fungi: There you go. | 22:59 |
| *** ddieterly has joined #openstack-infra | 22:59 | |
| zigo | fungi: Is this track-upstream like a cron job? How often does it run? | 22:59 |
| *** hasharAway has quit IRC | 23:00 | |
| openstackgerrit | Steve Martinelli proposed openstack-infra/project-config: fix osc gate: post_test_hook.sh not found https://review.openstack.org/368252 | 23:01 |
| zigo | Likely, my Add missing deb repo 4/5 and 5/5 will merge conflict after this patch is merged. | 23:01 |
| fungi | zigo: i was trying to work that out. i think we want to switch it to an independent cron job, since it looks like right now it runs as part of manage-projects so only when something in project-config for gerrit changes for at least one project (acl configs, project additions/removals, et cetera). that's pretty frequent at the moment, like ~daily at least, but i think we should probably make the | 23:01 |
| fungi | track-upstream updates happen hourly rather than on an ad-hoc basis | 23:01 |
| zigo | Ok. | 23:02 |
| openstackgerrit | Steve Martinelli proposed openstack-infra/project-config: fix osc gate: post_test_hook.sh not found https://review.openstack.org/368252 | 23:02 |
| stevemar | fungi: o/ | 23:02 |
| fungi | we've only been using it for one rarely-used repo so far, so haven't really been impacted by it yet | 23:02 |
| stevemar | fungi: anyway you could look at ^ i done broke the gate :( | 23:02 |
| openstackgerrit | Varun Gadiraju proposed openstack-infra/project-config: Fixes python 3 syntax issues with missing parentheses in call to 'print' https://review.openstack.org/368253 | 23:03 |
| zigo | fungi: Does it run in rackspace? | 23:03 |
| zigo | It'd be wise if it did, so it would be close to github. | 23:03 |
| fungi | zigo: it runs locally on review.openstack.org, which is currently hosted in rackspace's "dfw" region in texas, usa | 23:03 |
| zigo | Ok. | 23:03 |
| mwhahaha | fungi: i think it's puppet-lint-empty_string-check from puppet-mediawiki | 23:04 |
| *** _nadya_ has joined #openstack-infra | 23:04 | |
| mwhahaha | i tested against puppet-manilla and included that gem in the Gemfile and it puked | 23:05 |
| mwhahaha | w/o it and just the spec helper it was fine | 23:05 |
| * stevemar thanks fungi 1000 times | 23:05 | |
| *** ddieterly has quit IRC | 23:06 | |
| fungi | mwhahaha: oh, interesting. i wonder if we're not using the spec helper in that module yet? | 23:06 |
| mwhahaha | no you are | 23:06 |
| vgadiraj | Hi all, I am working on ironic and when I was running tox in py 3.5 jenkins-project was failing because a couple print statements were missing parentheses | 23:06 |
| mwhahaha | but that's an 'extra' one according to the comment | 23:06 |
| mwhahaha | let me debug further | 23:06 |
| fungi | ohhhh | 23:06 |
| vgadiraj | submitted a basic syntax fix | 23:07 |
| mwhahaha | https://rubygems.org/gems/puppet-lint-empty_string-check/versions/0.2.2 | 23:07 |
| mwhahaha | anyway so it's some sort of combination of bundler and that gem with the others :D | 23:08 |
| mwhahaha | but i would agree that it's a bundler problem, but if you pull that empty string check it seems to work just fine | 23:08 |
| fungi | mwhahaha: intresting, so if you just install https://git.openstack.org/openstack-infra/puppet-openstack_infra_spec_helper it works? | 23:08 |
| mwhahaha | yea i swapped out puppet-openstack_spec_helper with puppet-openstack_infra_spec_helper in puppet-manilla with bundler 1.13.0 and it was fine | 23:09 |
| fungi | unfortunately i'm not looking forward to needing to patch this across a bunch of our modules. that was more or less the promise of using a spec helper gem | 23:09 |
| mwhahaha | but as soon as i added that empty string check to the puppet-manilla Gemfile it blew up | 23:09 |
| mwhahaha | so it's probably related to the minimum version thing but in a round about way | 23:09 |
| *** _nadya_ has quit IRC | 23:09 | |
| *** fguillot has quit IRC | 23:10 | |
| fungi | mwhahaha: thanks, i've updated the description on https://github.com/bundler/bundler/issues/4961 accordingly | 23:11 |
| mwhahaha | or just pin puppetlabs_spec_helper to 1.1.1 | 23:11 |
| fungi | just confirmed, we have 71 puppet modules independently declaring a dependency on puppet-lint-empty_string-check | 23:13 |
| *** ddieterly has joined #openstack-infra | 23:13 | |
| mwhahaha | i'd pin spec_helper to 1..1.1 | 23:13 |
| mwhahaha | that'll solve it for sure | 23:13 |
| mwhahaha | the problem is that puppetlabs_spec_helper 1.2.x requires puppet-lint 2.0 (which in it's own is problematic) | 23:13 |
| fungi | giving it a shot now | 23:14 |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/puppet-openstack_infra_spec_helper: Pin puppetlabs_spec_helper < 1.2 https://review.openstack.org/368254 | 23:17 |
| fungi | mwhahaha: ^ like that? | 23:17 |
| openstackgerrit | Merged openstack-infra/project-config: upstream: and options: track-upstream for deb-* https://review.openstack.org/367343 | 23:18 |
| mwhahaha | '~> 1.1.1' | 23:18 |
| fungi | what's the difference between those? | 23:18 |
| mwhahaha | ~> 1.1.1 will track 1.1.x | 23:18 |
| mwhahaha | ~> 1.1 will track 1.x | 23:18 |
| fungi | weird | 23:18 |
| mwhahaha | rubyisms | 23:19 |
| fungi | yeah, i will readily admit i don't know the first thing about ruby | 23:19 |
| mwhahaha | < might work but i've usually used ~> or pinned to a specific version | 23:19 |
| fungi | so why ~> 1.1.1 and not ~> 1.1.0? | 23:19 |
| mwhahaha | could be either | 23:20 |
| mwhahaha | i just picked 1.1.1 cause that's the current known working version | 23:20 |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/puppet-openstack_infra_spec_helper: Pin puppetlabs_spec_helper ~> 1.1.1 https://review.openstack.org/368254 | 23:20 |
| fungi | thanks. updated ^ | 23:20 |
| *** armax has quit IRC | 23:20 | |
| fungi | i'll make a puppet-mediawiki change depends-on that and see if it passes jobs now | 23:21 |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/puppet-mediawiki: Skip CAPTCHA for autopatrol users https://review.openstack.org/368114 | 23:22 |
| fungi | that one ^ | 23:22 |
| mwhahaha | we shall see | 23:22 |
| *** sdake has joined #openstack-infra | 23:22 | |
| openstackgerrit | Thomas Goirand proposed openstack-infra/project-config: Fix add deb-nodepool https://review.openstack.org/368256 | 23:24 |
| zigo | fungi: This patch fixes what AJaeger found out (ie: nodepool was added, but missing from gerrit/projects.yaml_ | 23:24 |
| mwhahaha | boooo nope | 23:25 |
| mwhahaha | yea the next simpliest fix is to pin bundler to 1.12.5 | 23:26 |
| *** mrjk has quit IRC | 23:27 | |
| *** pradk has quit IRC | 23:27 | |
| openstackgerrit | Thomas Goirand proposed openstack-infra/project-config: Add missing deb repo 4/5 https://review.openstack.org/366734 | 23:27 |
| openstackgerrit | Merged openstack-infra/project-config: fix osc gate: post_test_hook.sh not found https://review.openstack.org/368252 | 23:27 |
| *** xarses has quit IRC | 23:28 | |
| openstackgerrit | Thomas Goirand proposed openstack-infra/project-config: Add missing deb repo 5/5 https://review.openstack.org/366960 | 23:31 |
| fungi | mwhahaha: that's what i was originally going to do, but that does mean changing jobs that are used not only by the infra team's modules but also openstack-puppet and fuel | 23:32 |
| mwhahaha | i happen to probably be able to speak for both of those :D | 23:32 |
| mwhahaha | i'm running a test to see if fuel-library is broken | 23:33 |
| *** zul has quit IRC | 23:33 | |
| *** roxanagh_ has quit IRC | 23:35 | |
| mwhahaha | fuel-library is not currently broken | 23:37 |
| mwhahaha | testing a puppet-openstack module | 23:37 |
| *** Swami has quit IRC | 23:39 | |
| mwhahaha | anyway it's time for me to wander off, i'd personally be ok with pinning bundler for now across the jobs but it is friday evening so maybe wait until monday? | 23:43 |
| mwhahaha | also who releases on a friday :D | 23:43 |
| *** armax has joined #openstack-infra | 23:43 | |
| *** kzaitsev_mb has quit IRC | 23:48 | |
| *** ddieterly has quit IRC | 23:54 | |
| *** mbacchi has quit IRC | 23:54 | |
| sdake | hey i suspect nobody is around | 23:55 |
| sdake | but i'm having a realy wierd gate problem | 23:55 |
| sdake | xhttp://logs.openstack.org/69/367269/16/check/gate-kolla-dsvm-deploy-centos-source-centos-7-nv/2ea3982/console.html#_2016-09-09_20_46_13_185316 | 23:55 |
| sdake | it seems like the ssh takes 3 minutes | 23:55 |
| *** fguillot has joined #openstack-infra | 23:58 | |
| *** gyee_ has joined #openstack-infra | 23:58 | |
| *** ddieterly has joined #openstack-infra | 23:59 | |
| *** ddieterly has quit IRC | 23:59 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!