| *** dkranz has joined #openstack-infra | 00:00 | |
| pleia2 | clarkb: hm, I'll refresh things on my end and see how it goes | 00:00 |
|---|---|---|
| *** gokrokve has quit IRC | 00:01 | |
| *** Krinkle|detached is now known as Krinkle | 00:03 | |
| *** ayoung has joined #openstack-infra | 00:03 | |
| pleia2 | clarkb: thanks, I'll have another go at it tomorrow (evening plans now) | 00:04 |
| clarkb | pleia2: thank you for double checking my documentation | 00:05 |
| clarkb | fungi: how does jjb updating go? | 00:06 |
| clarkb | we seem to be runing not many jobs | 00:07 |
| fungi | clarkb: it's still running | 00:07 |
| *** hemna is now known as hemnafk | 00:07 | |
| fungi | however, zuul is also basically caught up on its backlog now, per the job queue graph | 00:07 |
| *** claudiub has quit IRC | 00:08 | |
| *** gokrokve has joined #openstack-infra | 00:08 | |
| anteaya | yay | 00:11 |
| clarkb | fungi: maybe? looks like it needs to blast through the results queue then it will have a bunch more load | 00:11 |
| jeblair | fungi, clarkb: i think geard is straining a bit | 00:11 |
| clarkb | jeblair: ya | 00:11 |
| fungi | i can stop the jjb updates again | 00:11 |
| anteaya | jeblair: what are you looking at that tells you that, is it in graphite? | 00:12 |
| jeblair | anteaya: no, top. | 00:12 |
| anteaya | ah | 00:13 |
| jeblair | fungi: yeah, might be a good idea | 00:13 |
| anteaya | :( | 00:13 |
| clarkb | maybe leave one running? | 00:13 |
| jeblair | clarkb, fungi: yeah, maybe one at a time | 00:14 |
| jeblair | strace is all about CAN_DO | 00:14 |
| *** sputnik13 has quit IRC | 00:14 | |
| *** haypo has left #openstack-infra | 00:14 | |
| ianw | https://review.openstack.org/#/c/179689/ <- patchset 9 not picked up, been about 30 minutes, so i guess not caught up | 00:15 |
| *** mtanino has quit IRC | 00:15 | |
| ianw | although the test still won't work due to cffi , one of those days :) | 00:15 |
| clarkb | jeblair: anything you can do jenkins can do better? | 00:16 |
| clarkb | that should be a jenkins motto | 00:16 |
| *** achanda has quit IRC | 00:18 | |
| *** otter768 has joined #openstack-infra | 00:18 | |
| jeblair | (zuulv3 will not need each worker to register 7000 gearman functions) | 00:18 |
| *** dkranz has quit IRC | 00:18 | |
| *** achanda has joined #openstack-infra | 00:19 | |
| clarkb | you would basically say CAN_DO run_ansible right? | 00:19 |
| clarkb | then ansible can figure out what to do | 00:20 |
| jeblair | yeah, more or less | 00:21 |
| jeblair | fungi: did you stop jjb? | 00:22 |
| *** otter768 has quit IRC | 00:23 | |
| *** achanda has quit IRC | 00:23 | |
| clarkb | hrm we can probably update gearman-plugin to do additive CAN_DOs and only fallback on the unperformant RESET_ABILITIES then can do everything if we need to remove things | 00:24 |
| * clarkb checks if there is a CAN'T DO | 00:24 | |
| jeblair | clarkb: there is | 00:24 |
| clarkb | there is so we don't even need to fallback, just take a diff and do minimal updates | 00:24 |
| clarkb | I think the problem here is for every job that is updated we CAN_DO all the jobs | 00:24 |
| clarkb | so its an exponential update | 00:24 |
| sdague | so... gerrit event stream stopped again? | 00:24 |
| jeblair | clarkb: yep | 00:25 |
| clarkb | sdague: no, gearman is unable to keep up with job updates | 00:25 |
| sdague | because the fix changes pushed 15 minutes ago for the gate have not appeared in zuul yet | 00:25 |
| sdague | ok | 00:25 |
| *** rloo has quit IRC | 00:25 | |
| jeblair | er, that might also have happened | 00:25 |
| clarkb | gah | 00:25 |
| jeblair | http://paste.openstack.org/show/215173/ | 00:26 |
| jeblair | i think we learned something from my debug patch | 00:26 |
| clarkb | that one account has all the connections? | 00:27 |
| jeblair | yeah, all of the stuck threads are servicing that account | 00:27 |
| jeblair | and those times are the start times for when they switched to handling that connection | 00:28 |
| sdague | is that blockable? | 00:28 |
| jeblair | yeah, we can disable the account | 00:28 |
| sdague | until such time as they can get interactive to help figure out what they are doing that's causing this | 00:29 |
| jeblair | i believe that will prohibit stream-events | 00:29 |
| clarkb | jeblair: how do we know they are stuck? | 00:29 |
| clarkb | jeblair: we can also make a group without stream-events perms and put users in that group | 00:29 |
| clarkb | jeblair: either way should work | 00:29 |
| sdague | clarkb / fungi: you guys have the changes that are needed to unblock things, right? | 00:29 |
| clarkb | sdague: I am sure I can find them | 00:30 |
| sdague | so I can sign off and those can be shepharded | 00:30 |
| *** freerunner has quit IRC | 00:30 | |
| sdague | https://review.openstack.org/#/c/180341/ PS 3 | 00:30 |
| sdague | and it's depends on | 00:30 |
| clarkb | jeblair: looks like 02 and 07 are running a pair of jenkins-jobs each | 00:30 |
| jeblair | clarkb: because since april 30 is a long time to try to send a packet? | 00:30 |
| sdague | ok, night folks | 00:31 |
| jeblair | clarkb: i think fungi has afk'd; can you kill 3 of those | 00:31 |
| jeblair | ? | 00:31 |
| clarkb | jeblair: ya | 00:31 |
| jeblair | clarkb: you should run 'ssh review gerrit show-queue' | 00:32 |
| clarkb | oh actually its just one per, but two processes because sudo, killing on 07 | 00:32 |
| *** dims_ has joined #openstack-infra | 00:32 | |
| *** freerunner has joined #openstack-infra | 00:32 | |
| *** ildikov has quit IRC | 00:32 | |
| clarkb | ah and those are the active ones | 00:33 |
| clarkb | and have taken up all our threads? | 00:33 |
| *** tnovacik has quit IRC | 00:33 | |
| *** dims has quit IRC | 00:33 | |
| clarkb | jeblair: were you going to disable the account? I can do it if it helps | 00:34 |
| *** mriedem has quit IRC | 00:34 | |
| jeblair | clarkb: i'll do it | 00:35 |
| *** Sukhdev has quit IRC | 00:36 | |
| jeblair | clarkb: ready for me to restart gerrit? | 00:37 |
| clarkb | I think so, you want ti disable first otherwise the account stuff is cached and you ahve to wait for it to clear out or flush all caches | 00:38 |
| jeblair | i also have a thread dump | 00:38 |
| jeblair | yeah, should be disable now | 00:38 |
| clarkb | then I am ready | 00:39 |
| jeblair | clarkb: can you kill that last jjb run? | 00:39 |
| clarkb | I can | 00:39 |
| clarkb | done | 00:40 |
| fungi | i'm back, sorry. was otp | 00:41 |
| jeblair | #status notice Restarted gerrit due to stuck stream-events connections. Events since 23:49 were missed and changes uploaded since then will need to be rechecked. | 00:41 |
| openstackstatus | jeblair: sending notice | 00:41 |
| fungi | looks like you got the three jjb updates i had running killed off | 00:42 |
| clarkb | fungi: there was a third? | 00:42 |
| clarkb | fungi: I got 02 and 07 | 00:42 |
| -openstackstatus- NOTICE: Restarted gerrit due to stuck stream-events connections. Events since 23:49 were missed and changes uploaded since then will need to be rechecked. | 00:42 | |
| clarkb | and tcpdump on zuul seems to confirm that was it | 00:42 |
| clarkb | maybe the third one completed? | 00:42 |
| jeblair | geard is happy now | 00:43 |
| fungi | 04 was the other | 00:43 |
| fungi | seems it may have completed on its own | 00:43 |
| jeblair | fungi: can you start up one of those again and we'll see how it goes? | 00:43 |
| fungi | restarting now, yeah | 00:43 |
| clarkb | nothing running on 04 so it completed I think | 00:43 |
| fungi | going again on 07 | 00:43 |
| openstackstatus | jeblair: finished sending notice | 00:43 |
| jeblair | fungi: and we made progress on stream-events; see http://paste.openstack.org/show/215173/ | 00:44 |
| *** tiswanso has joined #openstack-infra | 00:45 | |
| fungi | jeblair: hrm... i saw that last time too. they have something like half a dozen different ci systems which use the same account (compare against show-connections and you'll see they're various ip addresses) | 00:45 |
| fungi | i asked around about it at the time | 00:45 |
| *** ildikov has joined #openstack-infra | 00:46 | |
| *** samueldmq has quit IRC | 00:47 | |
| *** tiswanso has quit IRC | 00:47 | |
| *** tiswanso has joined #openstack-infra | 00:48 | |
| *** Somay has quit IRC | 00:49 | |
| *** samueldmq has joined #openstack-infra | 00:50 | |
| jeblair | fungi: oh neat, this happened already? | 00:50 |
| *** Krinkle is now known as Krinkle|detached | 00:51 | |
| jeblair | fungi: so at least that's confirmation :) | 00:51 |
| fungi | yeah, you were travelling. i collected several fresh jstack traces with thread names properly resolved, and corresponding connections and queues outputs from gerrit | 00:51 |
| jeblair | fungi, clarkb: looks like geard can handle one full jjb run | 00:52 |
| clarkb | jeblair: thats good news | 00:52 |
| clarkb | I am playing with the start of a patch to basically do the CAN_DO CANT_DO diff | 00:52 |
| fungi | i still have them. unfortunately i didn't get much out of it other than confirming that there wasn't any immediately obvious correlation, and the method the stream events workers were blocking in matched the upstream bug | 00:52 |
| jeblair | so i think we can proceed with the jjb runs one at a time | 00:52 |
| clarkb | figure while the code is familiar to give it a go | 00:52 |
| fungi | jeblair: wfm, i can keep tabs on them for a while still and kick off the remaining ones in sequence after this one completes | 00:53 |
| anteaya | do you want me to get the gerrit id for netapp-ci? | 00:53 |
| jeblair | anteaya: no thanks, i already disabled it and sent an email | 00:54 |
| anteaya | jeblair: ah thank | 00:55 |
| anteaya | s | 00:55 |
| fungi | they apparently have different ci systems for a bunch of cinder drivers and share the same account between them rather than asking for separate accounts for them (from back when they still needed to request accounts) | 00:55 |
| anteaya | ah | 00:56 |
| anteaya | they have at least two different gerrit accounts | 00:56 |
| anteaya | NetApp FC CI and NetApp NAS CI | 00:57 |
| anteaya | which was the one that was disabled? | 00:57 |
| jeblair | anteaya: "NetApp CI" <xdl-openstack-jenkins@netapp.com> | 00:58 |
| *** freerunner has quit IRC | 00:58 | |
| anteaya | oh | 00:59 |
| jeblair | anteaya: "at lesat two" :) | 00:59 |
| anteaya | which my search in gerrit didnt' even turn up | 00:59 |
| anteaya | so I couldn't have found them anyway | 01:00 |
| anteaya | my experience with this group from the cinder threads is they drag their feet | 01:00 |
| anteaya | and aren't the most proactive | 01:00 |
| *** cody-somerville has quit IRC | 01:00 | |
| anteaya | cinder mailing list threads | 01:01 |
| dougwig | do you folks know the root cause of why they broke things, so the rest of us can avoid it? | 01:01 |
| anteaya | nope | 01:01 |
| *** patrickeast has quit IRC | 01:02 | |
| anteaya | just that all stuck threads were servicing them | 01:02 |
| *** gokrokve has quit IRC | 01:02 | |
| anteaya | hence the invitation for a chat | 01:02 |
| anteaya | hoping we can learn | 01:02 |
| anteaya | will share if we find anything | 01:02 |
| anteaya | thanks for asking | 01:02 |
| fungi | could be something as innocuous as they're all running some particular release of jenkins | 01:02 |
| dougwig | ok, i'll try to avoid launching 10 zuuls in the meantime. :) | 01:02 |
| anteaya | dougwig: thanks | 01:03 |
| fungi | er, of the jenkins gerrit-trigger plugin i mean | 01:03 |
| anteaya | yeah it might be some custom script they wrote | 01:04 |
| *** SumitNaiksatam has joined #openstack-infra | 01:04 | |
| *** Somay has joined #openstack-infra | 01:04 | |
| openstackgerrit | Darragh Bailey proposed openstack-infra/jenkins-job-builder: Separate out custom yaml tags into separate classes https://review.openstack.org/180370 | 01:05 |
| openstackgerrit | Darragh Bailey proposed openstack-infra/jenkins-job-builder: Support lazy resolving of include yaml tags https://review.openstack.org/63580 | 01:05 |
| *** ddieterly has joined #openstack-infra | 01:06 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/gearman-plugin: Register the diff of functions https://review.openstack.org/180371 | 01:06 |
| clarkb | jeblair: zaro ^ my initial testing says that works | 01:06 |
| clarkb | fungi: anteaya it could also be related to having one account hold many open sessions | 01:07 |
| clarkb | fungi: anteaya if perhaps there is some state that updates racily | 01:07 |
| clarkb | I need to pop out now, I think I have a handful of gearman-plugin changes now between mine and jeblairs to prep for inclusion on say 07 in the morning | 01:07 |
| *** weshay has quit IRC | 01:08 | |
| *** alexpilotti has quit IRC | 01:08 | |
| anteaya | clarkb: it is possible, yes | 01:09 |
| *** jtriley has quit IRC | 01:11 | |
| anteaya | I stand corrected | 01:12 |
| anteaya | akerr actually shows up to meetings: https://wiki.openstack.org/wiki/ThirdPartySystems/NetApp_CI | 01:12 |
| anteaya | and they are using zuul | 01:12 |
| anteaya | ameade also attends meetings | 01:12 |
| anteaya | hopefully they are responsive | 01:12 |
| ameade | im here! | 01:14 |
| ameade | anteaya: reading scrollback | 01:14 |
| *** shashankhegde has quit IRC | 01:16 | |
| ameade | weird there should only be one thing listening for that account, ill take a look | 01:17 |
| anteaya | ameade: thank you | 01:19 |
| anteaya | ameade: the other netapp account, netapp fc doesn't seem to be doing the same thing | 01:19 |
| *** zhiwei has joined #openstack-infra | 01:19 | |
| anteaya | ameade: what might be the difference between the two accounts? | 01:19 |
| ameade | kk interesting | 01:21 |
| anteaya | finding anything yet? | 01:21 |
| ameade | the two accounts represent two completely seperate systems | 01:21 |
| ameade | fibre channel has a unique set of challenges that our other system couldnt do, but we want to get it all back under one system soon | 01:21 |
| ameade | just grabbed my laptop | 01:22 |
| anteaya | thanks | 01:22 |
| anteaya | so the ci that was shut down is the non fibre channel one, yes? | 01:22 |
| *** alexsyip has quit IRC | 01:22 | |
| anteaya | I'm going with FC is fibre channel | 01:22 |
| anteaya | its a stretch I know | 01:22 |
| ameade | anteaya: lol correct | 01:23 |
| anteaya | so yay for figuring out fibre channel in a way that didn't bring down our gerrit | 01:23 |
| anteaya | now for the blocked system | 01:23 |
| anteaya | let's start with what tools you are running and versions, shall we? | 01:24 |
| anteaya | and any customizations you might have running | 01:24 |
| ameade | i image its zuul doing this command and that should be out of the box | 01:25 |
| anteaya | did you git clone zuul? | 01:25 |
| anteaya | do you recall the last time it was updated? | 01:25 |
| ameade | Zuul version: 2.0.0.225.g6d26658 | 01:26 |
| ameade | and darn we did clone it | 01:26 |
| ameade | it's been awhile then | 01:26 |
| anteaya | okay so we have that | 01:27 |
| anteaya | so zuul and what else do you have running? | 01:27 |
| ameade | Nodepool, Jenkins | 01:28 |
| *** Swami has quit IRC | 01:28 | |
| anteaya | how did you get nodepool, git again? | 01:28 |
| anteaya | and what version? | 01:28 |
| anteaya | or sha | 01:28 |
| ameade | we have a big fork of nodepool with custom logic | 01:29 |
| anteaya | let's dig into that a bit shall we? | 01:29 |
| *** alexpilotti has joined #openstack-infra | 01:29 | |
| anteaya | is this big fork of nodepool unique to this account? so the fc one is not running the same fork? | 01:29 |
| ameade | that's correct | 01:30 |
| anteaya | we have a big lead here then | 01:30 |
| anteaya | help me understand why you forked and what you needed the customization to do | 01:30 |
| anteaya | as long as I'm not crossing a proprietary boundary by asking | 01:30 |
| ameade | the customization just does bootstrapping of the netapp controllers and configures the nodes accordingly | 01:31 |
| Kennan | hi :clarkb | 01:31 |
| anteaya | have you any idea when this system with the customized nodepool came online? | 01:31 |
| anteaya | Kennan: he is out for a bit | 01:32 |
| Kennan | could I ask you one question ? | 01:32 |
| Kennan | :anteaya | 01:32 |
| anteaya | Kennan: ask away, he or someone else will do their best when they can | 01:32 |
| ameade | anteaya: oh it's been there from the start which has been at least 6 months | 01:32 |
| clarkb | Kennan: usually its best to just ask | 01:32 |
| Kennan | do you know how to make IRC show review link for kilo branch review? | 01:32 |
| Kennan | I sent review for kilo in our project | 01:33 |
| ameade | is the issue that ssh connections aren't going away? | 01:33 |
| anteaya | fungi clarkb jeblair do we have any idea when the stuck gerrit thread issue came to our attention? | 01:33 |
| Kennan | I did not find it have show in IRC chat | 01:33 |
| anteaya | ameade: correct | 01:33 |
| clarkb | anteaya: ya, jeblair's status announcement included the time it went under | 01:33 |
| Kennan | but seems master can work | 01:33 |
| anteaya | ameade: so threads get stuck, gerrit stops streaming events, zuul doesn't do anything because it can't hear events that aren't broadcast | 01:33 |
| *** gokrokve has joined #openstack-infra | 01:34 | |
| Kennan | :anteaya do you know how to make openstack show other branch code review link in IRC ? | 01:34 |
| anteaya | clarkb: sorry, like how many weeks/months this has been an issue | 01:34 |
| clarkb | Kennan: you need to make sure that https://git.openstack.org/cgit/openstack-infra/project-config/tree/gerritbot/channels.yaml specifies stable/kilo under your channel | 01:34 |
| clarkb | anteaya: oh that, I think it started when we upgraded to trusty | 01:34 |
| ameade | anteaya: kk, that makes sense at least...no idea why we would have zombie connections though..doing some digging | 01:34 |
| clarkb | anteaya: so that was march ya? | 01:34 |
| anteaya | ameade: thanks | 01:34 |
| Kennan | :clarkb OK thanks. Let me check that | 01:34 |
| anteaya | clarkb: sounds about right, I can go back to the announcements and look | 01:35 |
| anteaya | Saturday, March 21, 2015 | 01:35 |
| *** ddieterly has quit IRC | 01:41 | |
| *** spzala has quit IRC | 01:41 | |
| *** ddieterly has joined #openstack-infra | 01:43 | |
| *** ddieterly has quit IRC | 01:43 | |
| *** derekh_ has joined #openstack-infra | 01:46 | |
| *** signed8bit_ZZZzz is now known as signed8b_ | 01:47 | |
| *** signed8b_ is now known as signed8bit_ZZZzz | 01:47 | |
| ameade | anteaya: pretty sure i just figured it out | 01:48 |
| *** alexpilotti has quit IRC | 01:48 | |
| ameade | anteaya: do you want the long or the short? and how do I prove it to you? lol | 01:48 |
| anteaya | ameade: give it to me anyway you like | 01:49 |
| * anteaya listens | 01:49 | |
| *** derekh has quit IRC | 01:50 | |
| ameade | hehe, i had a python script for when I was setting up the FC system a month or so ago that listened to the event stream which i had running with the other accounts creds that has a code path that doesnt close the connection | 01:50 |
| anteaya | ameade: can we ensure that script doens't run anymore? | 01:51 |
| *** r-daneel has quit IRC | 01:52 | |
| ameade | anteaya: yeah definitely | 01:53 |
| anteaya | thank you | 01:53 |
| anteaya | after that can you reply to http://lists.openstack.org/pipermail/third-party-announce/2015-May/000177.html | 01:53 |
| ameade | anteaya: really sorry for the firedrill | 01:53 |
| anteaya | saying what you did, so others don't do the same | 01:53 |
| anteaya | and saying you won't do it again | 01:54 |
| anteaya | thanks for showing up so quickly and helping to track it down | 01:54 |
| anteaya | thank you | 01:54 |
| ameade | anteaya: sure thing, I appreciate the help | 01:54 |
| anteaya | it will be a relief for us to know the source | 01:54 |
| anteaya | ameade: great, glad we got to the bottom of it | 01:54 |
| anteaya | when you post in reply to the -announce thread you will get a reply saying an admin has to let it through | 01:57 |
| anteaya | that's me and I'm waiting | 01:57 |
| openstackgerrit | Ian Wienand proposed openstack-infra/devstack-gate: [WIP] run centos7 with neutron https://review.openstack.org/179689 | 01:57 |
| ameade | anteaya: kk | 01:58 |
| *** salv-orlando has joined #openstack-infra | 01:58 | |
| * ameade makes haste | 01:58 | |
| anteaya | thanks | 01:58 |
| *** ddieterly has joined #openstack-infra | 01:58 | |
| anteaya | I didn't mean it like that | 01:58 |
| *** ddieterly has quit IRC | 01:58 | |
| anteaya | just saying that I'm ready to let the post through | 01:58 |
| anteaya | not that you have to hurry | 01:58 |
| *** jtriley has joined #openstack-infra | 01:59 | |
| openstackgerrit | Huang Rui proposed openstack-infra/project-config: Move zvm-driver to stackforge-attic https://review.openstack.org/179738 | 02:00 |
| *** ajmiller has joined #openstack-infra | 02:01 | |
| *** ivar-lazzaro has quit IRC | 02:03 | |
| ameade | anteaya: sent :) | 02:03 |
| anteaya | thank you | 02:03 |
| *** salv-orlando has quit IRC | 02:04 | |
| anteaya | you sent to the third-party-announce mailing list? | 02:04 |
| *** ivar-lazzaro has joined #openstack-infra | 02:04 | |
| ameade | anteaya: oh crap i only replied, one sec | 02:05 |
| anteaya | you sent to jim | 02:05 |
| anteaya | thanks | 02:05 |
| ameade | anteaya: oh i did a reply all so it should have went to that ML | 02:06 |
| anteaya | interesting | 02:06 |
| anteaya | maybe the mail server takes a minute or so | 02:06 |
| anteaya | I don't yet see it in my admin page | 02:07 |
| anteaya | so in the mean time | 02:07 |
| anteaya | the script is stopped? | 02:07 |
| ameade | anteaya: yeah | 02:09 |
| anteaya | great thank you | 02:10 |
| anteaya | jeblair: do we need anything more from ameade before the account can be re-enabled? | 02:10 |
| *** fedexo has joined #openstack-infra | 02:10 | |
| anteaya | ameade: he may be offline for the night | 02:10 |
| *** signed8bit_ZZZzz has quit IRC | 02:11 | |
| anteaya | ameade: check back tomorrow and we will see where we are, thanks again for your quick response | 02:11 |
| anteaya | really appreciate it | 02:11 |
| ameade | anteaya: that sounds good, thanks | 02:11 |
| anteaya | dougwig: don't write a script that keeps connections open :) | 02:11 |
| anteaya | ameade: thank you | 02:12 |
| anteaya | dougwig: and if you write it, don't run it | 02:12 |
| dougwig | ok, writing it right now. | 02:12 |
| ameade | dougwig: at least not for any stretch of time lol | 02:12 |
| dougwig | :) | 02:12 |
| anteaya | too much fun for the day | 02:13 |
| anteaya | I'm off for the night | 02:13 |
| anteaya | enjoy your local time of day and see you tomorrow | 02:13 |
| dougwig | night | 02:13 |
| anteaya | night | 02:13 |
| *** ddieterly has joined #openstack-infra | 02:13 | |
| *** unicell1 has quit IRC | 02:16 | |
| *** ddieterl_ has joined #openstack-infra | 02:16 | |
| *** ddieterly has quit IRC | 02:17 | |
| *** shashankhegde has joined #openstack-infra | 02:17 | |
| fungi | jenkins07 jjb update has finished. 04 running now | 02:18 |
| *** sigmavirus24 is now known as sigmavirus24_awa | 02:19 | |
| *** ivar-laz_ has joined #openstack-infra | 02:21 | |
| *** yamahata has quit IRC | 02:22 | |
| frickler | so the gate pipeline is still wedged? lots of reviews with all checks finished, but no progress | 02:23 |
| *** mestery has joined #openstack-infra | 02:23 | |
| *** dims_ has quit IRC | 02:24 | |
| ameade | fungi: were you talking about the netapp-ci earlier when you mentioned 'half a dozen systems'? we only have two and i wanna make sure that's clear | 02:24 |
| *** ivar-lazzaro has quit IRC | 02:24 | |
| *** otter768 has joined #openstack-infra | 02:25 | |
| *** gokrokve_ has joined #openstack-infra | 02:29 | |
| *** gokrokve_ has quit IRC | 02:31 | |
| *** gokrokve_ has joined #openstack-infra | 02:31 | |
| *** mwagner_lap has joined #openstack-infra | 02:32 | |
| *** gokrokve has quit IRC | 02:33 | |
| *** jtriley has quit IRC | 02:34 | |
| *** changbl has joined #openstack-infra | 02:35 | |
| fungi | ameade: yeah, one of the other cinder devs had suggested that, but was apparently also mistaken. thanks for the clarification | 02:39 |
| ameade | fungi: also the issue should be fixed so sorry if yall chased your tails | 02:40 |
| *** tkelsey has joined #openstack-infra | 02:40 | |
| *** achanda has joined #openstack-infra | 02:40 | |
| *** baoli has joined #openstack-infra | 02:42 | |
| *** tkelsey has quit IRC | 02:45 | |
| *** otter768 has quit IRC | 02:46 | |
| *** Somay has quit IRC | 02:47 | |
| *** otter768 has joined #openstack-infra | 02:47 | |
| *** nelsnelson has joined #openstack-infra | 02:47 | |
| fungi | thanks again ameade! | 02:49 |
| openstackgerrit | Merged openstack/requirements: Add python-tuskarclient to requirements https://review.openstack.org/174233 | 02:49 |
| *** zz_dimtruck is now known as dimtruck | 02:49 | |
| fungi | jenkins04 wrapped, so 02 (last one) is going now | 02:50 |
| *** nelsnelson has quit IRC | 02:52 | |
| *** gokrokve_ has quit IRC | 02:52 | |
| openstackgerrit | Samuel de Medeiros Queiroz proposed openstack-infra/devstack-gate: Remove arbitrary devstack config options https://review.openstack.org/179982 | 02:52 |
| *** ivar-laz_ has quit IRC | 02:53 | |
| *** gokrokve has joined #openstack-infra | 02:59 | |
| *** sputnik13 has joined #openstack-infra | 03:04 | |
| *** sarob has joined #openstack-infra | 03:06 | |
| *** markvoelker has quit IRC | 03:10 | |
| *** fifieldt has joined #openstack-infra | 03:11 | |
| *** yamahata has joined #openstack-infra | 03:12 | |
| openstackgerrit | Merged openstack/requirements: Add python-manilaclient to global requirements. https://review.openstack.org/135007 | 03:15 |
| openstackgerrit | Merged openstack/requirements: Add new angular deps to global-requirements.txt for Horizon https://review.openstack.org/151186 | 03:15 |
| openstackgerrit | Merged openstack/requirements: add pycparser tracking to what-broke https://review.openstack.org/176120 | 03:15 |
| *** otter768 has quit IRC | 03:16 | |
| openstackgerrit | Merged openstack-infra/project-config: Add sahara tempest job to sahara and tempest https://review.openstack.org/167780 | 03:18 |
| *** ivar-lazzaro has joined #openstack-infra | 03:19 | |
| *** dimtruck is now known as zz_dimtruck | 03:20 | |
| *** fifieldt has quit IRC | 03:20 | |
| fungi | i'm passing out, but i'll reenable puppet agent on the jenkins masters tomorrow when i wake up | 03:22 |
| *** dims has joined #openstack-infra | 03:24 | |
| *** dims has quit IRC | 03:30 | |
| *** jtriley has joined #openstack-infra | 03:31 | |
| *** fedexo has quit IRC | 03:34 | |
| *** jtriley has quit IRC | 03:36 | |
| *** tiswanso has quit IRC | 03:37 | |
| *** krtaylor has quit IRC | 03:37 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/devstack-gate: Remove arbitrary devstack config options https://review.openstack.org/179982 | 03:38 |
| openstackgerrit | Ian Wienand proposed openstack-infra/devstack-gate: [WIP] run centos7 with neutron https://review.openstack.org/179689 | 03:40 |
| *** gokrokve has quit IRC | 03:40 | |
| *** gokrokve has joined #openstack-infra | 03:41 | |
| *** links has joined #openstack-infra | 03:41 | |
| *** Mmike has quit IRC | 03:42 | |
| *** frickler has quit IRC | 03:42 | |
| openstackgerrit | Merged openstack-infra/project-config: Puppet: don't run beaker jobs on previous stable branches https://review.openstack.org/180233 | 03:43 |
| *** ddieterl_ has quit IRC | 03:44 | |
| *** Kennan2 has joined #openstack-infra | 03:45 | |
| *** Kennan has quit IRC | 03:46 | |
| *** shashankhegde has quit IRC | 03:50 | |
| *** baoli has quit IRC | 03:50 | |
| *** fedexo has joined #openstack-infra | 03:50 | |
| *** sdake has joined #openstack-infra | 03:51 | |
| *** camunoz has quit IRC | 03:53 | |
| *** coolsvap|afk is now known as coolsvap | 03:53 | |
| *** patrickeast has joined #openstack-infra | 03:54 | |
| *** camunoz has joined #openstack-infra | 03:54 | |
| *** krtaylor has joined #openstack-infra | 03:54 | |
| openstackgerrit | Merged openstack/requirements: Update python-cinderclient to 1.2.0 https://review.openstack.org/177270 | 03:56 |
| openstackgerrit | Merged openstack/requirements: Add contextlib2 lib to aid in py2/py3 compatability https://review.openstack.org/164222 | 03:56 |
| *** sdake_ has joined #openstack-infra | 03:56 | |
| *** Mmike has joined #openstack-infra | 03:57 | |
| *** Mmike has quit IRC | 03:57 | |
| *** Mmike has joined #openstack-infra | 03:57 | |
| *** sdake has quit IRC | 04:00 | |
| sdake_ | jeblair thanks :) | 04:00 |
| *** sdake has joined #openstack-infra | 04:01 | |
| *** adrian_otto has joined #openstack-infra | 04:03 | |
| *** sdake__ has joined #openstack-infra | 04:03 | |
| adrian_otto | I created a branch in gerrit that I want to remove, but I'm not sure how. Any ideas? | 04:03 |
| *** sdake_ has quit IRC | 04:04 | |
| *** pal has joined #openstack-infra | 04:06 | |
| *** sdake has quit IRC | 04:07 | |
| *** yfried|afk is now known as yfried | 04:08 | |
| *** changbl has quit IRC | 04:10 | |
| *** markvoelker has joined #openstack-infra | 04:11 | |
| *** sdake__ is now known as sdake | 04:13 | |
| *** gokrokve has quit IRC | 04:15 | |
| *** markvoelker has quit IRC | 04:16 | |
| *** cfriesen has joined #openstack-infra | 04:17 | |
| *** cfriesen has left #openstack-infra | 04:18 | |
| *** gokrokve has joined #openstack-infra | 04:18 | |
| openstackgerrit | Merged openstack-dev/pbr: Issue #1451976: handle commits with non-utf8 text https://review.openstack.org/180265 | 04:20 |
| openstackgerrit | Adrian Otto proposed openstack-infra/project-config: Add delete to ACL rights for magnum https://review.openstack.org/180401 | 04:21 |
| *** bhunter71 has quit IRC | 04:25 | |
| *** sks has joined #openstack-infra | 04:27 | |
| *** mgedmin has quit IRC | 04:30 | |
| *** mgedmin has joined #openstack-infra | 04:31 | |
| *** adrian_otto has quit IRC | 04:32 | |
| *** salv-orlando has joined #openstack-infra | 04:35 | |
| *** salv-orlando has quit IRC | 04:38 | |
| *** ddieterly has joined #openstack-infra | 04:45 | |
| *** abregman has joined #openstack-infra | 04:48 | |
| *** ddieterly has quit IRC | 04:50 | |
| *** btully has joined #openstack-infra | 04:50 | |
| *** mrmartin has joined #openstack-infra | 04:50 | |
| *** patrickeast has quit IRC | 04:56 | |
| *** sarob has quit IRC | 04:56 | |
| *** gokrokve_ has joined #openstack-infra | 05:01 | |
| *** adrian_otto has joined #openstack-infra | 05:03 | |
| *** camunoz has quit IRC | 05:03 | |
| *** harlowja_at_home has joined #openstack-infra | 05:03 | |
| *** gokrokve has quit IRC | 05:03 | |
| *** yfried has quit IRC | 05:04 | |
| *** samueldmq has quit IRC | 05:05 | |
| *** gokrokve_ has quit IRC | 05:05 | |
| *** btully has quit IRC | 05:07 | |
| *** ivar-lazzaro has quit IRC | 05:08 | |
| *** ivar-lazzaro has joined #openstack-infra | 05:08 | |
| *** Swami has joined #openstack-infra | 05:11 | |
| *** hdd has joined #openstack-infra | 05:11 | |
| *** ivar-lazzaro has quit IRC | 05:11 | |
| *** markvoelker has joined #openstack-infra | 05:12 | |
| *** ivar-lazzaro has joined #openstack-infra | 05:12 | |
| *** deepakcs has joined #openstack-infra | 05:15 | |
| *** camunoz has joined #openstack-infra | 05:15 | |
| *** markvoelker has quit IRC | 05:16 | |
| *** crc32 has quit IRC | 05:17 | |
| *** mrmartin has quit IRC | 05:17 | |
| *** otter768 has joined #openstack-infra | 05:17 | |
| *** SumitNaiksatam has quit IRC | 05:19 | |
| *** jtriley has joined #openstack-infra | 05:20 | |
| *** SumitNaiksatam has joined #openstack-infra | 05:20 | |
| *** otter768 has quit IRC | 05:22 | |
| *** sdake_ has joined #openstack-infra | 05:23 | |
| *** ildikov has quit IRC | 05:25 | |
| *** jtriley has quit IRC | 05:26 | |
| *** sdake has quit IRC | 05:27 | |
| *** BharatK has joined #openstack-infra | 05:27 | |
| *** shashankhegde has joined #openstack-infra | 05:32 | |
| *** subscope_ has joined #openstack-infra | 05:35 | |
| *** ildikov has joined #openstack-infra | 05:38 | |
| AJaeger | good morning | 05:39 |
| openstackgerrit | Sergey Kolekonov proposed openstack-infra/project-config: Add apps-catalog-ci group for apps-catalog project https://review.openstack.org/180201 | 05:45 |
| *** Somay has joined #openstack-infra | 05:45 | |
| *** adrian_otto has quit IRC | 05:45 | |
| *** gokrokve has joined #openstack-infra | 05:49 | |
| *** niska has quit IRC | 05:49 | |
| *** markvoelker has joined #openstack-infra | 05:51 | |
| *** frickler has joined #openstack-infra | 05:52 | |
| *** markvoelker has quit IRC | 05:55 | |
| *** rlandy has joined #openstack-infra | 05:57 | |
| deepakcs | AJaeger, morning, saw ur comment in https://review.openstack.org/#/c/173408/ - looks like using the new parameterised job template to skip tests is the reccomended apporach | 05:57 |
| *** SumitNaiksatam has quit IRC | 05:57 | |
| *** Somay has quit IRC | 05:57 | |
| deepakcs | AJaeger, thanks for helping out! | 05:58 |
| AJaeger | deepakcs, hope the others agree - that'S what we discsused last night here at least ;) | 05:58 |
| AJaeger | deepakcs, you're welcome | 05:58 |
| deepakcs | AJaeger, lets hope :) this issue has seen many approaches/iterations till now | 05:58 |
| AJaeger | ;) | 05:58 |
| *** Somay has joined #openstack-infra | 05:58 | |
| AJaeger | clarkb, fungi, jeblair : loquacities released last night (around 3:00 UTC) version 0.26 of openstack-doc-tools but this one does not show up on pypi. What needs to be done to get it up there? | 06:00 |
| AJaeger | cinerama, StevenK, pleia2 Hi! | 06:00 |
| *** SumitNaiksatam has joined #openstack-infra | 06:01 | |
| cinerama | greetings | 06:01 |
| StevenK | AJaeger: Hai! | 06:01 |
| AJaeger | pleia2, are you around? | 06:02 |
| *** subscope_ has quit IRC | 06:02 | |
| *** niska has joined #openstack-infra | 06:03 | |
| *** Swami has quit IRC | 06:03 | |
| AJaeger | is there any way to ping pleia2 or shall we start without her? | 06:03 |
| cinerama | hmm | 06:03 |
| *** fedexo has quit IRC | 06:04 | |
| *** masayukig_ has quit IRC | 06:04 | |
| cinerama | maybe let's go ahead and get started | 06:04 |
| cinerama | bit late here | 06:04 |
| cinerama | so i should probably go over our status | 06:05 |
| AJaeger | cinerama, I have no idea how far are you and where my input is needed, so please go ahead ;) | 06:05 |
| *** Longgeek has joined #openstack-infra | 06:05 | |
| *** isviridov_away is now known as isviridov | 06:05 | |
| *** camunoz has quit IRC | 06:05 | |
| cinerama | so right now we have translate-dev.openstack.org up and running for us to play around with a bit. so all the puppetizing, etc on the server side is done modulo any hiccups we discover later | 06:06 |
| cinerama | and we have puppetizing for installing the client on the proposal slaves | 06:06 |
| *** jcoufal has joined #openstack-infra | 06:06 | |
| cinerama | so the next big piece is the proposal scripts | 06:06 |
| cinerama | which i have a draft of almost ready to land | 06:06 |
| StevenK | \o/ | 06:06 |
| AJaeger | great! | 06:07 |
| cinerama | oh by the way, you can have a look at notes and stuff here: https://etherpad.openstack.org/p/zanata-install | 06:07 |
| cinerama | the proposal scripts are going to need some love from someone who has a good grasp of our process on this...i'm coming mostly from the zanata side | 06:07 |
| AJaeger | cinerama, that looks involved ;( | 06:08 |
| cinerama | in the true spirit of software development i've thrown away more than i've actually written | 06:08 |
| *** masayukig_ has joined #openstack-infra | 06:08 | |
| cinerama | s/written/intend to actually ship/ | 06:08 |
| StevenK | I have been digging into matching up locales that we already have in OpenStack projects to the list of what languages are up on translate-dev: http://paste.openstack.org/show/215326/ | 06:08 |
| cinerama | oh cool | 06:08 |
| AJaeger | I should have a good grasp on the infra side but not sure what I would be signing up for ;) | 06:08 |
| StevenK | I've read through the scripts, I think I get the push side | 06:08 |
| *** mrunge has joined #openstack-infra | 06:08 | |
| cinerama | StevenK: looks like i should add some manual mappings as per your findings | 06:08 |
| cinerama | so there | 06:09 |
| StevenK | But I think we also need a script that pre-populates based on what we have | 06:09 |
| cinerama | s a list of bugs in that etherpad, which i will add to tomorrow | 06:09 |
| AJaeger | cinerama, did you look at all langauges in *transifix*? There are 70+ languages in use | 06:09 |
| StevenK | Yeah. translate-dev has 35 | 06:09 |
| AJaeger | sorry, StevenK | 06:09 |
| StevenK | *34 | 06:09 |
| cinerama | StevenK: prepopulates the zanata server? | 06:09 |
| StevenK | cinerama: So this whole mapping thing is just silly, IMO | 06:10 |
| *** salv-orlando has joined #openstack-infra | 06:10 | |
| AJaeger | StevenK, so, we need to get all 70+ languages out from transifex and into zanata | 06:10 |
| cinerama | StevenK: oh the mapping from zanata locale to locale we're using? | 06:10 |
| StevenK | Other way round, but yes | 06:10 |
| AJaeger | cinerama, we can rename languages also in the projects, let's make this a *one time* mapping | 06:10 |
| StevenK | I think we should fix zanata | 06:10 |
| StevenK | I can generate a list of languages to add and ones to remove | 06:11 |
| cinerama | so one of the things my stuff does is to generate the zanata.xml for the project on the fly. part of that file can store the mappings between their locales and ours so we don't need to change | 06:11 |
| StevenK | And then feed that to cinerama or pleia2 | 06:11 |
| AJaeger | StevenK, whatever works but I don't want to have a mapping as part of pushing/pulling every time. | 06:11 |
| cinerama | it seems like the way they want the zanata.xml to be is to have it just live in the project but it is okay to generate it since we can pull down most of it from the server | 06:11 |
| StevenK | zanata can do the mapping as part of the push/pull, but I'd prefer the locale directories match what is in zanata | 06:11 |
| AJaeger | cinerama, if that mapping is easier, it's fine - but IMO we shouldn't need it | 06:11 |
| cinerama | i don't have an opinion on whether we should make the projects change or whether we handle it | 06:12 |
| pleia2 | apologies, had a meeting elsewhere that ran late | 06:12 |
| StevenK | We have some projects that don't match -- as it were. Some use tr and others use tr_TR, for example | 06:12 |
| AJaeger | The use case for a generated zanata.xml is that a nwe language gets added on the server, it needs to be added to the project without manual adding. | 06:12 |
| cinerama | the current issue with handling those mappings in zanata is that the pull will also create the directory with the zanata locale name; imo this is a bug, which i have filed with them | 06:13 |
| AJaeger | pleia2, glad to have you with us! | 06:13 |
| *** scheuran has joined #openstack-infra | 06:13 | |
| cinerama | AJaeger: that case you mention *should* work - if the language is in the project, it will get pulled down | 06:13 |
| StevenK | cinerama: Which is my point about fixing the language list on zanata | 06:13 |
| AJaeger | cinerama, great. | 06:13 |
| *** camunoz has joined #openstack-infra | 06:13 | |
| *** HeOS has quit IRC | 06:14 | |
| cinerama | i haven't gotten a response yet about my bug w/the pull but i just filed it this afternoon | 06:14 |
| AJaeger | cinerama, StevenK, if we have any mismatches between locales, let's clean those up when there's no reason for it. | 06:14 |
| *** tnovacik has joined #openstack-infra | 06:14 | |
| StevenK | AJaeger: When is there no reason for them? :-) | 06:14 |
| AJaeger | I'm fine with pushing a change to 20 projects renaming language files from tr to tr_TR if needed to make our tooling easier ;) | 06:14 |
| *** ivar-laz_ has joined #openstack-infra | 06:15 | |
| cinerama | so other than that - one of the big issues was with the minimum percentages. AJaeger alerted me to the fact that we can just use the method for cleaning up files with unacceptable percentages using the method we have now to clean up po files | 06:15 |
| *** ddieterly has joined #openstack-infra | 06:15 | |
| cinerama | ugh that's a bad sentence, but hopefully you know what i mean | 06:15 |
| *** ivar-lazzaro has quit IRC | 06:15 | |
| AJaeger | StevenK, we have en_US and en_GB since those are different, and if we have tr and tr_TR because they are different, it's fine. | 06:15 |
| AJaeger | But if tr is actually tr_TR, let's rename | 06:15 |
| AJaeger | StevenK, like we shouldn't rename en_AU to en_US ;) | 06:16 |
| StevenK | AJaeger: Absolutely | 06:16 |
| cinerama | anyway the zanata folks have marked the percentages bug as high priority. i'll grab the link for a sec because i would like someone who knows transifex better to respond to them | 06:16 |
| StevenK | For example, zanata has es-mx, and we use es_MX. I'd prefer that we just add es_MX to zanata | 06:17 |
| cinerama | https://bugzilla.redhat.com/show_bug.cgi?id=1215274 | 06:17 |
| openstack | bugzilla.redhat.com bug 1215274 in Component-zanata-client "Should be able to specify minimum percentage completion on pull" [High,New] - Assigned to djansen | 06:17 |
| cinerama | this bug | 06:17 |
| cinerama | StevenK: one of the biggest mapping issues is zh_* iirc | 06:17 |
| StevenK | zh_* isn't in zanata at all | 06:17 |
| pleia2 | cinerama: ah thanks, I was just looking for it in my bug mail | 06:18 |
| AJaeger | cinerama, StevenK: We can ask Daisy from the translation team for help, especially with zh_* | 06:18 |
| StevenK | Those two look fine, it's just zh_TW and zh_CN, there is no bare zh | 06:18 |
| cinerama | oh that's right, they're all zh-Hans-* | 06:19 |
| *** pal has quit IRC | 06:19 | |
| cinerama | that is the weirdness | 06:19 |
| cinerama | or at least that is what i get when i search | 06:19 |
| AJaeger | please ask Daisy for these | 06:19 |
| cinerama | so more mapping | 06:19 |
| cinerama | um anyway | 06:19 |
| *** pal has joined #openstack-infra | 06:19 | |
| cinerama | so we should be able to basically do what we need to but there is going to be some weirdness | 06:20 |
| pleia2 | added the mapping bug to the etherpad | 06:20 |
| *** Ala has joined #openstack-infra | 06:20 | |
| *** pal has joined #openstack-infra | 06:20 | |
| *** ddieterly has quit IRC | 06:20 | |
| cinerama | i kind of got on a long side quest about calculating percentages per file and then pulling down each individual file | 06:20 |
| cinerama | but i don't think we're going to use that | 06:20 |
| StevenK | cinerama: Searching https://translate-dev.openstack.org/language/list for zh gives me nothing | 06:21 |
| *** Longgeek has quit IRC | 06:21 | |
| AJaeger | Regarding the bug: WE can either do the following: | 06:21 |
| *** Longgeek has joined #openstack-infra | 06:21 | |
| AJaeger | 1) Do everything on our side as post-processing and close that bug as not-needed-anymore | 06:21 |
| cinerama | StevenK: weird. when i go to the "add new language" thingy i get zh-Hans-CN zh-Hans-HK etc | 06:21 |
| StevenK | I have no add new language, but I'm no admin. | 06:22 |
| AJaeger | 2) Do it on project-level and do post-processing for openstack-manuals since that's the only odd one | 06:22 |
| AJaeger | 3) Do it on document level like transifex does today | 06:22 |
| cinerama | AJaeger: so i think it is in their interests as well as ours to have feature parity w/transifex | 06:22 |
| cinerama | AJaeger: oh so with transifex it is per document? | 06:22 |
| AJaeger | cinerama, with transifex it'S per document | 06:22 |
| *** Longgeek has quit IRC | 06:23 | |
| cinerama | AJaeger: that was what i was asking about earlier :) | 06:23 |
| *** Longgeek has joined #openstack-infra | 06:23 | |
| AJaeger | Let me write up "feature parity with transifex" in this part and send you as email. | 06:23 |
| AJaeger | That will give us the most flexibility. | 06:23 |
| AJaeger | Could one of you then add to redhat bugzilla, please? I don't have an account there currently... | 06:23 |
| AJaeger | But if more discussion is needed, I can create one... | 06:24 |
| *** camunoz has quit IRC | 06:24 | |
| *** yfried has joined #openstack-infra | 06:24 | |
| StevenK | Hmmm, I think my RedHat bugzilla account is dead because it was @canonical.com, but that will require archeology | 06:25 |
| cinerama | so to do that ourselves, we have to use the rest api to grab the translations and then assemble the po files. the python client they have, while not currently maintained (i think?) has an example of how to do this | 06:25 |
| pleia2 | AJaeger: please do, you need an account to comment on bugs too, which we've had to do some as they seek more information | 06:25 |
| pleia2 | and zanata upstream is very committed to prioritizing our bugs, so make sure we add them to the bug section of the etherpad so we can keep track and bring them to their attention | 06:26 |
| AJaeger | pleia2, let me still send around my answer for review. | 06:26 |
| *** Somay has quit IRC | 06:26 | |
| AJaeger | pleia2, ok | 06:26 |
| *** gokrokve_ has joined #openstack-infra | 06:27 | |
| *** ildikov has quit IRC | 06:27 | |
| pleia2 | it's several emails a day, but I also subscribed to https://www.redhat.com/mailman/listinfo/zanata-bugs to keep an eye on things (both our bugs and other things that come along which may impact us) | 06:28 |
| *** gokrokve has quit IRC | 06:28 | |
| cinerama | so aside from the zanata.xml generation, a lot of the stuff for the proposal scripts could be in flux depending on timelines for the bugs getting resolved vs when it's desirable to deploy For Real(tm) | 06:28 |
| AJaeger | we can start anytime with *pushing* to zanata like we push to transifex | 06:29 |
| StevenK | Yup | 06:29 |
| AJaeger | That will show us whether that part works fine | 06:29 |
| StevenK | And we should do that first | 06:29 |
| cinerama | agreed, the push changes very little | 06:29 |
| StevenK | Except for the whole missing languages thing | 06:29 |
| cinerama | StevenK: feel free to try your shiny new admin powers | 06:29 |
| StevenK | GASP | 06:30 |
| StevenK | When did I get those? | 06:30 |
| AJaeger | And then I suggest to pick a single project and move that over to zanata | 06:30 |
| cinerama | StevenK: heh heh | 06:30 |
| AJaeger | ONce that's done, do the rest... | 06:30 |
| cinerama | AJaeger: that sounds like a pretty sensible way of going about it | 06:30 |
| AJaeger | All of that needs to be properly scheduled, so we need some "project plan" for it. | 06:31 |
| cinerama | well, we should be able to push as soon as we get the push script adapted, at least to the dev instance | 06:31 |
| *** Somay has joined #openstack-infra | 06:31 | |
| StevenK | "Underscores should be replaced with dashes." | 06:31 |
| AJaeger | Horizon and openstack-manuals are the projects with most translations. So, we should ask Daisy and translation team whether we start with Horizon/openstack-manuals, or with a project with far fewer strings like nova | 06:31 |
| StevenK | Ho hum | 06:31 |
| *** camunoz has joined #openstack-infra | 06:32 | |
| AJaeger | StevenK, that would be a simple mapping, replace dashes with underscores when pushing/pulling | 06:32 |
| StevenK | Right, so Zanata doesn't like underscores | 06:32 |
| cinerama | AJaeger: so i had been using nova for my personal tests, but in a way a complicated project will turn up more bugs | 06:32 |
| StevenK | AJaeger: We can't do that currently, it isn't quite that simple. | 06:32 |
| AJaeger | openstack-manuals is the most complicated ;) | 06:33 |
| *** harlowja_at_home has quit IRC | 06:33 | |
| cinerama | i am generating simple mappings like that for the zanata.xml now | 06:33 |
| AJaeger | StevenK, that's sad | 06:33 |
| StevenK | zh-cn would work, since we can transform zh_CN by .lower() and then replacing _ with - | 06:33 |
| *** dhp has quit IRC | 06:33 | |
| StevenK | But there isn't a consistency in the current list either | 06:34 |
| StevenK | You have en-gb and en-AU | 06:34 |
| StevenK | My CDO is twitching | 06:34 |
| *** dhp has joined #openstack-infra | 06:34 | |
| pleia2 | heh | 06:34 |
| *** notnownikki has joined #openstack-infra | 06:34 | |
| StevenK | Can I declare lower case with dashes and bend zanata to my will? | 06:35 |
| openstackgerrit | Kai Qiang Wu(Kennan) proposed openstack-infra/project-config: Add new branch for channels work https://review.openstack.org/180416 | 06:35 |
| *** ivar-lazzaro has joined #openstack-infra | 06:35 | |
| cinerama | they look like this | 06:35 |
| cinerama | <locale map-from="en_AU">en-AU</locale> | 06:35 |
| *** mgedmin has quit IRC | 06:36 | |
| cinerama | so the mapping can be pretty arbitrary | 06:36 |
| StevenK | cinerama: Yup, but we need to store that information somewhere. | 06:36 |
| *** wayneeseguin has quit IRC | 06:36 | |
| AJaeger | cinerama, can't we change this to en-au in zanata? WE should be able to enforce some conventions that might make life easier | 06:36 |
| StevenK | Exactly my point | 06:36 |
| *** jyuso has joined #openstack-infra | 06:36 | |
| *** ivar-laz_ has quit IRC | 06:36 | |
| cinerama | StevenK: the arbitrary mappings? i agree. i had been generating simple ones | 06:36 |
| cinerama | StevenK: but some, like the zh ones, are less obvious | 06:37 |
| cinerama | StevenK: we could maintain a list of the arbitrary mappings for the xml generation to rely on | 06:37 |
| cinerama | or of course we can change the projects | 06:37 |
| *** wayneeseguin has joined #openstack-infra | 06:38 | |
| cinerama | oh WOW | 06:38 |
| cinerama | i can add en-gb *and* en-GB | 06:38 |
| *** alex_xu has quit IRC | 06:38 | |
| cinerama | not sure that is a good idea | 06:39 |
| StevenK | cinerama: http://i.stack.imgur.com/jiFfM.jpg | 06:39 |
| cinerama | StevenK: actual LOL | 06:39 |
| *** jyuso1 has quit IRC | 06:39 | |
| AJaeger | :) | 06:40 |
| *** wayneeseguin has quit IRC | 06:40 | |
| *** sc` has quit IRC | 06:40 | |
| cinerama | so...that's worth filing a bug about i think | 06:40 |
| StevenK | Agreed | 06:40 |
| *** sc` has joined #openstack-infra | 06:40 | |
| StevenK | Huh, you can't remove languages? | 06:41 |
| cinerama | so it sounds like we need to 0) sort out the locales mess enough that we're happy 1) get push going 2) start with 1 project ... | 06:41 |
| cinerama | StevenK: aaaaand there's another bug to file | 06:41 |
| *** tteggel has quit IRC | 06:41 | |
| *** bkero has quit IRC | 06:41 | |
| StevenK | Because Slovenian is 'sl-' | 06:41 |
| pleia2 | cinerama: yep, that's how I see it | 06:41 |
| cinerama | StevenK: yeah i saw that, that's what it suggested | 06:41 |
| *** mgedmin has joined #openstack-infra | 06:41 | |
| *** yamahata has quit IRC | 06:41 | |
| *** tteggel has joined #openstack-infra | 06:41 | |
| StevenK | Sad face | 06:41 |
| cinerama | they are going to need to sort out some of this stuff on the locales end | 06:42 |
| cinerama | StevenK: you could do it in the database trololo | 06:42 |
| *** alex_xu has joined #openstack-infra | 06:42 | |
| StevenK | Mapping it back to GNU, sl_ makes NO SENSE | 06:42 |
| cinerama | so i added that but i didn't do it, i swear | 06:42 |
| cinerama | that is what it suggested | 06:42 |
| cinerama | in the little drop down thing | 06:42 |
| *** bkero has joined #openstack-infra | 06:42 | |
| cinerama | bug #3 | 06:43 |
| openstack | bug 3 in Launchpad itself "Custom information for each translation team" [Low,Fix released] https://launchpad.net/bugs/3 | 06:43 |
| cinerama | ssh no | 06:43 |
| StevenK | sl or sl_SL (normalized to sl-sl) | 06:43 |
| StevenK | Haha | 06:43 |
| *** tkelsey has joined #openstack-infra | 06:43 | |
| *** masayukig_ has quit IRC | 06:44 | |
| cinerama | so do we each have some "action items"? | 06:44 |
| * AJaeger has the AI to answer the Red Hat bug | 06:45 | |
| cinerama | me: put up xml generation & pull stuff | 06:45 |
| pleia2 | once we have the new bugs added to the etherpad, I'll email carlos to make sure his team is in the loop with our latest collection ;) | 06:45 |
| StevenK | cinerama: I think I will continue to look at locales | 06:46 |
| AJaeger | could somebody write up the next steps in the etherpad? | 06:46 |
| StevenK | Come up with a list to add and a list to remove when we can | 06:46 |
| cinerama | right and StevenK will look at locales | 06:46 |
| cinerama | pleia2: would you like to update/file bugs? | 06:47 |
| *** tkelsey has quit IRC | 06:47 | |
| pleia2 | cinerama: I won't get to it until tomorrow, but I can give it a go tomorrow | 06:48 |
| cinerama | they are updating our bugs as we speak which is very cool :) | 06:48 |
| *** masayukig_ has joined #openstack-infra | 06:48 | |
| pleia2 | I'll let you all know if I need more info for filing them | 06:48 |
| AJaeger | great! | 06:48 |
| *** wayneeseguin has joined #openstack-infra | 06:48 | |
| *** yamahata has joined #openstack-infra | 06:48 | |
| cinerama | pleia2: cool. i just want someone less sleepy to write it :) | 06:48 |
| *** viktors|afk is now known as viktors | 06:48 | |
| cinerama | does anyone have anything else? | 06:48 |
| cinerama | pleia2: that is probably going to wind up being one of Future Us | 06:49 |
| *** jcoufal_ has joined #openstack-infra | 06:49 | |
| pleia2 | cinerama: hehe, indeed | 06:50 |
| cinerama | this reminds me of the seinfeld thing about Day Guy vs Night Guy | 06:50 |
| cinerama | where Night Guy thinks it's a good idea to stay up late and watch a movie or whatever, resulting in Day Guy cursing him yet again | 06:51 |
| *** soren has joined #openstack-infra | 06:51 | |
| pleia2 | sounds about right | 06:51 |
| AJaeger | ;) | 06:52 |
| AJaeger | cinerama, pleia2: Thanks for staying up late for this discussion! | 06:52 |
| *** markvoelker has joined #openstack-infra | 06:52 | |
| cinerama | anyway meeting adjourned, thanks everyone for their time | 06:52 |
| * flaper87 drops this for AJaeger and pleia2 https://review.openstack.org/#/c/180333/ | 06:52 | |
| *** jcoufal has quit IRC | 06:52 | |
| *** wayneeseguin has quit IRC | 06:53 | |
| pleia2 | thanks all | 06:53 |
| AJaeger | Thanks everybody, have a great night or day! | 06:53 |
| deepakcs | AJaeger, good night | 06:53 |
| AJaeger | fungi, clarkb, jeblair: None of the daily proposal jobs run so far, they should have started 45 mins ago and systems seem idle | 06:53 |
| AJaeger | deepakcs, thanks - but it's 8am for me ;) | 06:53 |
| pleia2 | flaper87: alas, too sleepy, if no one else gets to it I'll try to have a look in my morning :) | 06:53 |
| *** asselin__ has quit IRC | 06:53 | |
| deepakcs | AJaeger, ah , i thought its night | 06:54 |
| *** shardy_z is now known as shardy | 06:54 | |
| *** asselin__ has joined #openstack-infra | 06:54 | |
| AJaeger | flaper87, I'll check - had my morning green tea already ;) | 06:54 |
| flaper87 | pleia2: ah no worries, have a great night :) | 06:54 |
| flaper87 | they are easy patches | 06:54 |
| flaper87 | at least the first one | 06:54 |
| flaper87 | AJaeger: thank YOU!!!!! | 06:54 |
| *** hdd has quit IRC | 06:56 | |
| *** markvoelker has quit IRC | 06:56 | |
| StevenK | AJaeger, cinerama, pleia2: Thanks! | 06:57 |
| AJaeger | Yeah, the first one is easy, the second one needs more time ;) | 06:57 |
| *** GheRivero_ has joined #openstack-infra | 07:02 | |
| AJaeger | Thanks, StevenK ! | 07:02 |
| flaper87 | AJaeger: replied :) | 07:02 |
| AJaeger | ;) | 07:03 |
| AJaeger | flaper87, +2 ed ;) | 07:03 |
| flaper87 | AJaeger: thank you! | 07:04 |
| *** hdd has joined #openstack-infra | 07:06 | |
| *** achanda has quit IRC | 07:07 | |
| *** mpavone has joined #openstack-infra | 07:08 | |
| waynr | i have successfully generated jenkins xml using pure python classes to produce the necessary data structure for the XmlBuilder in JJB | 07:08 |
| * waynr dances | 07:08 | |
| * AJaeger applaudes waynr | 07:08 | |
| *** achanda has joined #openstack-infra | 07:11 | |
| waynr | i've also made tons of progress in making various JJB modules and classes usefully useable as a library | 07:11 |
| *** sabeen has quit IRC | 07:11 | |
| *** achanda has quit IRC | 07:11 | |
| waynr | and now time for bed | 07:11 |
| *** shashankhegde has quit IRC | 07:13 | |
| *** GheRivero_ has quit IRC | 07:13 | |
| *** frickler_ has joined #openstack-infra | 07:14 | |
| *** frickler_ has quit IRC | 07:14 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Optionally pass regex to skip to tempest-dsvm-full-plugin jobs https://review.openstack.org/180309 | 07:15 |
| AJaeger | sleep well, waynr ! | 07:16 |
| *** ddieterly has joined #openstack-infra | 07:16 | |
| *** ildikov has joined #openstack-infra | 07:16 | |
| *** ddieterly has quit IRC | 07:20 | |
| *** woodster_ has quit IRC | 07:20 | |
| *** pal has quit IRC | 07:22 | |
| *** dizquierdo has joined #openstack-infra | 07:24 | |
| *** markus_z has joined #openstack-infra | 07:27 | |
| *** gsagie has joined #openstack-infra | 07:28 | |
| openstackgerrit | Julien Danjou proposed openstack/requirements: Bump pbr to 0.11 https://review.openstack.org/180423 | 07:30 |
| *** sergsh_ has joined #openstack-infra | 07:31 | |
| *** sergsh has joined #openstack-infra | 07:31 | |
| *** samuelBartel has joined #openstack-infra | 07:34 | |
| *** panda|Zz is now known as panda|bfast | 07:38 | |
| *** _nadya_ has joined #openstack-infra | 07:41 | |
| *** hdd_ has joined #openstack-infra | 07:45 | |
| *** jlanoux has joined #openstack-infra | 07:46 | |
| *** hdd has quit IRC | 07:46 | |
| *** panda|bfast is now known as panda | 07:47 | |
| *** panda is now known as Guest56018 | 07:47 | |
| *** Guest56018 is now known as panda | 07:48 | |
| *** HeOS has joined #openstack-infra | 07:49 | |
| *** gsagie has quit IRC | 07:51 | |
| AJaeger | flaper87, could you import translations again, please? https://review.openstack.org/#/c/176633/ and https://review.openstack.org/#/c/179630/ are open | 07:51 |
| *** hdd_ has quit IRC | 07:57 | |
| *** jistr has joined #openstack-infra | 07:59 | |
| *** hdd has joined #openstack-infra | 07:59 | |
| *** hasharMeeting is now known as hashar | 08:01 | |
| hashar | jeblair: thank you for the nodepool and gear tags! | 08:01 |
| *** Somay has joined #openstack-infra | 08:03 | |
| ajo | hi :) | 08:05 |
| *** hdd has quit IRC | 08:05 | |
| ajo | I have two accounts in gerrit, one with an old email (which I used for 10 days..) and one with my current email (2 years) | 08:05 |
| *** hdd has joined #openstack-infra | 08:05 | |
| ajo | is it possible to delete the old one? | 08:06 |
| ajo | I have people sending me review requests on the old one | 08:06 |
| *** dizquierdo has quit IRC | 08:06 | |
| *** ociuhandu has joined #openstack-infra | 08:09 | |
| *** _nadya_ has quit IRC | 08:12 | |
| *** hdd has quit IRC | 08:13 | |
| *** hdd has joined #openstack-infra | 08:13 | |
| *** yfried is now known as yfried|afk | 08:15 | |
| openstackgerrit | Jerome Hourquebie proposed stackforge/python-jenkins: Adds Cloudbees folder plugin support https://review.openstack.org/180185 | 08:16 |
| *** ddieterly has joined #openstack-infra | 08:17 | |
| *** Hal has joined #openstack-infra | 08:17 | |
| *** Hal is now known as Guest55056 | 08:17 | |
| *** salv-orlando has quit IRC | 08:18 | |
| *** salv-orlando has joined #openstack-infra | 08:18 | |
| *** yfried|afk is now known as yfried | 08:20 | |
| *** jtomasek has joined #openstack-infra | 08:20 | |
| *** ddieterly has quit IRC | 08:21 | |
| jtomasek | hey, can aynone please help me with openstack/tuskar-ui gate? the tests won't get run at all https://review.openstack.org/#/c/180211/ | 08:22 |
| *** e0ne has joined #openstack-infra | 08:22 | |
| *** ivar-lazzaro has quit IRC | 08:23 | |
| *** sputnik13 has quit IRC | 08:26 | |
| *** sputnik13 has joined #openstack-infra | 08:29 | |
| *** ssam2 has joined #openstack-infra | 08:31 | |
| *** dguitarbite has joined #openstack-infra | 08:32 | |
| *** MaxV has joined #openstack-infra | 08:34 | |
| openstackgerrit | Andrew Smith proposed openstack-infra/jenkins-job-builder: Add support for docker builder plugin https://review.openstack.org/178980 | 08:35 |
| *** _nadya_ has joined #openstack-infra | 08:36 | |
| *** ildikov has quit IRC | 08:41 | |
| *** e0ne has quit IRC | 08:43 | |
| *** Somay has quit IRC | 08:44 | |
| *** dbite has joined #openstack-infra | 08:44 | |
| *** tkelsey has joined #openstack-infra | 08:44 | |
| *** sputnik13 has quit IRC | 08:46 | |
| *** sputnik13 has joined #openstack-infra | 08:48 | |
| openstackgerrit | Davide Guerri proposed openstack-infra/shade: Add floating IP pool resource methods https://review.openstack.org/180255 | 08:51 |
| openstackgerrit | Davide Guerri proposed openstack-infra/shade: Add port resource methods https://review.openstack.org/179968 | 08:51 |
| openstackgerrit | Davide Guerri proposed openstack-infra/shade: Add keystone service resource methods https://review.openstack.org/177621 | 08:51 |
| *** MaxV has quit IRC | 08:52 | |
| openstackgerrit | Davide Guerri proposed openstack-infra/shade: Add keystone endpoint resource methods https://review.openstack.org/180328 | 08:52 |
| *** sputnik13 has quit IRC | 08:53 | |
| *** ildikov has joined #openstack-infra | 08:56 | |
| *** jtriley has joined #openstack-infra | 08:57 | |
| *** jtomasek has quit IRC | 08:58 | |
| *** smccully has quit IRC | 08:58 | |
| *** russellb has quit IRC | 08:58 | |
| *** trey has quit IRC | 08:58 | |
| *** gema has quit IRC | 08:58 | |
| *** adreznec has quit IRC | 08:58 | |
| *** mfink- has quit IRC | 08:58 | |
| *** SotK has quit IRC | 08:58 | |
| *** tsufiev has quit IRC | 08:58 | |
| *** mrda has quit IRC | 08:58 | |
| *** ianw has quit IRC | 08:58 | |
| *** shardy_ has joined #openstack-infra | 08:59 | |
| *** shardy has quit IRC | 09:00 | |
| *** fhubik has joined #openstack-infra | 09:00 | |
| *** jtomasek has joined #openstack-infra | 09:00 | |
| *** smccully has joined #openstack-infra | 09:00 | |
| *** russellb has joined #openstack-infra | 09:00 | |
| *** trey has joined #openstack-infra | 09:00 | |
| *** gema has joined #openstack-infra | 09:00 | |
| *** adreznec has joined #openstack-infra | 09:00 | |
| *** mfink- has joined #openstack-infra | 09:00 | |
| *** SotK has joined #openstack-infra | 09:00 | |
| *** tsufiev has joined #openstack-infra | 09:00 | |
| *** mrda has joined #openstack-infra | 09:00 | |
| *** ianw has joined #openstack-infra | 09:00 | |
| *** fhubik has quit IRC | 09:00 | |
| *** fhubik has joined #openstack-infra | 09:00 | |
| *** jtriley has quit IRC | 09:02 | |
| *** shardy_ has quit IRC | 09:04 | |
| *** shardy has joined #openstack-infra | 09:05 | |
| *** freerunner has joined #openstack-infra | 09:06 | |
| *** _nadya_ has quit IRC | 09:10 | |
| *** redixin has joined #openstack-infra | 09:13 | |
| *** cdent has joined #openstack-infra | 09:16 | |
| *** ddieterly has joined #openstack-infra | 09:17 | |
| *** otter768 has joined #openstack-infra | 09:19 | |
| *** freerunner has quit IRC | 09:20 | |
| openstackgerrit | Davide Guerri proposed openstack-infra/shade: Add floating IP pool resource methods https://review.openstack.org/180255 | 09:20 |
| *** ddieterly has quit IRC | 09:22 | |
| *** otter768 has quit IRC | 09:23 | |
| *** e0ne has joined #openstack-infra | 09:25 | |
| *** pblaho has joined #openstack-infra | 09:25 | |
| *** hdd_ has joined #openstack-infra | 09:27 | |
| *** freerunner has joined #openstack-infra | 09:28 | |
| *** hdd has quit IRC | 09:28 | |
| ams__ | Hi, I just wanted to check if there's anything else I need to do on this review - https://review.openstack.org/#/c/178980/ I see the +1 from Yolanda. Not entirely clear if I have to do anything else at this point. | 09:34 |
| yolanda | ams_, well, i'm not core there, so it's just an opinion | 09:35 |
| yolanda | you need to wait for core reviews | 09:35 |
| *** redixin has left #openstack-infra | 09:36 | |
| *** fhubik is now known as fhubik_afk | 09:41 | |
| *** _nadya_ has joined #openstack-infra | 09:41 | |
| *** fhubik_afk is now known as fhubik | 09:41 | |
| *** fhubik is now known as fhubik_afk | 09:42 | |
| ams__ | yolanda: ah ok, so I just have to wait. Cool, thanks! | 09:43 |
| yolanda | yes, it needs more feedback from other people | 09:43 |
| *** zhiwei has quit IRC | 09:45 | |
| *** tnovacik has quit IRC | 09:49 | |
| *** macjack has joined #openstack-infra | 09:51 | |
| *** Longgeek has quit IRC | 09:53 | |
| openstackgerrit | David Caro proposed openstack-infra/jenkins-job-builder: Added parallelization options https://review.openstack.org/180463 | 09:54 |
| *** fhubik_afk is now known as fhubik | 09:56 | |
| *** e0ne is now known as e0ne_ | 09:59 | |
| openstackgerrit | Merged openstack-infra/project-config: Make zaqarclient functional gate non-voting https://review.openstack.org/180332 | 10:03 |
| *** freerunner has quit IRC | 10:03 | |
| openstackgerrit | Merged openstack-infra/project-config: Extend Zaqar's gate jobs https://review.openstack.org/180333 | 10:04 |
| *** freerunner has joined #openstack-infra | 10:04 | |
| *** e0ne_ is now known as e0ne | 10:05 | |
| *** freerunner has quit IRC | 10:05 | |
| sdague | I pushed a bunch of rechecks into the queue to hopefully lessen the pain on the fires yesterday | 10:07 |
| *** dims has joined #openstack-infra | 10:07 | |
| sdague | there were a remarkable number of changes in gerrit where there was no jenkins vote on the current patch | 10:08 |
| *** hdd_ has quit IRC | 10:11 | |
| *** jtriley has joined #openstack-infra | 10:11 | |
| *** hdd_ has joined #openstack-infra | 10:12 | |
| *** hdd__ has joined #openstack-infra | 10:15 | |
| *** hdd_ has quit IRC | 10:16 | |
| *** jtriley has quit IRC | 10:16 | |
| *** fhubik is now known as fhubik_afk | 10:17 | |
| *** ddieterly has joined #openstack-infra | 10:18 | |
| *** ddieterly has quit IRC | 10:23 | |
| rakhmerov | hi, we need help with stackforge/python-mistralclient tags. We now have several tags starting with 2015.1 (done by mistake) and now we can't user normal versions like 0.2.0 because looks like pbr checks that versions are not rolling back (it looks at git history) | 10:26 |
| rakhmerov | setting version = 0.2 in setup.cfg doesn't help either | 10:26 |
| rakhmerov | any recommendations on this? | 10:27 |
| *** lennyb has joined #openstack-infra | 10:28 | |
| rakhmerov | the only solution I see is to remove tags 2015.1.x at all but I heard it's not so easy to do | 10:28 |
| *** Longgeek has joined #openstack-infra | 10:35 | |
| *** coolsvap is now known as coolsvap|afk | 10:38 | |
| *** claudiub has joined #openstack-infra | 10:38 | |
| *** pc_m has joined #openstack-infra | 10:42 | |
| openstackgerrit | Fabien Boucher proposed openstack-infra/puppet-zuul: Attempt to add an acceptance test with beaker for puppet-zuul https://review.openstack.org/180214 | 10:45 |
| AJaeger | fungi, clarkb, jeblair: The post jobs like heat-upstream-translation-update get a NOT_REGISTERED - looks related to the daily proposals | 10:46 |
| *** yfried has quit IRC | 10:50 | |
| rakhmerov | fungi, jeblair: can you comment on tagging problem? | 10:51 |
| *** yfried has joined #openstack-infra | 10:55 | |
| *** freerunner has joined #openstack-infra | 10:56 | |
| *** ildikov has quit IRC | 10:57 | |
| *** dizquierdo has joined #openstack-infra | 11:02 | |
| *** freerunner has quit IRC | 11:03 | |
| fungi | AJaeger: if we added those jobs in the past day, puppet is not yet reenabled on the jenkins masters so they don't have it yet. about to check them over and reenable | 11:06 |
| AJaeger | fungi, these are ancient jobs - on jenkins.openstack.org | 11:07 |
| fungi | sdague: yeah, there were several unrelated incidents yesterday (between zuul emergency restarts and the gerrit stream-events problem) where we lost change events | 11:08 |
| *** fhubik_afk is now known as fhubik | 11:08 | |
| AJaeger | fungi, the job I mentioned is now on jenkins.openstack.org | 11:09 |
| fungi | AJaeger: odd. i'll look into it as soon as i can | 11:09 |
| AJaeger | fungi, thanks. | 11:09 |
| sdague | fungi: yeh, and all the gerrit event stream dies over the last couple of months | 11:09 |
| sdague | some of these went back a ways | 11:10 |
| AJaeger | fungi, https://jenkins.openstack.org/view/All/job/sahara-upstream-translation-update/ - and status.openstack.org shows NON_REGISTERED for this job | 11:10 |
| AJaeger | (see post jobs for sahara with the red light) | 11:10 |
| sdague | there were about 100 reviews without a jenkins vote | 11:10 |
| *** freerunner has joined #openstack-infra | 11:10 | |
| fungi | rakhmerov: tags can't really be deleted. they are forever. your options are to continue releasing versions later than your highest tags or stop using pbr or start a new project | 11:10 |
| fungi | sdague: not at all surprising | 11:11 |
| AJaeger | fungi, post and proposal job shouldn't keep you from your coffee - just wanted to point it out while I see it | 11:12 |
| rakhmerov | fungi: that s..cks ) Ok, thanks. I got it | 11:12 |
| fungi | rakhmerov: or another option is to get involved with pbr development... submit a patch which allows you to configure a list of tags to ignore or something | 11:13 |
| rakhmerov | fungi: yeah, ok | 11:14 |
| rakhmerov | fungi: one more question | 11:14 |
| rakhmerov | given that mistral is still in stackforge can we do something while moving it to openstack/ ? Like ignoring python-mistralclient at all and uploading it into openstack/python-mistralclient from scratch? | 11:15 |
| rakhmerov | or removing the tags again | 11:15 |
| *** ildikov has joined #openstack-infra | 11:16 | |
| fungi | rakhmerov: that would make it a little easier to prevent from impacting our infrastructure, but doesn't solve the problem of local clones developers/end users might be maintaining. the ultimate problem is that tag deletion doesn't register on a remote update, so everyone would need to know to reclone or to manually delete local copies of those tags | 11:17 |
| fungi | rather than just changing their git remote after the rename (which would not get rid of those stray tags) | 11:18 |
| rakhmerov | fungi: yeah, I see, you're right | 11:18 |
| *** ddieterly has joined #openstack-infra | 11:19 | |
| *** otter768 has joined #openstack-infra | 11:20 | |
| *** isviridov is now known as isviridov_away | 11:20 | |
| openstackgerrit | Fabien Boucher proposed openstack-infra/puppet-zuul: Attempt to add an acceptance test with beaker for puppet-zuul https://review.openstack.org/180214 | 11:22 |
| *** ddieterly has quit IRC | 11:23 | |
| *** freerunner has quit IRC | 11:24 | |
| *** otter768 has quit IRC | 11:24 | |
| sdague | fungi: it's also nice to see what the gate looks like now that we're working well. recheck of 100 patches burst us just about to quota, but are all flushed out now | 11:26 |
| *** yfried is now known as yfried|afk | 11:27 | |
| *** boris-42 has joined #openstack-infra | 11:27 | |
| openstackgerrit | Renat Akhmerov proposed openstack-infra/project-config: Rename stackforge/mistral to openstack/mistral https://review.openstack.org/175328 | 11:27 |
| fungi | sdague: yeah, it's too bad every day can't be a dance party like that | 11:28 |
| *** BobBall has quit IRC | 11:29 | |
| *** yamahata has quit IRC | 11:30 | |
| fungi | AJaeger: looks like none of the upstream-translation-update jobs reregistered in zuul after it was restarted. probably a problem with jenkins.o.o or at least the proposal worker. i'll see what i need to do to get that reengaged momentarily | 11:30 |
| sdague | fungi: https://review.openstack.org/#/c/165240 - another review seems blocked on that | 11:30 |
| sdague | about getting us a wheel mirror going | 11:30 |
| *** rfolco has joined #openstack-infra | 11:31 | |
| *** yfried|afk is now known as yfried | 11:31 | |
| *** samueldmq has joined #openstack-infra | 11:33 | |
| samueldmq | morning | 11:33 |
| *** BobBall has joined #openstack-infra | 11:34 | |
| *** BobBall is now known as Guest21766 | 11:34 | |
| sdague | fungi / AJaeger : https://review.openstack.org/#/c/179409 ... I think the sdk effort is blocked until that is landed | 11:35 |
| *** _nadya_ has quit IRC | 11:35 | |
| fungi | AJaeger: okay, the proposal jobs are properly registered in zuul following a graceful restart of jenkins.o.o | 11:36 |
| openstackgerrit | Bharat Kumar Kobagana proposed openstack-infra/project-config: Skip test_volume_boot_pattern for GlusterFS until bug 1444806 is fixed https://review.openstack.org/173408 | 11:36 |
| openstack | bug 1444806 in Cinder "test_volume_boot_pattern tempest test failure for glusterfs backend - Part 2" [Undecided,In progress] https://launchpad.net/bugs/1444806 - Assigned to Bharat Kumar Kobagana (bharat-kobagana) | 11:36 |
| *** woodster_ has joined #openstack-infra | 11:37 | |
| *** freerunner has joined #openstack-infra | 11:38 | |
| sdague | fungi: hey, can you look at the gerrit event stream again? | 11:39 |
| fungi | sdague: sure, checking | 11:39 |
| sdague | oh... zuul is processing a bunch of events, might just be normal load | 11:40 |
| *** hashar has quit IRC | 11:40 | |
| fungi | looks like it. newest change in check was 4 minutes ago and zuul is churning through events/results | 11:40 |
| sdague | yeh, I just pushed a bunch in | 11:40 |
| *** hashar has joined #openstack-infra | 11:40 | |
| sdague | I was surprised they didn't show up yet | 11:40 |
| fungi | also i have a jjb update going on one of the masters which is putting geard under some load, slowing it down | 11:41 |
| sdague | gotcha | 11:41 |
| *** yfried is now known as yfried|afk | 11:41 | |
| sdague | Queue lengths: 472 events, 108 results. | 11:42 |
| *** freerunner has quit IRC | 11:42 | |
| sdague | yeh, the events and results seem to be climbing, I guess that's the geard load? | 11:42 |
| *** zz_dimtruck is now known as dimtruck | 11:43 | |
| sdague | ah, there it popped | 11:43 |
| sdague | I guess it was computing all the merge checks | 11:43 |
| *** yfried|afk is now known as yfried | 11:44 | |
| *** hashar has quit IRC | 11:44 | |
| *** hashar has joined #openstack-infra | 11:44 | |
| *** shardy_ has joined #openstack-infra | 11:45 | |
| *** shardy has quit IRC | 11:46 | |
| *** hashar_ has joined #openstack-infra | 11:47 | |
| mgedmin | fungi, lifeless told me to ask you to review https://review.openstack.org/#/c/180060/ | 11:49 |
| mgedmin | (it's not urgent) | 11:49 |
| *** deepakcs has quit IRC | 11:49 | |
| *** hashar has quit IRC | 11:49 | |
| *** weshay has joined #openstack-infra | 11:50 | |
| *** shardy_ has quit IRC | 11:50 | |
| *** shardy has joined #openstack-infra | 11:51 | |
| *** dprince has joined #openstack-infra | 11:51 | |
| *** gulic has joined #openstack-infra | 11:52 | |
| *** gulic has left #openstack-infra | 11:52 | |
| *** Guest21766 has quit IRC | 11:53 | |
| *** ibiris is now known as ibiris_away | 11:54 | |
| *** dkranz has joined #openstack-infra | 11:54 | |
| *** alexpilotti has joined #openstack-infra | 11:55 | |
| *** _nadya_ has joined #openstack-infra | 11:56 | |
| *** ldnunes has joined #openstack-infra | 11:56 | |
| *** BobBall_ has joined #openstack-infra | 11:57 | |
| samueldmq | how do I follow the merge process (in zuul ?) of a review which received workflow+1 | 11:58 |
| samueldmq | for example, https://review.openstack.org/#/c/179983/ | 11:58 |
| *** markvoelker has joined #openstack-infra | 11:59 | |
| *** freerunner has joined #openstack-infra | 11:59 | |
| *** kgiusti has joined #openstack-infra | 11:59 | |
| openstackgerrit | Merged openstack-infra/project-config: Adds refs/heads/stable/* to compute-hyperv https://review.openstack.org/180187 | 11:59 |
| *** ibiris_away is now known as ibiris | 12:00 | |
| *** _nadya_ has quit IRC | 12:00 | |
| *** jtriley has joined #openstack-infra | 12:00 | |
| samueldmq | sdague, ^ see me question just above, I think you may have the answer :) | 12:01 |
| *** jyuso has quit IRC | 12:01 | |
| *** jyuso1 has joined #openstack-infra | 12:01 | |
| sdague | samueldmq: can you reframe the question | 12:02 |
| sdague | samueldmq: I'm not sure I know what you are asking exactly | 12:03 |
| samueldmq | sdague, I want to see where my change exactly is | 12:03 |
| sdague | samueldmq: ok, so http://status.openstack.org | 12:03 |
| sdague | samueldmq: ok, so http://status.openstack.org/zuul | 12:03 |
| samueldmq | sdague, if it is being reverified, in merge process, whatever | 12:03 |
| openstackgerrit | Merged openstack-infra/project-config: Adds refs/heads/stable/* to networking-hyperv https://review.openstack.org/180160 | 12:04 |
| *** bknudson has joined #openstack-infra | 12:04 | |
| openstackgerrit | Merged openstack-infra/project-config: Adds pypi-jobs to compute-hyperv https://review.openstack.org/179782 | 12:04 |
| samueldmq | sdague, yes that' what I want, thanks | 12:04 |
| *** Mithrandir has left #openstack-infra | 12:04 | |
| openstackgerrit | Merged openstack-infra/project-config: Adds publish-to-pypi to networking-hyperv https://review.openstack.org/179844 | 12:04 |
| *** btully has joined #openstack-infra | 12:05 | |
| *** jtriley has quit IRC | 12:05 | |
| *** sdake has joined #openstack-infra | 12:07 | |
| *** hashar_ has quit IRC | 12:08 | |
| *** mherdad has joined #openstack-infra | 12:08 | |
| *** aysyd has joined #openstack-infra | 12:10 | |
| openstackgerrit | Qiming Teng proposed openstack/requirements: Add senlin and its client to project list https://review.openstack.org/180501 | 12:10 |
| openstackgerrit | Merged openstack-infra/project-config: Add missing ceilometer services to heat job https://review.openstack.org/178472 | 12:11 |
| *** sdake_ has quit IRC | 12:12 | |
| *** ddieterly has joined #openstack-infra | 12:14 | |
| *** erlon has joined #openstack-infra | 12:15 | |
| *** eharney has quit IRC | 12:17 | |
| *** btully has quit IRC | 12:20 | |
| *** dimtruck is now known as zz_dimtruck | 12:21 | |
| *** gyee has joined #openstack-infra | 12:22 | |
| *** ryanpetrello_ has quit IRC | 12:22 | |
| *** e0ne is now known as e0ne_ | 12:22 | |
| *** ryanpetrello_ has joined #openstack-infra | 12:22 | |
| *** ryanpetrello_ has quit IRC | 12:25 | |
| *** hashar has joined #openstack-infra | 12:25 | |
| *** _nadya_ has joined #openstack-infra | 12:25 | |
| *** ryanpetrello_ has joined #openstack-infra | 12:25 | |
| *** mherdad has quit IRC | 12:26 | |
| *** mherdad has joined #openstack-infra | 12:27 | |
| *** sabeen1 has joined #openstack-infra | 12:29 | |
| *** e0ne_ is now known as e0ne | 12:29 | |
| *** abregman has quit IRC | 12:31 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/shade: Update server API for get/list/search interface https://review.openstack.org/180293 | 12:31 |
| *** BharatK has quit IRC | 12:32 | |
| *** dkranz has quit IRC | 12:34 | |
| openstackgerrit | Merged openstack-infra/project-config: Use surveil gerrit acl for python-surveilclient https://review.openstack.org/180257 | 12:35 |
| openstackgerrit | Merged openstack-infra/project-config: adding akanda horizon plugin to stackforge https://review.openstack.org/180294 | 12:36 |
| openstackgerrit | Merged openstack-infra/project-config: Add apps-catalog-ci group for apps-catalog project https://review.openstack.org/180201 | 12:36 |
| *** freerunner has quit IRC | 12:36 | |
| *** ajmiller has quit IRC | 12:38 | |
| *** rbradfor has joined #openstack-infra | 12:39 | |
| *** bswartz has quit IRC | 12:43 | |
| *** freerunner has joined #openstack-infra | 12:44 | |
| *** BharatK has joined #openstack-infra | 12:47 | |
| *** abregman has joined #openstack-infra | 12:47 | |
| *** baoli has joined #openstack-infra | 12:48 | |
| AJaeger | thanks, fungi ! | 12:49 |
| AJaeger | sdague, approved it. | 12:49 |
| *** dkranz has joined #openstack-infra | 12:50 | |
| sdague | AJaeger: thanks | 12:50 |
| fungi | still struggling with jjb on jenkins02. getting occasional proxy errors trying to update there. may need to put it in shutdown, but before i do i'll make sure to bring all the other masters out of shutdown | 12:50 |
| *** hashar has quit IRC | 12:52 | |
| *** dims has quit IRC | 12:52 | |
| *** openstackgerrit has quit IRC | 12:53 | |
| *** openstackgerrit has joined #openstack-infra | 12:53 | |
| *** dims has joined #openstack-infra | 12:53 | |
| *** miqui has joined #openstack-infra | 12:53 | |
| mordred | morning fungi | 12:54 |
| *** hashar has joined #openstack-infra | 12:54 | |
| *** mattt has left #openstack-infra | 12:54 | |
| *** links has quit IRC | 12:54 | |
| fungi | morning mordred | 12:56 |
| *** baoli_ has joined #openstack-infra | 12:57 | |
| AJaeger | sdague, could you review this patch and the followon ones, please? https://review.openstack.org/#/c/180308/ | 12:57 |
| *** gordc has joined #openstack-infra | 12:58 | |
| *** BobBall_ has quit IRC | 12:58 | |
| sdague | AJaeger: so... why exactly? | 12:59 |
| mordred | fungi: I'm online for a little bit before heading back into meeting land ... anything I can help with? | 12:59 |
| sdague | because that seems to expose internal details that we don't really care about | 12:59 |
| sdague | and when ceph support moves into a plugin we now need to change all the job names? | 13:00 |
| *** baoli has quit IRC | 13:00 | |
| fungi | mordred: do you like java? do you like jenkins? do you like reviewing java patches to jenkins plugins? | 13:00 |
| fungi | if so, i have just the thing for you... | 13:00 |
| mordred | fungi: oh god | 13:01 |
| AJaeger | sdague, you're talking about the first one? We had some fun debugging why https://review.openstack.org/173408 failed and that came out as name clash. | 13:01 |
| mordred | fungi: I mean | 13:01 |
| *** spzala has joined #openstack-infra | 13:01 | |
| AJaeger | sdague, I'd like to avoid such name clashes to make it clear which job is used. | 13:01 |
| dstufft | meeting land sounds like a terrible place to visit | 13:01 |
| *** sks has quit IRC | 13:01 | |
| fungi | dstufft: you just have to hope your car doesn't break down while you're passing through | 13:01 |
| AJaeger | YEs, might be some more renaming if you change something - but if you look at a job in zuul/layout.yaml, you know where it comes from | 13:02 |
| sdague | AJaeger: ... I'm still confused | 13:02 |
| dstufft | fungi: gotta lock them car doors and pray | 13:02 |
| AJaeger | sdague, confusion is bad ;) What are you confused about? | 13:02 |
| sdague | AJaeger: so the intent in devstack is to push a lot of things into external plugins | 13:02 |
| *** Somay has joined #openstack-infra | 13:02 | |
| mordred | dstufft: it's very dangerous | 13:03 |
| *** BobBall has joined #openstack-infra | 13:03 | |
| sdague | so we're going to add -plugin- to all trove jobs? | 13:03 |
| sdague | and zaqar? and everything on stackforge... ? | 13:03 |
| sdague | this just doesn't seem useful | 13:03 |
| *** BobBall is now known as Guest4240 | 13:03 | |
| sdague | dstufft: or find the monorail out | 13:03 |
| Clint | mmm.. monorail | 13:03 |
| dstufft | sdague: piublic transportation? what kind of socialist utopia is this meeting land! | 13:04 |
| fungi | Clint: what's that word? | 13:04 |
| sdague | :) | 13:04 |
| *** jswarren has joined #openstack-infra | 13:04 | |
| fungi | that's right! monorail! | 13:05 |
| AJaeger | sdague, it would indeed make such a change a few more lines. But if you look at the debugging we needed to do for the other patch, it could help us avoid that time. | 13:05 |
| sdague | AJaeger: mostly it seems like a massive leak of an abstraction out of the system | 13:05 |
| *** ddieterly has quit IRC | 13:05 | |
| *** wayneeseguin has joined #openstack-infra | 13:05 | |
| AJaeger | sdague, I'm not getting that point | 13:06 |
| fungi | (well, sir, there's nothing on earth like a genuine, bona fide, electrified, six-car monorail) | 13:07 |
| * fungi stops | 13:07 | |
| sdague | AJaeger: we do a lot of configuration of jobs in the system | 13:07 |
| sdague | whether or not a devstack plugin is being used doesn't seem like top level information that's useful most of the time | 13:07 |
| *** antonym has quit IRC | 13:08 | |
| fungi | sdague: i mainly suggested namespacing the plugin jobs because AJaeger expressed a concern about collisions between parameterized job name extensions and existing jobs of the same name | 13:08 |
| fungi | it has come up exactly once, as far as i know, so may not be worth actually spending time to fix long-term | 13:09 |
| openstackgerrit | Bharat Kumar Kobagana proposed openstack-infra/project-config: Skip test_volume_boot_pattern for GlusterFS until bug 1444806 is fixed https://review.openstack.org/173408 | 13:09 |
| openstack | bug 1444806 in Cinder "test_volume_boot_pattern tempest test failure for glusterfs backend - Part 2" [Undecided,In progress] https://launchpad.net/bugs/1444806 - Assigned to Bharat Kumar Kobagana (bharat-kobagana) | 13:09 |
| *** eharney has joined #openstack-infra | 13:09 | |
| *** Guest4240 has quit IRC | 13:09 | |
| AJaeger | sdague, fungi: If you all think, it's not worth it, I'll stop arguing. We had the same problem with name clashes when mordred moved the docs infra jobs. | 13:10 |
| openstackgerrit | Merged openstack-infra/gearman-plugin: Stop sending status updates https://review.openstack.org/180249 | 13:11 |
| sdague | AJaeger: so is the real concern that a parameterized job can stomp a non parameterized job (or visa versa)? | 13:11 |
| AJaeger | sdague, yes. Or that two different parameterized jobs can clash | 13:11 |
| sdague | so can't we test for that in unit tests? | 13:11 |
| sdague | 2 things evaled to the same name? | 13:11 |
| *** radez_g0n3 is now known as radez | 13:12 | |
| AJaeger | sdague, unit tests show us if we really have two clashed instantions. | 13:12 |
| fungi | if you have a template foo{bar}{baz} and another template foo-quux{baz} and you instantiate them both and pass foo:-quux to the first when doing so, you can end up with two jobs colliding on the same name | 13:12 |
| sdague | fungi: sure | 13:12 |
| fungi | er, bar:-quux | 13:12 |
| AJaeger | But then jjb will throw an error and you have to figure out why it suddenly fails | 13:12 |
| sdague | AJaeger: right, so isn't that the place to put the check, and make it explicit about what the failure is | 13:13 |
| fungi | anyway, yes i think if the jjb error says you have two jobs with the same name and tells you the name, hunting that down should be easy once you know this is possible | 13:13 |
| AJaeger | Or by accident you instantiate the wrong one. | 13:13 |
| AJaeger | Let me check again the failure... | 13:13 |
| *** tnovacik has joined #openstack-infra | 13:13 | |
| sdague | it seems like we are catching it already, so it's just a matter of being more helpful on "oh, these 2 things collided" | 13:13 |
| fungi | if the jjb error doesn't make that clear, we should definitely do something to fix that | 13:13 |
| *** sks has joined #openstack-infra | 13:13 | |
| AJaeger | sorry, meeting time now, will catch up later... | 13:14 |
| sdague | and ideally even backtrack to the templates that were the basis for both of those | 13:14 |
| *** BobBall has joined #openstack-infra | 13:14 | |
| *** BobBall is now known as Guest20558 | 13:14 | |
| fungi | i've forgotten too much of jjb's internals to remember if it keeps track of the files where it got the various pieces it loaded | 13:14 |
| sdague | I mostly just loath adding 7 new characters to basically all job names, which are already so long they break zuul formatting half the time | 13:15 |
| openstackgerrit | Merged openstack-infra/project-config: added chef-rake-jobs for openstack-chef-repo https://review.openstack.org/179721 | 13:15 |
| sdague | because a thing happened once | 13:15 |
| fungi | so doing that may entail some additional improvement to its job datastructure, but seems like a good thing for it to have if it doesn't yet | 13:15 |
| openstackgerrit | Merged openstack-infra/project-config: New stackforge project: Bansho https://review.openstack.org/180166 | 13:16 |
| *** mherdad has quit IRC | 13:17 | |
| fungi | as a separate point, improving the zuul status display to not break when presented with long job names (perhaps by truncating them and then adding an alt-name/hover tooltip with the full name) should also be on the table | 13:17 |
| openstackgerrit | Sergey Kraynev proposed openstack-infra/project-config: Add functional test job for python-heatclient https://review.openstack.org/180536 | 13:19 |
| *** e0ne is now known as e0ne_ | 13:19 | |
| mordred | clarkb: I reviewed your fix race patch (shudder) and left a +2 and also 2 questions | 13:20 |
| *** otter768 has joined #openstack-infra | 13:21 | |
| *** e0ne_ is now known as e0ne | 13:21 | |
| *** bswartz has joined #openstack-infra | 13:21 | |
| *** peristeri has joined #openstack-infra | 13:21 | |
| *** tcammann has joined #openstack-infra | 13:22 | |
| *** changbl has joined #openstack-infra | 13:22 | |
| *** fhubik has quit IRC | 13:22 | |
| *** tiswanso has joined #openstack-infra | 13:23 | |
| sdague | fungi: yeh, css only truncation is interesting, and otherwise you are playing font metrics games. I've actually spent some time looking at that. | 13:24 |
| *** Somay has quit IRC | 13:25 | |
| *** otter768 has quit IRC | 13:25 | |
| *** gokrokve has joined #openstack-infra | 13:25 | |
| *** Guest20558 has quit IRC | 13:26 | |
| sdague | fungi: unrelated - I do believe I've got a reasonably patch series to bring back the inline zuul status with all the optimizations asked for, make it easy to test, and turned off by default until a final patch flips a flag - https://review.openstack.org/#/q/status:open+project:openstack-infra/system-config+branch:master+topic:hideci_zuul_optimize,n,z | 13:26 |
| sdague | and because that effort started a long time ago, I'd love to get some reviews in there | 13:26 |
| *** esker has joined #openstack-infra | 13:26 | |
| *** pabelanger has joined #openstack-infra | 13:26 | |
| *** pabelanger has joined #openstack-infra | 13:26 | |
| sc68cal | dtroyer: I +1'd this but it's one of those things that shows how complicated and organic the neutron devstack lib has become - https://review.openstack.org/#/c/179787/1/lib/neutron_plugins/ml2 | 13:28 |
| *** freerunner has quit IRC | 13:28 | |
| *** gokrokve_ has quit IRC | 13:29 | |
| mordred | sdague: "We need to fetch zuul targeted status urls over http as well because | 13:30 |
| mordred | default python 2.7 apparently blows it's stack on zuul's ssl cert." | 13:30 |
| mordred | that seems like an unhappy thing | 13:30 |
| *** BobBall_ has joined #openstack-infra | 13:30 | |
| pabelanger | morning | 13:31 |
| sdague | mordred: yes, fungi pushed a fix for a cert chain issue | 13:31 |
| sdague | I do not know if that landed | 13:31 |
| mordred | as in - default python 2.7 not being happy with zuul's cert... | 13:31 |
| mordred | oh, good | 13:31 |
| sdague | it was this code that discovered that issue | 13:31 |
| *** freerunner has joined #openstack-infra | 13:32 | |
| sdague | once that's landed, I'll fix this code to use https by default | 13:32 |
| fungi | sdague: mordred: yep, still needs reviews... https://review.openstack.org/179935 | 13:33 |
| *** ddieterly has joined #openstack-infra | 13:33 | |
| *** mriedem has joined #openstack-infra | 13:34 | |
| *** antonym has joined #openstack-infra | 13:35 | |
| *** sigmavirus24_awa is now known as sigmavirus24 | 13:35 | |
| mordred | fungi: hahaha. nice | 13:37 |
| anteaya | morning pabelanger | 13:37 |
| sdague | fungi: hmmm... https://jenkins01.openstack.org/job/gate-infra-puppet-apply-centos6/27/consoleFull | 13:37 |
| *** freerunner has quit IRC | 13:37 | |
| sdague | centos6 timeouts in puppet apply | 13:37 |
| sdague | going to fail out a bunch of project-config changes | 13:37 |
| pabelanger | anteaya, You still in Ontario? Or have you relocated some where for openstack? | 13:37 |
| sdague | 2015-05-06 13:15:28.046 | Error: /Stage[main]/Openstack_project::Thick_slave/Package[requests]/ensure: change from 2.7.0 to latest failed: Could not get latest version: HTTP-Error: 503 No healthy backends | 13:38 |
| sdague | 2015-05-06 13:36:49.624 | Build timed out (after 30 minutes). Marking the build as failed. | 13:38 |
| *** openstackgerrit has quit IRC | 13:38 | |
| fungi | sdague: yep, known issue. seems there's some issue with the git on centos 6.x which surfaced right after the "ghost" vulnerability update to glibc | 13:38 |
| sdague | oh, fun... | 13:38 |
| *** openstackgerrit has joined #openstack-infra | 13:38 | |
| fungi | sdague: unfortunately i've never been able to reproduce it on demand to get a full strace of the process | 13:38 |
| fungi | basically it seems to end up blocked on a socket read at some point during remote operations | 13:39 |
| *** packet has joined #openstack-infra | 13:39 | |
| sdague | oh.... | 13:39 |
| sdague | so the git_timed that was added to devstack might have been for centos6 work around | 13:39 |
| fungi | which, given the bits of the system libs eth ghost fix backports touched, is not entirely unlikely it's caused some new bug | 13:39 |
| sdague | because that seems to be the kind of thing it would recover from | 13:39 |
| sdague | and would make sense coming from ianw | 13:40 |
| fungi | sdague: yep, hence why i suggested we might want similar timeout/retry built into zuul-cloner (what the apply jobs are using) | 13:40 |
| *** akshai has joined #openstack-infra | 13:40 | |
| sdague | fungi: gotcha | 13:40 |
| sdague | ok | 13:40 |
| sdague | mordred: thanks for kicking those first 2 patches in | 13:41 |
| *** sdake_ has joined #openstack-infra | 13:42 | |
| *** lifeless has quit IRC | 13:42 | |
| mordred | sdague: sure - I'm looking at the next one - I think the feature flag is a great idea there | 13:42 |
| *** mpavone has quit IRC | 13:43 | |
| openstackgerrit | James E. Blair proposed stackforge/gertty: Fix diff display of deleted empty files https://review.openstack.org/180546 | 13:43 |
| *** sdake__ has joined #openstack-infra | 13:43 | |
| mordred | it looks like jeblair is up! | 13:44 |
| *** sdake has quit IRC | 13:44 | |
| *** gyee has quit IRC | 13:45 | |
| *** craigbr has joined #openstack-infra | 13:46 | |
| *** freerunner has joined #openstack-infra | 13:46 | |
| *** mriedem has quit IRC | 13:46 | |
| *** gyee has joined #openstack-infra | 13:46 | |
| *** mriedem has joined #openstack-infra | 13:47 | |
| *** sdake_ has quit IRC | 13:48 | |
| *** freerunner has quit IRC | 13:48 | |
| fungi | okay, i've brought jenkins05 and 06 back online from when they were in prepare for shutdown yesterday (left 03 alone for now in case jeblair is still poking that one) and put 02 into prepare for shutdown so i can try to get its job updates to complete successfully | 13:48 |
| jeblair | fungi: no, i'll put 03 back in | 13:49 |
| fungi | 02 was pretty clearly starting to suffer from the thread leak issue we see regularly, which explains why jjb kept getting proxy errors on update | 13:49 |
| fungi | thanks jeblair | 13:49 |
| fungi | once i confirm all the masters are back online and have jjb updated properly, then i'll manually apply puppet updates and make sure jjb still runs to completion in a timely manner, then leave puppet agent reenabled on all of them | 13:51 |
| sdague | mordred: yeh, given that we've played revert games with this in the past, it seems like a feature flag lets us have more reviewable code for fixes, instead of it being a giant chunk of revert / revert | 13:51 |
| mordred | sdague: yes. also - it means you can probably be fancy and enable it for yourself for a while | 13:51 |
| fungi | 05 already has new slaves and is running jobs on them, so that's a good sign | 13:51 |
| sdague | mordred: correct | 13:51 |
| mordred | sdague: kindof like how I run tip of pbr and git-review locally | 13:51 |
| fungi | same for 06 | 13:51 |
| sdague | mordred: also it means a subset can test more easily | 13:52 |
| dtroyer | sc68cal: since that is not a new variable, just a new check it 'feels' ok to me. I don't know enough to verify that the assertion in the commit message is true though… | 13:52 |
| sdague | especially given that the proxy doesn't support logging in to gerrit, because I don't know how to intercept the openid call | 13:52 |
| mordred | jeblair, clarkb: when you get a chance, could I get a rereview on 175891? there are currently a couple of brokens in the upstream ansible modules that landed that we need to cut a shade release for, and I'd like to get that patch in the release | 13:54 |
| openstackgerrit | Merged openstack-infra/devstack-gate: Archive glusterfs logs and config https://review.openstack.org/171500 | 13:54 |
| *** JayJ_ has joined #openstack-infra | 13:54 | |
| openstackgerrit | Merged openstack-infra/devstack-gate: Remove heat from default configuration https://review.openstack.org/170608 | 13:54 |
| openstackgerrit | Merged openstack-infra/devstack-gate: Remove pip cache move https://review.openstack.org/178902 | 13:54 |
| openstackgerrit | Merged openstack-infra/devstack-gate: Make sysctl vm dump errexit safe https://review.openstack.org/178903 | 13:54 |
| openstackgerrit | Merged openstack-infra/devstack-gate: Check for mount before creating swap https://review.openstack.org/178904 | 13:54 |
| openstackgerrit | Merged openstack-infra/project-config: enable python-jobs for networking-zvm https://review.openstack.org/179730 | 13:54 |
| jeblair | sdague: so when that merges, we will have three zuul status pages. do you or jhesketh have a plan to consolidate the code base onto the work that jhesketh has done in zuul? | 13:55 |
| *** dboik has joined #openstack-infra | 13:55 | |
| sdague | jeblair: so I think we discussed this the last time, this was about having a minimal indicator inside of gerrit | 13:56 |
| jhesketh | jeblair: I haven't read all the backlog, but I assume you're talking about the live zuul stuff | 13:56 |
| jhesketh | jeblair: are the 3 places you're counting: upstream zuul, status.o.o/zuul and now gerrit? | 13:57 |
| jeblair | sdague: yeah, and i think the results of last time, where we discovered it wasn't using any of the stuff like jquery-visibility, etc, suggested that maybe the idea of code reuse really is a good idea | 13:57 |
| jeblair | jhesketh: yep. "upstream" of course is running on zuul.o.o | 13:57 |
| openstackgerrit | Joshua Hesketh proposed openstack/requirements: Add python-magic to global requirements https://review.openstack.org/180552 | 13:57 |
| *** freerunner has joined #openstack-infra | 13:57 | |
| sdague | jeblair: ok, I took your comments as "this should also do that" | 13:58 |
| jeblair | sdague: yeah, i'm not going to -2 your patches or anything, but i would like to still have a conversation about how to move forward and not have 3 javascript status pages mantained by a bunch of people who don't know javascript :) | 13:58 |
| jhesketh | jeblair: okay.. so we should definately update status.o.o to use what's in the zuul repo. The stuff in gerrit though is actually quite different at this point | 13:58 |
| anteaya | I didn't even know about zuul.o.o | 13:59 |
| sdague | jeblair: ok, seems like a good conversation to have over a meal at summit. | 13:59 |
| jeblair | one of the reasons i like jhesketh's in-zuul-tree stuff is it actually gets reviewed and patched by javascript folks :) | 13:59 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/os-loganalyze: Create devstack plugin for testing os-loganalyze https://review.openstack.org/180553 | 13:59 |
| jeblair | sdague: i better pack a bunch of energy bars if i want to eat anything ;) | 14:00 |
| sdague | heh | 14:00 |
| jhesketh | jeblair: because what is in gerrit is quite different to the other two places, I think the first step is to bring some of that code into the zuul repo. Then all we have to do is consume that within gerrit rather than having it out-of-tree | 14:01 |
| jhesketh | sdague: 180553 might interest you btw | 14:01 |
| jhesketh | sdague: I don't think I'll tackle the tests tonight, but that's the next step | 14:01 |
| sdague | jhesketh: \o/ | 14:01 |
| sdague | thank you | 14:01 |
| mordred | jhesketh: oh - you're still awake - 175891? | 14:02 |
| jhesketh | no trouble, sorry it took so long | 14:02 |
| * jhesketh isn't sure when he's awake anymore | 14:02 | |
| jeblair | jhesketh: okay cool, sounds like a plan (or a plan for a plan) :) | 14:02 |
| sdague | I will play with that today and have review feedback by the time you get up tomorrow | 14:02 |
| jhesketh | sdague: thanks buddy | 14:02 |
| jhesketh | jeblair: yeah, I might look at doing the easier of those tasks this week (updating status.o.o to upstream javascript) | 14:03 |
| jhesketh | (assuming personal life doesn't get in the way which it might for $reasons) | 14:03 |
| sdague | so... upstream js being zuul.o.o? | 14:03 |
| *** gokrokve has quit IRC | 14:03 | |
| sdague | because, zuul.o.o is way heavier on my browser than status.o.o/zuul | 14:04 |
| jhesketh | sdague: well what's in zuul's repo, yes. Not sure if we'd hot-link across domains or host a second copy | 14:04 |
| sdague | to the point that I found it unusable | 14:04 |
| *** macjack has left #openstack-infra | 14:04 | |
| *** tonytan4ever has joined #openstack-infra | 14:04 | |
| sdague | like chrome goes into odd lockups on it | 14:04 |
| jhesketh | :-( | 14:04 |
| jhesketh | okay so it might need some optimising | 14:04 |
| *** signed8bit has joined #openstack-infra | 14:05 | |
| *** isviridov_away is now known as isviridov | 14:05 | |
| sdague | yeh, like the whole UI thread seems to hang on every update, so you'll be scrolling then it will lose all the UI events and just stop | 14:05 |
| jhesketh | sdague: any clues or how to reproduce? | 14:05 |
| jhesketh | yuck | 14:06 |
| sdague | yeh, just try to keep scrolling up and down | 14:06 |
| sdague | it will hang | 14:06 |
| sdague | it comes back | 14:06 |
| sdague | but that kind of responsiveness issue is... pretty suboptimal | 14:06 |
| *** abregman has quit IRC | 14:06 | |
| AJaeger | fungi, sdague: here'S the error message for the duplicate definition http://logs.openstack.org/08/173408/8/check/gate-project-config-layout/0931ead/console.html | 14:06 |
| openstackgerrit | Merged openstack-infra/project-config: Move DIB test dep installation into DIB https://review.openstack.org/179209 | 14:06 |
| *** craigbr has quit IRC | 14:06 | |
| sdague | I believe also when I was on my laptop on battery the page was noticably more power draw as well | 14:07 |
| openstackgerrit | Merged openstack-infra/project-config: Pylint job for Murano project https://review.openstack.org/172887 | 14:07 |
| openstackgerrit | Merged openstack-infra/project-config: Look for yum instead of rpm when caching devstack https://review.openstack.org/179687 | 14:07 |
| sdague | I remember looking at the QA sprint | 14:07 |
| AJaeger | it basically is only "Duplicate definitions for job 'check-tempest-dsvm-full-glusterfs' specified" | 14:07 |
| AJaeger | And then you're lost | 14:07 |
| openstackgerrit | Merged openstack-infra/project-config: Do a better job of identifying CentOS 6 https://review.openstack.org/179694 | 14:07 |
| openstackgerrit | Merged openstack-infra/project-config: Add Python SDK to PROJECTS for SDK gate https://review.openstack.org/179409 | 14:07 |
| *** dustins has joined #openstack-infra | 14:07 | |
| openstackgerrit | Merged openstack-infra/system-config: create proxy for testing hideci https://review.openstack.org/179360 | 14:07 |
| jhesketh | sdague: I'd hardly call that responsive | 14:07 |
| fungi | AJaeger: git grep glusterfs? | 14:07 |
| *** dustins has quit IRC | 14:08 | |
| AJaeger | fungi, this is followup to the name clash discussion where I stepped out an hour ago. | 14:08 |
| *** dustins has joined #openstack-infra | 14:08 | |
| fungi | AJaeger: yep, just suggesting the next step i'd have done after seeing that error | 14:08 |
| *** craigbr has joined #openstack-infra | 14:08 | |
| fungi | AJaeger: though i agree that improving jjb to be able to report the files in which the offending jobs or job-templates originated | 14:08 |
| fungi | is a good idea | 14:08 |
| sdague | jhesketh: yeh that UI thread halt also means that if it happens to be in that refresh loop and you hit pgup or pgdown nothing happens, and then you are confused and start hitting lots of keys :) | 14:08 |
| mordred | fungi, AJaeger: ++ | 14:09 |
| AJaeger | fungi, ah! Yes, seems that improving jjb to throw better errors might be the right approach - just one that I won't sign up for ;) | 14:09 |
| *** lifeless has joined #openstack-infra | 14:10 | |
| fungi | AJaeger: and i can see sdague's point that making more verbose job names just to avoid the risk of future name collisions is sort of a hackish workaround | 14:10 |
| AJaeger | fungi, stepping out and rereading, I'm agreeing with him now. | 14:11 |
| fungi | AJaeger: but improving jjb's error handling there is probably a fairly low-hanging-fruit patch we can sick a new python-savvy contributor on | 14:11 |
| sdague | jhesketh: if you run the heap profile tool in chrome web dev you can actually get a nice visual as well | 14:11 |
| sdague | on zuul.o.o you see basically a 1 second freeze of the tools every 5 seconds during the refresh | 14:12 |
| fungi | AJaeger: especially since it doesn't need any special infrastructure to implement and test | 14:12 |
| sdague | on s.o.o/zuul it's about < 100ms | 14:12 |
| *** isviridov is now known as isviridov_away | 14:12 | |
| AJaeger | fungi, yes agreed. | 14:12 |
| AJaeger | So, I'll abandon my patches and let the glusterfs guys take what they like or go back to the previous version. | 14:13 |
| AJaeger | fungi, sdague thanks for the discussion! | 14:13 |
| sdague | AJaeger: no prob | 14:13 |
| *** amitgandhinz has joined #openstack-infra | 14:19 | |
| *** abregman has joined #openstack-infra | 14:19 | |
| openstackgerrit | Merged openstack-infra/system-config: add dbg function to hideci https://review.openstack.org/179573 | 14:22 |
| openstackgerrit | Merged openstack-infra/project-config: Pass through arbitrary devstack config options https://review.openstack.org/179983 | 14:22 |
| openstackgerrit | Merged openstack-infra/system-config: Revert "Revert zuul status in gerrit" https://review.openstack.org/179361 | 14:22 |
| AJaeger | sdague, I give a procedural -2 to https://review.openstack.org/#/c/179714 so that you don't approve it again ;) | 14:23 |
| sdague | AJaeger: oh, yeh | 14:24 |
| sdague | I saw the deletes | 14:24 |
| sdague | I guess I messed that one up | 14:24 |
| jhesketh | sdague: okay, I'll have a poke at some profiling stuff.. I haven't done anything like that before so it'll be a good learning experience | 14:24 |
| sdague | jhesketh: yeh, that 1 second hang also causes clicks to expand, collapse to go missing | 14:25 |
| sdague | and this is on my 4 core haswell, on 50 mbs fiber | 14:25 |
| sdague | which means I expect that might be worse on more modest machines / network | 14:26 |
| mtreinish | sdague: you obviously need to pay for more bandwidth and hardware | 14:26 |
| sdague | mtreinish: :) | 14:26 |
| jhesketh | heh, there's your problem! ;-) | 14:26 |
| *** e0ne is now known as e0ne_ | 14:27 | |
| *** jlvillal has quit IRC | 14:28 | |
| *** jlvillal has joined #openstack-infra | 14:29 | |
| *** gokrokve has joined #openstack-infra | 14:32 | |
| AJaeger | fungi, do you know what happened with openstack-doc-tools 0.26 release? It never showed up on pypi. | 14:33 |
| *** nelsnelson has joined #openstack-infra | 14:34 | |
| *** gargs has joined #openstack-infra | 14:34 | |
| *** gargs has left #openstack-infra | 14:34 | |
| AJaeger | https://pypi.python.org/pypi/openstack-doc-tools/ has 0.25 only | 14:34 |
| kashyap | fungi: Hi, I think OpenStack Infra also uses Supybot, doesn't it? | 14:37 |
| *** stevemar has joined #openstack-infra | 14:37 | |
| *** e0ne_ is now known as e0ne | 14:37 | |
| kashyap | fungi: If so, does OpenStack Infra mind borrowing this little useful Feature from Fedora? :-) https://github.com/fedora-infra/supybot-fedora/commit/1ba62ced08487fe4dcc8b5040c8fc64ae3b8ce0f | 14:37 |
| *** jtriley has joined #openstack-infra | 14:38 | |
| jhesketh | mordred: okay, reviewed 175891 | 14:39 |
| jhesketh | now it's probably time for sleep | 14:39 |
| AJaeger | good night, jhesketh ! | 14:39 |
| anteaya | kashyap: we haven't adopted teh behaviour of using bots to drive social change | 14:39 |
| jhesketh | o/ | 14:40 |
| fungi | kashyap: yep, the "openstack" bot is basically supybot running the meetbot plugin along with another plugin now to generate links to bug reports | 14:40 |
| anteaya | kashyap: feel free to share your thoughts about how you would like to be addressed with those with whom you interact | 14:40 |
| fungi | kashyap: http://docs.openstack.org/infra/system-config/irc.html#meetbot | 14:40 |
| kashyap | anteaya: Sure. I can see what you're saying. In this case, it _is_ useful because - getting 20 empty pings can be stressful over the course of a day. | 14:40 |
| anteaya | kashyap: sure, so I am learning to ignore them | 14:41 |
| kashyap | anteaya: As the blog indicates in that commit message. | 14:41 |
| anteaya | kashyap: as I ignore questions addressed to "guys" | 14:41 |
| kashyap | anteaya: Also this -- http://openstack.redhat.com/IRC_etiquette | 14:41 |
| anteaya | kashyap: as fungi indicates you are welcome to offer a patch | 14:41 |
| anteaya | kashyap: if you don't like how you are being treated tell people | 14:42 |
| anteaya | different people like to be treated different ways | 14:42 |
| kashyap | anteaya: People treat me just fine, I don't have an issue :-) | 14:42 |
| anteaya | great | 14:42 |
| *** craigbr has quit IRC | 14:42 | |
| fungi | AJaeger: it likely ran afoul of the upload job not being registered in zuul, like the translation jobs you pointed out | 14:43 |
| AJaeger | fungi, so, what can we do? Tag a 0.26.1 release? | 14:43 |
| fungi | AJaeger: i can manually trigger the jobs which never ran for it. will take me a few minutes to get to | 14:43 |
| AJaeger | fungi, no urgency for that one... Thanks! | 14:44 |
| *** jtriley has quit IRC | 14:44 | |
| *** JayJ_ has quit IRC | 14:44 | |
| fungi | kashyap: so, yes, a supybot plug-in might make more sense as i'm not sure we want to maintain a supybot fork | 14:45 |
| *** jtriley has joined #openstack-infra | 14:45 | |
| kashyap | fungi: Yep, agreed. | 14:45 |
| fungi | kashyap: aside from that, i have no opinion on the naked ping issue. like anteaya, i basically ignore those and _if_ someone persists with them i ask them to stop, and _if_ they don't i /ignore their nick altogether | 14:46 |
| *** claudiub has quit IRC | 14:46 | |
| *** craigbr has joined #openstack-infra | 14:47 | |
| fungi | i've also started to intentionally ignore questions directed at me specifically even though they could be answered by many other people in-channel | 14:47 |
| fungi | for example, the one you just asked ;) | 14:48 |
| kashyap | :-) | 14:48 |
| kashyap | Thanks for the response, anyhow. | 14:48 |
| Daviey | fungi: ping. | 14:48 |
| fungi | you're welcome! | 14:48 |
| * fungi adds Daviey to his /ignore list (okay, not really) | 14:49 | |
| Daviey | /trolling :) | 14:49 |
| fungi | "trollping" | 14:49 |
| anteaya | can we get a bot to just ignore Daviey? | 14:50 |
| anteaya | :) | 14:50 |
| fungi | i thought Daviey _was_ a bot? | 14:50 |
| *** bhunter71 has joined #openstack-infra | 14:52 | |
| *** Sukhdev has joined #openstack-infra | 14:53 | |
| *** bknudson has quit IRC | 14:55 | |
| Daviey | fungi: I might be as consistent, high out, multi-threaded and deterministic as a bot.. but I am indeed a humanoid. | 15:01 |
| openstackgerrit | David Caro proposed openstack-infra/jenkins-job-builder: Removed unnecessary parameter from delete_old_jobs https://review.openstack.org/180575 | 15:01 |
| openstackgerrit | Merged openstack-infra/project-config: Add new branch for channels work https://review.openstack.org/180416 | 15:02 |
| *** pc_m has quit IRC | 15:02 | |
| *** gyee has quit IRC | 15:03 | |
| *** yfried has quit IRC | 15:04 | |
| *** marun has joined #openstack-infra | 15:04 | |
| *** BobBall_ has quit IRC | 15:05 | |
| *** sks has quit IRC | 15:05 | |
| *** gyee has joined #openstack-infra | 15:06 | |
| openstackgerrit | David Caro proposed openstack-infra/jenkins-job-builder: Added parallelization options https://review.openstack.org/180463 | 15:06 |
| *** marcusvrn_ has joined #openstack-infra | 15:07 | |
| jeblair | fungi: ping # noqa | 15:09 |
| *** BobBall has joined #openstack-infra | 15:09 | |
| *** Sukhdev has quit IRC | 15:10 | |
| *** freerunner has quit IRC | 15:10 | |
| fungi | heh | 15:12 |
| *** sputnik13 has joined #openstack-infra | 15:12 | |
| hashar | jeblair: thanks for the nodepool/gear tagging :] | 15:12 |
| jeblair | hashar: you're welcome; i might try my hand at zuul soon! :) | 15:13 |
| *** freerunner has joined #openstack-infra | 15:13 | |
| *** armax has joined #openstack-infra | 15:13 | |
| hashar | jeblair: zuul has a bunch of patches pending a final approval. Would be nice to have them land in | 15:13 |
| jeblair | yeah, i'll try to do that first | 15:13 |
| *** sdake has joined #openstack-infra | 15:14 | |
| hashar | some I have been running them for a few months already :} | 15:14 |
| hashar | there is one that alter the zuul-cloner git cache behavior though and that has an impact on your setup https://review.openstack.org/#/c/117626/ | 15:15 |
| BobBall | Anyway to throttle how often nodepool tries to recreate nodes if (for example) quota is exhausted? | 15:17 |
| *** freerunner has quit IRC | 15:17 | |
| *** sdake__ has quit IRC | 15:17 | |
| *** ayoung is now known as eliza | 15:18 | |
| BobBall | I've got an API quota for /servers - once that's exceeded nodepool will just spin trying to create new servers AND keep the quota as exhausted | 15:18 |
| *** lennyb has quit IRC | 15:18 | |
| *** sks has joined #openstack-infra | 15:18 | |
| openstackgerrit | Merged openstack/requirements: Updated oslo_middleware to 1.2.0 https://review.openstack.org/179926 | 15:18 |
| *** eliza is now known as ayoung | 15:18 | |
| jeblair | BobBall: yeah, you can set the 'rate' parameter to match your api rate limit | 15:18 |
| BobBall | Bingo. Thanks. | 15:18 |
| *** rlandy has quit IRC | 15:19 | |
| fungi | BobBall: also set your max-servers to something less than your known quotas (taking instance count, ram, cores, et cetera into account) | 15:19 |
| BobBall | yeah - max-servers is set | 15:19 |
| BobBall | but that didn't stop me hitting the API limit | 15:20 |
| jeblair | BobBall: ('rate' is a poorly named option and is actually the inverse of the rate -- so if you can do 10/sec rate should be 0.1) | 15:20 |
| fungi | and give it a little breathing room if you're using snapshot image builds | 15:20 |
| fungi | BobBall: also check for leaks using nodepool alien-list | 15:20 |
| BobBall | *nod* Do that regularly. Surprisingly I don't see any leaked for ages :) | 15:21 |
| *** otter768 has joined #openstack-infra | 15:21 | |
| fungi | BobBall: nodepool should _not_ try to exceed the configured maximum, so sounds like something is wrong there | 15:21 |
| BobBall | It didn't exceed the maximum number of servers | 15:22 |
| BobBall | but IAD couldn't build servers for a while (networking issues that I'm still trying to track down) | 15:22 |
| *** ayoung is now known as ayoung-eliza | 15:22 | |
| BobBall | and that ended up with very quick churn | 15:22 |
| BobBall | which meant that eventually I exceeded 1000 calls to /servers in 24 hours | 15:22 |
| fungi | BobBall: oh! so you were running afoul of api throttles | 15:22 |
| BobBall | Yup | 15:22 |
| fungi | "limits" as nova calls them | 15:23 |
| *** ayoung-eliza is now known as ayoung | 15:23 | |
| fungi | BobBall: then yes, the rate option is definitely what you want. we use it for exactly the same reasons | 15:23 |
| BobBall | *nod* | 15:23 |
| *** sputnik13 has quit IRC | 15:24 | |
| *** r-daneel has joined #openstack-infra | 15:24 | |
| jeblair | BobBall: and if you (perhaps later) find you need higher limits, i believe you can ask rax for them. i think they are set at a default 'try to protect the users from themselves' level :) | 15:24 |
| BobBall | Does each server create get allocated a Task? i.e. if I set the rate to 60.0 (which I believe is seconds) it'll only create one server per minute at most, right? | 15:25 |
| BobBall | I've got a support ticket to increase the rate too, but I want to throttle it so I'm confident I won't abuse it. | 15:25 |
| jeblair | BobBall: yes; there are other tasks that factor in as well, such as list server calls, etc.... | 15:25 |
| mordred | Shrews, SpamapS: jhesketh made an excellent point about duplicate server names in the inventory code - I think it's a good and hard enough point that we need to solve it properly - so I'd like to go ahead and release what we've got now and go back and attack inventory better | 15:26 |
| jeblair | BobBall: nodepool doesn't break those out though, it's just one value for all api calls. ideally, this could be automated. however, i think it's completely different on different clouds. | 15:26 |
| jeblair | yay openstack standardization | 15:26 |
| mordred | Shrews, SpamapS: is there anything in that patch we should pull out to get into this release? | 15:26 |
| *** otter768 has quit IRC | 15:26 | |
| jeblair | maybe we can add that to shade ;) | 15:26 |
| * mordred reads jeblair and BobBall scrollback | 15:26 | |
| BobBall | *nod* that's fine. I'll just go for "rate: 60.0" and tune it if it's too slow | 15:27 |
| BobBall | It's not that exciting mordred :) | 15:27 |
| * mordred saw "completely different on different clouds" and figured it might be ... but yeah, you're on it | 15:27 | |
| jeblair | mordred: oh, yeah supporting dupes sounds important | 15:28 |
| *** xyang1 has joined #openstack-infra | 15:28 | |
| mordred | jeblair: uhm, yeah | 15:28 |
| Shrews | mordred: yeah, that's a good catch. we should fix that | 15:29 |
| mordred | Shrews: we should consider what to do about the "cloud appends random suffix" problem too | 15:29 |
| mordred | Shrews: I betcha the two will wind up being related | 15:29 |
| Shrews | mordred: nuke the provider that does that??? | 15:29 |
| mordred | Shrews: well, yes. we will do that too | 15:29 |
| Shrews | that's just multi-colored crazy pants | 15:30 |
| *** adrian_otto has joined #openstack-infra | 15:30 | |
| openstackgerrit | Bharat Kumar Kobagana proposed openstack-infra/project-config: Skip test_volume_boot_pattern for GlusterFS until bug 1444806 is fixed https://review.openstack.org/173408 | 15:31 |
| openstack | bug 1444806 in Cinder "test_volume_boot_pattern tempest test failure for glusterfs backend - Part 2" [Undecided,In progress] https://launchpad.net/bugs/1444806 - Assigned to Bharat Kumar Kobagana (bharat-kobagana) | 15:31 |
| *** krtaylor has quit IRC | 15:31 | |
| *** hashar is now known as hasharMeeting | 15:32 | |
| jeblair | Shrews: will you be wearing yours to the summit? | 15:33 |
| clarkb | mordred: I don't understand your first comment on https://review.openstack.org/#/c/179988/5/src/main/java/hudson/plugins/gearman/MyGearmanWorkerImpl.java LinkedList is not thread safe | 15:33 |
| Shrews | jeblair: i have decided not to wear pants at this summit | 15:33 |
| mordred | clarkb: your comment said it was | 15:33 |
| anteaya | Shrews: thanks for the advanced warning | 15:33 |
| clarkb | mordred: which comment? | 15:33 |
| Shrews | anteaya: :) | 15:34 |
| anteaya | Shrews: I hope your kilts are in fine form | 15:34 |
| anteaya | utilikilts are the best | 15:34 |
| mordred | clarkb: "handleSessionEvent() method puts all events on a thread | 15:34 |
| mordred | safe queue for synchronous processing" | 15:34 |
| mordred | clarkb: I may have read that wrong though | 15:34 |
| anteaya | multi-coloured crazy kilts | 15:35 |
| anteaya | bit of a ring to that | 15:35 |
| clarkb | mordred: ya thats the new code | 15:35 |
| clarkb | mordred: which uses a thread safe list | 15:35 |
| mordred | clarkb: kk | 15:36 |
| fungi | Shrews: we can be known by our crazyshorts instead | 15:37 |
| *** openstackgerrit has quit IRC | 15:37 | |
| anteaya | and crazyshirts | 15:37 |
| *** openstackgerrit has joined #openstack-infra | 15:37 | |
| clarkb | mordred: note the type is changing and we initialize the variable in the constructor | 15:37 |
| * fungi already wears crazyshirts | 15:37 | |
| *** sks has quit IRC | 15:37 | |
| anteaya | you do | 15:37 |
| anteaya | and I can spot you across a crowded room because of it | 15:38 |
| anteaya | not sure if that is a bug or feature | 15:38 |
| *** cody-somerville has joined #openstack-infra | 15:39 | |
| *** hdd__ has quit IRC | 15:40 | |
| pblaho | hi all.... since when http://logs.openstack.org returns html logs as application/octet-stream ? | 15:41 |
| *** Sukhdev has joined #openstack-infra | 15:43 | |
| *** dizquierdo has quit IRC | 15:44 | |
| *** yamahata has joined #openstack-infra | 15:46 | |
| *** bknudson has joined #openstack-infra | 15:46 | |
| *** dkranz has quit IRC | 15:48 | |
| *** akshai has quit IRC | 15:48 | |
| *** dannywilson has joined #openstack-infra | 15:49 | |
| *** akshai has joined #openstack-infra | 15:49 | |
| *** ashleighfarnham has joined #openstack-infra | 15:51 | |
| *** patrickeast has joined #openstack-infra | 15:51 | |
| *** Swanson has joined #openstack-infra | 15:53 | |
| *** signed8bit has quit IRC | 15:53 | |
| timrc | clarkb, jeblair, fungi, mordred and anyone else that may know: Where do upstream zuul-mergers get their refs? Do you push out to git mirrors? I was under the impression that refs were fetched from gerrit repos directly, but is that wrong? | 15:53 |
| *** patrickeast_ has joined #openstack-infra | 15:54 | |
| *** viktors is now known as viktors|afk | 15:54 | |
| jeblair | timrc: they are fetched from gerrit directly; i think clarkb started a patch to make that an option, but it hasn't landed yet? | 15:54 |
| fungi | timrc: the upstream zuul mergers pull from gerrit's git service and then create the zuul refs | 15:54 |
| clarkb | jeblair: correct, gozer uses that patch for reasons though | 15:55 |
| mordred | yah | 15:55 |
| timrc | cody-somerville, There you have it ^^ | 15:55 |
| clarkb | I think jesusaurus has been trying to curate the patch /me has continued to review it in spurts | 15:55 |
| clarkb | mordred: responded to your comments, let me know if that helps | 15:56 |
| *** patrickeast has quit IRC | 15:56 | |
| *** patrickeast_ is now known as patrickeast | 15:56 | |
| timrc | clarkb, It's failing tests for some mysterious reason. | 15:56 |
| timrc | That's the patch I was telling jeblair about that needs to do have a retry for timeouts connecting to the git mirror (arguably there should not be timeouts to the git mirror, but yay resilience) | 15:56 |
| clarkb | well you have to retry because until gerrit 2.10 or something there is no gerrit event for replication completed | 15:57 |
| *** scheuran has quit IRC | 15:57 | |
| clarkb | so there is no way to know if your data has arrived until you check, and those checks may fail | 15:57 |
| mordred | clarkb: ah - I grok the disconnect | 15:57 |
| mordred | clarkb: my brain wasnt catching that that was the constructor because I'm stupid | 15:57 |
| mordred | clarkb: yeah - I think the patch looks great | 15:58 |
| sdague | clarkb / fungi / dstufft - so given you all were knee deep in the cffi bit last night - https://review.openstack.org/#/c/180549/ is a different way to get around the install vs. build issue in devstack than what we landed, which is a little cleaner. Your thoughts are appreciated. | 15:59 |
| clarkb | mordred: cool, thanks for reviewing | 16:00 |
| clarkb | timrc: I will take a look at that change again today as soon as I am awak | 16:00 |
| *** sarob has joined #openstack-infra | 16:00 | |
| *** dkranz has joined #openstack-infra | 16:00 | |
| *** hdd__ has joined #openstack-infra | 16:01 | |
| timrc | clarkb, Well we are timing out on the clone.... it just sit there until the timeout limit is reached and then fails. Developers then recheck and eventually their change passes tests. So I think we need to fix the actual timeout issue but also add a retry so things are tried again N number of times after a timeout occurs. | 16:01 |
| *** claudiub has joined #openstack-infra | 16:01 | |
| timrc | That way developer rechecks don't effectively become that retry. | 16:01 |
| clarkb | timrc: honestly, if that is becoming a problem you should upgrade to latest gerrit and update the feature to use the replication events | 16:02 |
| clarkb | timrc: because you can't fix it that way, you can only add more duct tape | 16:02 |
| *** hdd__ has quit IRC | 16:03 | |
| *** sergsh has quit IRC | 16:03 | |
| *** sergsh_ has quit IRC | 16:03 | |
| timrc | clarkb, I think upgrading gerrit is on our road map, but it's a two step process. First upgrading to the last version that can run on Precise and then again to the Trusty version some time in the future. | 16:03 |
| clarkb | also I rmember looking at these fails once, and going "I have no idea what is happening" maybe second time around will be more enlightening | 16:03 |
| jeblair | clarkb: i agree that would be a better solution -- but if the cloner knows that a git operation that should have succeeded failed, then why not retry? | 16:04 |
| timrc | It's good to know there's a gerrit event we can listen for. | 16:04 |
| clarkb | jeblair: it already does retry is what I am saying | 16:04 |
| jeblair | oh | 16:04 |
| clarkb | jeblair: so timrc is suggesting adding more retries which IMO is pointless | 16:04 |
| jeblair | clarkb: we have 1 retry and the suggestion is N retries? | 16:04 |
| *** jlanoux has quit IRC | 16:04 | |
| jeblair | where N>1 | 16:04 |
| clarkb | jeblair: its a configurable retry that is arbitrary iirc | 16:04 |
| timrc | clarkb, I didn't think it retried... it only attempts the clone once and keeps checking the clock to see if it's timed out and then fails. | 16:04 |
| *** tkelsey has left #openstack-infra | 16:04 | |
| jeblair | oh, er.... so what's the question? :) | 16:04 |
| clarkb | ya https://review.openstack.org/#/c/76057/28/doc/source/zuul.rst | 16:05 |
| clarkb | poll_interval and poll_timeout are completely configurable so can be set to an arbitrary retry and timeout for what you want | 16:05 |
| jeblair | clarkb: that translates to a number of retries? | 16:05 |
| clarkb | jeblair: yes it should do a fetch every interval seconds until the timeout elapses | 16:06 |
| clarkb | if fetch time is > timeout then you will only do one | 16:06 |
| openstackgerrit | Victor Sergeyev proposed openstack-infra/project-config: Move Ironic python3.4 job from experimental to gate https://review.openstack.org/180599 | 16:06 |
| jeblair | clarkb: got it | 16:08 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/shade: Update images API for get/list/search interface https://review.openstack.org/180600 | 16:08 |
| timrc | clarkb, Hm. | 16:08 |
| *** ajmiller has joined #openstack-infra | 16:08 | |
| timrc | So I would have to set the timeout to some multiple of the timeout git uses to clone to get multiple tries? | 16:08 |
| *** bknudson has quit IRC | 16:09 | |
| clarkb | timrc: yes I think so | 16:09 |
| *** shashankhegde has joined #openstack-infra | 16:09 | |
| clarkb | timrc: but its not a clone fwiw | 16:09 |
| clarkb | timrc: its just a fetch of the ref | 16:09 |
| timrc | clarkb, Doh, fetch. | 16:11 |
| clarkb | the initial clone of the repo by the merger will only be attempted once but usually thats fine because you can have a bit more contorl over iniial project configuration than you do for every ref replication | 16:13 |
| *** gyee has quit IRC | 16:13 | |
| *** bknudson has joined #openstack-infra | 16:14 | |
| openstackgerrit | Merged openstack-infra/devstack-gate: Force openvswitch as the Neutron agent for Ironic https://review.openstack.org/172118 | 16:15 |
| openstackgerrit | Merged openstack-infra/devstack-gate: Add dstat to subnode in multinode mode https://review.openstack.org/177970 | 16:16 |
| clarkb | ok so it looks like the tests are timing out, I am going to bump the timeout but I am going to guess that we end up retrying fetches for the refs in the test framework indefinitely for some reason | 16:16 |
| openstackgerrit | Merged openstack-infra/devstack-gate: Follow up with some comments to clarify OVS commands https://review.openstack.org/173621 | 16:16 |
| *** markus_z has quit IRC | 16:16 | |
| openstackgerrit | Merged openstack-infra/devstack-gate: Add host counter to ansible inventory https://review.openstack.org/177920 | 16:16 |
| jlvillal | Is it already a known issue that the "Toggle CI" button is missing? It is for me, at least. | 16:21 |
| clarkb | jlvillal: it is there for me let me try without caches | 16:21 |
| clarkb | jlvillal: also you might try refreshing those | 16:21 |
| *** gokrokve has quit IRC | 16:21 | |
| fungi | jlvillal: i bet someone approved a change to some of the gerrit javascript | 16:21 |
| *** gokrokve has joined #openstack-infra | 16:21 | |
| clarkb | ah yup a hard refresh for me has removed it | 16:22 |
| fungi | i'll touch the proper file | 16:22 |
| *** shashankhegde has quit IRC | 16:22 | |
| fungi | clarkb: jlvillal: it's fixed for me now... you? | 16:22 |
| clarkb | back for me | 16:22 |
| jlvillal | clarkb: It is working now :) | 16:23 |
| jlvillal | fungi: ^^ | 16:23 |
| fungi | there's something about the 'touch ~gerrit2/review_site/etc/GerritSiteHeader.html' exec in our puppetry which isn't taking care of this automagically. i wonder if adding a sleep to it would help | 16:23 |
| jlvillal | fungi: clarkb: Thanks! | 16:23 |
| *** soren has quit IRC | 16:25 | |
| *** Krinkle|detached is now known as Krinkle | 16:25 | |
| clarkb | timrc: jesusaurus if I bump the test timeout that change passes, so I think either the test timeout was reduced or things changed such that the polling is slower now and can't fit in under the timeout | 16:26 |
| clarkb | will leave a comment | 16:26 |
| *** shashankhegde has joined #openstack-infra | 16:26 | |
| clarkb | looks like the longest test takes ~35 seconds and timeout is 30 seconds | 16:26 |
| timrc | clarkb, Ah. Thanks. | 16:27 |
| *** jcoufal_ has quit IRC | 16:27 | |
| *** amitgandhinz has quit IRC | 16:28 | |
| openstackgerrit | Fabien Boucher proposed openstack-infra/puppet-zuul: Manage logging configuration files (feature flag) https://review.openstack.org/180614 | 16:28 |
| *** jistr has quit IRC | 16:29 | |
| openstackgerrit | Alec Hothan proposed openstack-infra/project-config: Need to allow merge commits in vmtp.git https://review.openstack.org/180616 | 16:31 |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/system-config: Avoid race on reload_gerrit_header https://review.openstack.org/180617 | 16:32 |
| fungi | jlvillal: clarkb: ^ | 16:32 |
| jlvillal | fungi: Thanks | 16:32 |
| *** claudiub has quit IRC | 16:33 | |
| *** bhunter71 has quit IRC | 16:34 | |
| *** freerunner has joined #openstack-infra | 16:34 | |
| *** krtaylor has joined #openstack-infra | 16:35 | |
| *** dannywilson has quit IRC | 16:35 | |
| SpamapS | mordred: can you restate or point me to the point about duplicate hostnames? | 16:37 |
| *** sputnik13 has joined #openstack-infra | 16:37 | |
| *** Guest55056 has quit IRC | 16:38 | |
| *** bhunter71 has joined #openstack-infra | 16:39 | |
| *** marcusvrn1 has joined #openstack-infra | 16:39 | |
| *** ssam2 has quit IRC | 16:39 | |
| *** freerunner has quit IRC | 16:40 | |
| mordred | SpamapS: restating will be easier | 16:40 |
| *** freerunner has joined #openstack-infra | 16:41 | |
| openstackgerrit | Derek Higgins proposed openstack-infra/system-config: Add a Fedora 21 node to tripleo cloud https://review.openstack.org/169778 | 16:41 |
| mordred | SpamapS: one can have duplicated hostnames both inside of a single cloud, and also potentially across different clouds | 16:41 |
| *** whoops has joined #openstack-infra | 16:41 | |
| *** freerunner has quit IRC | 16:41 | |
| mordred | SpamapS: so when inventory reads your hosts and combines them into the inventory, at the moment, you'd be losing data if you had such a situation | 16:41 |
| mordred | I think I've got a solution though | 16:41 |
| *** marcusvrn1 has quit IRC | 16:41 | |
| *** marcusvrn1 has joined #openstack-infra | 16:41 | |
| SpamapS | mordred: indeed. a) Don't do that! b) scream and yell when dupes are detected c) Add cloud name when dupes are detected. | 16:42 |
| *** marcusvrn has quit IRC | 16:42 | |
| *** marcusvrn_ is now known as marcusvrn | 16:42 | |
| mordred | SpamapS: hang on - it's a really important usecase | 16:42 |
| *** e0ne has quit IRC | 16:43 | |
| SpamapS | I like c, but I've been accused of favoring fail-open in the past.. which woudl we bad if I ever took over designing nuclear plants. | 16:43 |
| ameade | jeblair: hey did you see my response about the netapp-ci? please let me know if you need any more info | 16:44 |
| *** freerunner has joined #openstack-infra | 16:44 | |
| mordred | SpamapS: which is to reverse the hostanme/instance-$uuid group creation we're doing right now - and to return the 'hostname' as 'instance-$uuid' and then make a group for each hostname | 16:45 |
| mordred | SpamapS: that way, referring to a hostname will work like you expect - and if you need to reference a specific instance that shares a name, you have that ability | 16:45 |
| SpamapS | mordred: That is the more accurate solution for sure. | 16:45 |
| mordred | SpamapS: important use case is that we have tons of pets aroudn here | 16:45 |
| mordred | and that's not likely to ever change | 16:46 |
| SpamapS | Its ok, I'm not coming for your pets. I just don't want to adopt any more. :) | 16:46 |
| mordred | :) | 16:46 |
| *** patrickeast has quit IRC | 16:46 | |
| SpamapS | I specifically wrote language last night in my infra-cloud doc that points out it's not a requirement of infra-cloud to run pets. :) | 16:47 |
| *** wolsen_ is now known as wolsen | 16:47 | |
| * SpamapS should finish that and get it up for review. :_P | 16:47 | |
| mordred | SpamapS: well, it may need to run a some not-very-loved pets | 16:47 |
| mordred | SpamapS: for instance, we run pypi mirrors in each nodepool cloud region | 16:47 |
| SpamapS | thats not really a pet | 16:48 |
| mordred | SpamapS: they're not 100% ephemeral - but we don't create/delete them frequently | 16:48 |
| mordred | yah | 16:48 |
| SpamapS | It's expensive cattle. | 16:48 |
| SpamapS | There's nothing special about each mirror other than you only need one. | 16:48 |
| SpamapS | Some day somebody should figure out how to mirror pypi onto swift really. | 16:48 |
| SpamapS | and git | 16:49 |
| SpamapS | and all the things | 16:49 |
| mordred | SpamapS: well, that's actually the opposite direction from what we've been looking at | 16:49 |
| SpamapS | running devpi? | 16:49 |
| *** marun has quit IRC | 16:49 | |
| mordred | god no | 16:49 |
| mordred | because it would increase rather than decrease the exposure to non-atomic updates | 16:49 |
| mordred | devpi not so much with being scalable | 16:50 |
| *** shashankhegde has quit IRC | 16:50 | |
| SpamapS | Yeah swift can't do rsync/btrfs-cow-snap+symlink magic can it? | 16:50 |
| mordred | SpamapS: although, I DO agree that mirroring in to swift is a general thing that people shold figure out for this | 16:50 |
| *** marcusvrn2 has joined #openstack-infra | 16:51 | |
| *** shashankhegde has joined #openstack-infra | 16:51 | |
| fungi | SpamapS: keep in mind that the design of our pypi mirrors is mostly intended to mitigate the issue that networking in and at the edge of clouds sucks | 16:51 |
| jeblair | ameade: we think that the problem has been going on for some time (longer than apr 29), but we didn't have the info in place to determine what accounts were involved. | 16:51 |
| *** abregman has quit IRC | 16:51 | |
| SpamapS | fungi: typically object storage has a fairly fat pipe to compute .. unless you're doing it wrong. | 16:51 |
| mordred | SpamapS: well, our current plan for atomic updates is based on afs volume publishing, rather than btrfs - but yes, that is the nature of the problem with using something like swift for this specific usecase | 16:51 |
| *** marun has joined #openstack-infra | 16:51 | |
| mordred | SpamapS: not to object storage | 16:51 |
| mordred | SpamapS: the problem that we're solving is that network access to pypi things outside of the cloud is bad | 16:52 |
| jeblair | ameade: and in general, it should be fine to leave the stream event ssh connections open, but i could see the possibility that if your local script stopped reading from them, gerrit might hang on a write call if it isn't select()ing properly for writes | 16:52 |
| jeblair | ameade: so i guess that sounds like a good starting point -- maybe we should re-enable the account then and see if it happens again? | 16:52 |
| SpamapS | mordred: right, I got discombobulated in that context. | 16:52 |
| *** Ala has quit IRC | 16:53 | |
| *** marcusvrn1 has quit IRC | 16:53 | |
| jeblair | SpamapS, mordred: (maybe swift with two containers and some dns switching or something) | 16:54 |
| jeblair | but yeah, honestly it'd be better if that were _in_ swift | 16:54 |
| jeblair | container symlink? | 16:54 |
| jeblair | or cow/release | 16:55 |
| *** crc32 has joined #openstack-infra | 16:55 | |
| openstackgerrit | Fabien Boucher proposed openstack-infra/system-config: Let puppet-zuul manage log configuration files https://review.openstack.org/180624 | 16:55 |
| ameade | jeblair: yeah i think that is the only way to be sure, the script was indeed running before april 29th, it just got restarted then | 16:56 |
| *** claudiub has joined #openstack-infra | 16:56 | |
| *** sdake_ has joined #openstack-infra | 16:56 | |
| ameade | jeblair: it's at least easier for yall to detect now right? | 16:56 |
| *** gordc has quit IRC | 16:57 | |
| *** peristeri has quit IRC | 16:57 | |
| openstackgerrit | Renat Akhmerov proposed openstack-infra/project-config: Rename stackforge/mistral to openstack/mistral https://review.openstack.org/175328 | 16:58 |
| SpamapS | jeblair: I wonder if CEPH allows CoW via the swift frontend. | 16:58 |
| mordred | jeblair, SpamapS: I could see something like that potentially working - it's really atomic update of the index file that you care about - especially if you handle purges in a lazy manner | 16:58 |
| jeblair | ameade: well, unfortunately we tend to only notice when it goes belly-up; but yeah, we can associate the problem connections with an account now | 16:58 |
| SpamapS | like, "snapshot this container, now expose it as a new container, now atomically replace the old one with the new one" | 16:58 |
| jeblair | ameade: okay, it's re-enabled; let's give it a shot | 16:59 |
| ameade | jeblair: awesome, thanks and sorry for the pain | 17:00 |
| *** sdake has quit IRC | 17:00 | |
| *** hasharMeeting has quit IRC | 17:00 | |
| fungi | jeblair: if you want to compare the jstack trace i have from a couple weeks ago with a contemporary connections and queue list (from shortly after we got your patch onto review.o.o successfully), let me know. i still have it all | 17:01 |
| *** derekh_ has quit IRC | 17:01 | |
| openstackgerrit | Alec Hothan proposed openstack-infra/project-config: Need to allow merge commits in vmtp.git https://review.openstack.org/180616 | 17:01 |
| *** shashankhegde has quit IRC | 17:02 | |
| *** pc_m has joined #openstack-infra | 17:02 | |
| jeblair | fungi: thanks, i'm pretty sure i recognized the stack traces as the usual one we've been seeing | 17:02 |
| *** xyang1 has quit IRC | 17:02 | |
| fungi | jeblair: i meant as far as correlating to who the stuck account was from them as well | 17:03 |
| *** xyang1 has joined #openstack-infra | 17:03 | |
| fungi | or i'll see if i can map it up myself from what's there | 17:03 |
| *** shashankhegde has joined #openstack-infra | 17:04 | |
| *** ayoung has quit IRC | 17:04 | |
| jeblair | fungi: oh, i thought you said it was the same? | 17:04 |
| fungi | jeblair: no, from a prior similar incident while you were travelling | 17:04 |
| *** tonytan4ever has quit IRC | 17:04 | |
| fungi | jeblair: april 29 | 17:04 |
| fungi | so i guess ~week ago | 17:05 |
| *** SumitNaiksatam has quit IRC | 17:05 | |
| *** xyang1 has quit IRC | 17:05 | |
| pc_m | Anyone know of a way to copy some files with commit history from one repo to an existing stackforge repo? | 17:06 |
| openstackgerrit | Sirushti Murugesan proposed openstack-infra/project-config: Add grenade jobs for Heat https://review.openstack.org/178352 | 17:07 |
| fungi | jeblair: on review.o.o in my homedir there are three jstack traces (jstack.1 through .3) taken ~10-20 seconds apart while stream-events was hung, and i just copied files named show-connections and show-queue up there which were redirects from stdout taken in the same timeframe | 17:07 |
| *** gyee has joined #openstack-infra | 17:08 | |
| jeblair | fungi: yeah, so all the stream events workers were servicing netapp-ci | 17:08 |
| *** martingc has left #openstack-infra | 17:09 | |
| *** patrickeast has joined #openstack-infra | 17:10 | |
| anteaya | pc_m: not easily, what has happened that you are in this situation | 17:10 |
| fungi | that seems like a good correlation with yesterday in that case | 17:10 |
| *** sdake has joined #openstack-infra | 17:10 | |
| jeblair | yolanda, clarkb: so when i look at 161663, i actually wonder how much jeepyb stuff should be in the gerrit module? | 17:10 |
| pc_m | anteaya: There are some vendor plugin files in VPN repo that I'd like to move out to stackforge, but would like to preserve history if possible. | 17:11 |
| clarkb | jeblair: you mean collapse the jeepyb module into gerrit? | 17:11 |
| jeblair | yolanda, clarkb: does that belong in openstack_modules/gerrit or puppet-openstackci, etc...? i suppose it depends on how much we think jeepyb is actually required to use gerrit | 17:11 |
| jeblair | clarkb: no, the opposite | 17:11 |
| mordred | jeblair: move things rfom gerrit into jeepyb module | 17:11 |
| jeblair | no let me start over | 17:11 |
| anteaya | pc_m: to a new stackforge repo, easy to do, to an existing stackforge repo tricky to impossible | 17:12 |
| jeblair | 161663 is a change to puppet-gerrit that invokes code in puppet-jeepyb. that means puppet-gerrit knows about jeepyb and i'm not sure it should | 17:12 |
| cinerama | pleia: hi there | 17:12 |
| *** yamahata has quit IRC | 17:12 | |
| clarkb | jeblair: I see, | 17:12 |
| anteaya | pc_m: the impossible part being the amount of infra support you would need to get there | 17:12 |
| cinerama | pleia2: so i added some bugs, but i feel like i'm missing one | 17:12 |
| pc_m | anteaya: existing repo, where there are already other drivers/plugins. | 17:12 |
| clarkb | jeblair: basically puppet-gerrit and puppet-jeepyb should be invoked at the same level, but not nested like that | 17:12 |
| clarkb | jeblair: I think that makes sense to me | 17:12 |
| pleia2 | cinerama: added bugs to buzilla? | 17:13 |
| jeblair | ya, that's my gut reaction. i haven't actually traced things out to make sure it works :) | 17:13 |
| mordred | jeblair: ++ | 17:13 |
| pc_m | anteaya: can you elaborate on what you mean by "infra support"? | 17:13 |
| cinerama | pleia2: this thing is kind of weird. so i thought we couldn't add e.g. zh-tw but it seems to detect it; it just doesn't appear in the little dropdown thing | 17:13 |
| cinerama | pleia2: yep. i added them to the etherpad | 17:13 |
| anteaya | pc_m: well you would have to convince fungi or one of the others to walk you through the process | 17:13 |
| cinerama | pleia2: i feel like there is one more bug we discussed last night that needs to be added | 17:13 |
| anteaya | pc_m: but after we did this in december with the *aas repos we said we wouldn't do it again | 17:13 |
| pleia2 | cinerama: huh, that's weird | 17:13 |
| anteaya | pc_m: it isn't a supported workflow | 17:13 |
| *** sdake__ has joined #openstack-infra | 17:14 | |
| pleia2 | cinerama: I'll re-browse logs to see if I can dig it up | 17:14 |
| *** sdake_ has quit IRC | 17:14 | |
| cinerama | pleia2: you can try it for yourself if you go to add language. also you can add totally random locales that make no sense | 17:14 |
| pleia2 | cinerama: oh goodie, testing... | 17:14 |
| pc_m | anteaya: I'm looking at this purely as a move of modules from VPN repo to stackforge. I know I can just copy the files and then remove from vpn repo, and I guess the history will still be there in neutron-vpnaas, but would like to have history in stackforge. | 17:14 |
| fungi | pc_m: you won't be able to put the additional files through code review and still retain their original history. there are ways to directly manipulate the files in the repository "behind gerrit's back" so to speak, but they involve having permission to push commits past gerrit | 17:15 |
| openstackgerrit | Samuel de Medeiros Queiroz proposed openstack-infra/devstack-gate: Remove arbitrary devstack config options https://review.openstack.org/179982 | 17:15 |
| pc_m | anteaya: Not looking for infra support, other than ideas on what my options are and how I could do it. | 17:15 |
| sdake__ | hey quick q relating to creating a branch | 17:15 |
| sdake__ | I submitted a change request to add https://github.com/openstack-infra/project-config/blob/master/gerrit/acls/stackforge/kolla.config#L3 | 17:15 |
| sdake__ | yet rgerrit doesn't have a create branch button | 17:15 |
| anteaya | pc_m: split the code from the existing repo and create a new stackforge repo with teh new split out code | 17:15 |
| anteaya | pc_m: that is the supported workflow | 17:16 |
| *** dizquierdo has joined #openstack-infra | 17:16 | |
| pleia2 | cinerama: I don't understand this add language UI at all | 17:16 |
| *** achanda has joined #openstack-infra | 17:17 | |
| fungi | pc_m: basically our gerrit is not set up in such a way as to allow projects to use the full flexibility of git when it comes to repository management. it's implemented with the intent of preventing anyone from bypassing code review for changes to a repository it manages. those two goals tend to be in conflict with one another at times | 17:17 |
| * pleia2 seeks docs | 17:17 | |
| pc_m | anteaya: How is the commit history preserved? | 17:17 |
| mordred | fungi: I disagree | 17:17 |
| mordred | there is no reason a stackforge project coudl not grant itself permissions to do direct push to support something like this | 17:17 |
| cinerama | pleia2: file a bug | 17:17 |
| cinerama | pleia2: :) | 17:18 |
| *** sdake has quit IRC | 17:18 | |
| mordred | we don't enforce policy on stackforge folks | 17:18 |
| clarkb | jeblair: I am going to take your gearman-plugin change, stack mine on it locally, and test the pair with my test setup. If that works I move for merging both changes and restarting one master with them | 17:18 |
| fungi | mordred: you disagree that our current gerrit implementation is designed to support projects recombining bits and pieces from multiple existing git repositories? | 17:18 |
| pc_m | fungi: forgive my ignorance on this... if the destination is a vendor repo, is there some way to bring in the full history? | 17:18 |
| cinerama | pleia2: but seriously, yes, it is kind of strange. i see the possible intent (namely, let the user add the language with the name they want) | 17:18 |
| jeblair | mordred: we don't what? | 17:18 |
| pleia2 | cinerama: nods, going to see if I can figure out how they think it works, and tell them that my brain explodes on it | 17:18 |
| clarkb | jeblair: that seem sane? and while that is going on I can make my use CANT_DO patch better | 17:18 |
| cinerama | pleia2: but if we can't delete stuff, that is a recipe for having lots of dupe entries | 17:18 |
| pc_m | fungi: Or some way to replay the commits? | 17:19 |
| fungi | mordred: i suppose we could just grant push --force permissions in stackforge repo acls and tell the devs using those that they're on their own if they shoot themselves in the foot with it | 17:19 |
| clarkb | so I wrote the initial "this is stackforge" doc on ci.openstack.org | 17:19 |
| jeblair | fungi, mordred: please no | 17:19 |
| clarkb | it explicitly says "it is a way to use our tools and process" | 17:19 |
| jeblair | clarkb: yes | 17:19 |
| clarkb | if people want force push use github | 17:19 |
| sdake__ | hey folks, I'm in the kolla release group here: https://review.openstack.org/#/admin/groups/461,members but I can't create a branch in gerrit | 17:19 |
| sdake__ | any suggestions? | 17:19 |
| mordred | wow. ok. that's not what I'm suggesting | 17:19 |
| mordred | I'm suggesting that we have two conflicting things here | 17:19 |
| mordred | one is that for a project to be able to slurp in some additional history usually requires one of us to do a push to a ref location as admins | 17:20 |
| mordred | and we don't like doing admin things like that for stackforge projects if we can avoid it | 17:20 |
| mordred | so - one of the solutions to that woudl be for the stackforge project to self-service the sequence of actions needed to accomplish such a thing | 17:20 |
| fungi | pc_m: you could repropose each non-merge commit in a branch created by using git filter-branch to split out the history for the files you want, as a way to retain their original authorship information | 17:20 |
| *** HeOS has quit IRC | 17:21 | |
| mordred | including temporaryily granting permisions to push to a ref and then revoking it after. or we could do a push for someone. | 17:21 |
| fungi | pc_m: but it wouldn't preserve other related parts of the git history such as timestamps, review details, et cetera | 17:21 |
| mordred | it's not that it's not possible | 17:21 |
| pc_m | anteaya: fungi: Let me try to rephrase the question... If I have vendor service and driver modules in the neutron-vpnaas repo, and I want to move those files to a stackforge repo (networking-cisco), what options do I have and what are the consequences of the options? | 17:21 |
| cinerama | pleia2: i'm kind of wondering how we will test the proposal scripts | 17:21 |
| cinerama | pleia2: suggestions are welcome | 17:22 |
| *** gyee has quit IRC | 17:22 | |
| *** shardy has quit IRC | 17:22 | |
| jeblair | pc_m: do the two git repos share a common git history (perhaps far in the past?) | 17:22 |
| *** otter768 has joined #openstack-infra | 17:22 | |
| *** dizquierdo has quit IRC | 17:23 | |
| fungi | pc_m: you could copy those files from one repo to the other and propose it as a new change to the destination repo, followed by a change to delete them from the original repo (and include some provenance details in the commit message, such as co-authored-by lines and original commit shas). you lose all actual git metadata that way, but it requires no modification to current acls at least | 17:23 |
| *** amitgandhinz has joined #openstack-infra | 17:23 | |
| pleia2 | cinerama: the actual commit-to-gerrit bit? | 17:23 |
| sdake__ | for kolla shouldn't I be able to create a branch here: https://review.openstack.org/#/admin/projects/stackforge/kolla | 17:23 |
| sdake__ | I am in the create group in project-config | 17:23 |
| clarkb | sdake__: https://review.openstack.org/#/admin/projects/stackforge/kolla,branches there | 17:24 |
| pc_m | jeblair: Well... the neutron-vpnaas repo came from neutron repo. The networking-cisco repo came from neutron and has ML2 stuff in it I think. | 17:24 |
| sdake__ | clarkb thanks! | 17:24 |
| cinerama | pleia2: yeah. i guess i can kind of comment the gerrit bits out and have at it | 17:24 |
| pleia2 | cinerama: we can use the sandbox repo https://git.openstack.org/cgit/openstack-dev/sandbox | 17:24 |
| clarkb | sdake__: the link is in the top panel tab thing fwiw | 17:24 |
| cinerama | pleia2: that might be a good idea | 17:25 |
| sdake__ | clarkb I guess I missed that :) | 17:25 |
| *** sabeen1 has quit IRC | 17:25 | |
| *** ivar-lazzaro has joined #openstack-infra | 17:26 | |
| pc_m | fungi: Do you mean, in the new commit, refer to the shas from the old repo's modules? | 17:26 |
| fungi | yeah, if you wanted to retain some paper trail at least | 17:27 |
| *** SumitNaiksatam has joined #openstack-infra | 17:27 | |
| *** tonytan4ever has joined #openstack-infra | 17:27 | |
| *** otter768 has quit IRC | 17:27 | |
| fungi | pc_m: you could also git filter-branch the files you want to move into a separate branch, pull it into a new branch in a local clone of the target repository, then rebase those changes onto the intended branch and repropose the result to code review (which would have the effect of preserving the authorship info, though not really much else) | 17:27 |
| asselin__ | lifeless, so I submitted these 2 patches, but I thought you said you audited pbr settings for upstream? https://review.openstack.org/#/c/180626/ https://review.openstack.org/#/c/180622/ | 17:27 |
| *** ivar-lazzaro has quit IRC | 17:27 | |
| pleia2 | cinerama: so it seems like the add translations interface is built around adding a language that already exists in their database, right? how did you add different cased en-au? | 17:27 |
| pc_m | fungi: The copying was what I was originally thinking. I was just wondering if the new repo could maintain the git history too. | 17:28 |
| *** boris-42 has quit IRC | 17:28 | |
| *** ivar-lazzaro has joined #openstack-infra | 17:28 | |
| cinerama | pleia2: you just hit save on whatever you type rather than selecting from the dropdown | 17:29 |
| clarkb | oh sweet the stop sending status updates change has laredy merged, makes my testing a little easier | 17:29 |
| jeblair | woo! | 17:29 |
| *** marcusvrn has quit IRC | 17:29 | |
| jeblair | clarkb: let me know if that doesn't work ;) | 17:29 |
| fungi | pc_m: to retain the git history, you'd need to do something similar but merge the new branch into the existing one and push --force it into gerrit (and cross your fingers that it works, because you're bypassing code review testing) | 17:30 |
| clarkb | jeblair: will do | 17:30 |
| pleia2 | cinerama: I see, and if you create one that's not in their database, it autofills info below... but gives no way to edit it | 17:30 |
| pc_m | fungi: 99% of the files were authored by me, so I'm not too worried about that. More of just an interest to have the commit history, in case people wanted to know why changes were made. | 17:30 |
| cinerama | pleia2: yup. so you can have lots of spam | 17:30 |
| pleia2 | cinerama: could you find a way to delete them? | 17:30 |
| cinerama | pleia2: also i did something weird to my project i was testing on and i can't fix it | 17:30 |
| cinerama | pleia2: no, that's a different bug i filed :) | 17:31 |
| pleia2 | aha :) | 17:31 |
| cinerama | pleia2: see if you can reproduce this: go on your project and find the thing where you can manage languages | 17:31 |
| cinerama | pleia2: there is a choice (i forget what now) where you can limit the languages | 17:31 |
| cinerama | pleia2: but if you pick it IIRC the state looks the same the next time you visit the dialog but getting the xml from rest won't grab all the languages | 17:32 |
| jeblair | mordred: why did you not remove python-yaml in 167610? | 17:33 |
| pc_m | fungi: on the new branch option... would an empty directory structure be added to the target repo and committed as the first commit, and then branch to do the rest? Trying to understand how to stage that in the new repo. | 17:33 |
| pleia2 | cinerama: so there's a "Inherit languages from project settings." checkbox | 17:34 |
| *** AJaeger_ has joined #openstack-infra | 17:35 | |
| clarkb | jeblair: I am still seeing a WORK_STATUS between GRAB_JOB and SUBMIT_JOB, I assume that is expected since the statuses that were removed happen after SUBMIT_JOB | 17:35 |
| fungi | pc_m: filter-branch creates a new branch in the starting repository, then you'd want to pull that branch into the target repository, probably as a new branch so you can have a little more flexibility on how you combine it with the intended existing branch there | 17:35 |
| *** ivar-laz_ has joined #openstack-infra | 17:35 | |
| cinerama | pleia2: yes, i think once ticked, there is no going back | 17:35 |
| asselin__ | jeblair, not sure we can +w this yet...https://review.openstack.org/167426 | 17:36 |
| jeblair | clarkb: er, i expect: >GRAB_JOB_UNIQ <JOB_ASSIGN_UNIQ >WORK_STATUS >WORK_COMPLETE | 17:36 |
| *** mattfarina has joined #openstack-infra | 17:36 | |
| jeblair | asselin__: depends on should dtrt | 17:36 |
| *** neoice_ is now known as neoice | 17:36 | |
| mordred | jeblair: hrm. I feel like there was a reason I did not | 17:36 |
| fungi | pc_m: pull may be a poor choice of wording there. you'd want to add the starting repository as a remote and then checkout that remote as a new local branch in the target repo | 17:36 |
| asselin__ | jeblair, dtrt? | 17:36 |
| pleia2 | cinerama: wow, yeah, that's a mess | 17:36 |
| asselin__ | ok...do the right thing | 17:36 |
| jeblair | asselin__: do the right thing. sorry -- the depends-on header in the change i +Ad will prevent it from actually landing until the one it depends on lands. and zuul should actually correctly enqueue it automatically now. | 17:37 |
| fungi | pc_m: where in this case the "remote" repo is likely a direct file path or file:// url to your local surgery in the originating repo | 17:37 |
| clarkb | jeblair: geard doesn't seem to log the JOB_ASSIGN_UNIQ let me find a good chunk of logs to paste | 17:37 |
| jeblair | asselin__: so basically i said "i'm comfortable landing this as soon as the dependency lands, especially sinc zuul will still test it one last time right before it lands" | 17:38 |
| *** pelix has quit IRC | 17:38 | |
| pc_m | fungi: I think I got it. thanks. | 17:38 |
| mordred | jeblair: nope. I think you're righ t- I think it should come out | 17:38 |
| pleia2 | cinerama: want me to submit this bug? | 17:38 |
| *** ivar-lazzaro has quit IRC | 17:38 | |
| jeblair | asselin__: that way i can knock it off my list and we don't have to worry about remembering it later. | 17:38 |
| cinerama | pleia2: please do. it's a stumper | 17:38 |
| pleia2 | k | 17:38 |
| openstackgerrit | Monty Taylor proposed openstack-infra/puppet-nodepool: Consume puppet-diskimage_builder https://review.openstack.org/167610 | 17:38 |
| mordred | jeblair: thanks | 17:38 |
| *** btully has joined #openstack-infra | 17:38 | |
| asselin__ | jeblair, ok thanks. I will see what's left on the depends on so we can get that pushed through too | 17:38 |
| pc_m | fungi: Any suggestions on which of those approaches would make the most sense to attempt (given the conditions: existing repos, me as author on files)? | 17:39 |
| *** emagana has joined #openstack-infra | 17:39 | |
| clarkb | jeblair: oh I may have mixed some of the streams, noticing the ports are different now, http://paste.openstack.org/show/215482/ | 17:40 |
| jeblair | mordred: i see a lot of dib nodepool patches in merge conflict; what's the skinny? | 17:40 |
| clarkb | jeblair: I think that looks correct to me if I account for the ports being different workers | 17:40 |
| mordred | jeblair: I need to make another pass through the | 17:40 |
| fungi | pc_m: part of that determination depends on how many non-merge commits you end up with in the filter-branch result, and whether you really need to preserve the original timestamps | 17:40 |
| clarkb | jeblair and submits coming from the clients | 17:40 |
| jeblair | clarkb: ah yeah, crossing streams bad | 17:40 |
| mordred | jeblair: I'll get that cleaned up today | 17:40 |
| jeblair | mordred: ok, wasn't sure if we were waiting on something | 17:40 |
| fungi | pc_m: and also if you have multiple branches (for example stable branches) you need to do similar things to | 17:41 |
| AJaeger_ | cinerama: I tested the proposal scripts locally | 17:41 |
| pc_m | fungi: Is there a way I can introspect that on the files in the neutron-vpnaas repo? | 17:41 |
| AJaeger_ | cinerama: I basically comment out the function for sending the patch to gerrit and also some setup - and then run locally in my environment | 17:41 |
| cinerama | AJaeger_: ok sounds good | 17:41 |
| *** alexpilotti has quit IRC | 17:41 | |
| fungi | pc_m: i believe git log <some filename> will give you commit logs relevant to a specific file, without having to dig into how to actually split those commits out first | 17:41 |
| AJaeger_ | Once that works fine, I send the patch and monitor that everything really works as it should | 17:42 |
| jeblair | clarkb: ah, yeah, forgot initial work_data. looks good though. | 17:42 |
| asselin__ | clarkb, perhaps you can readd your +2 to this and we can merge? https://review.openstack.org/#/c/167425/ | 17:42 |
| AJaeger_ | cinerama: So, checking https://review.openstack.org/#/q/status:open++branch:master+topic:transifex/translations,n,z and the latest builds here https://jenkins.openstack.org/ | 17:42 |
| AJaeger_ | So, I review all failures for the proposal jobs and the content of the send patches... | 17:43 |
| AJaeger_ | cinerama: The proposal job runs every morning at 6:00 UTC | 17:43 |
| jeblair | nibalizer: did you ever have that conversation about your patch? | 17:43 |
| *** Sukhdev has quit IRC | 17:43 | |
| clarkb | asselin__: I would need to rereview it | 17:43 |
| *** eharney has quit IRC | 17:43 | |
| fungi | AJaeger_: what was the name of the repo for the tag jobs you wanted triggered? | 17:44 |
| AJaeger_ | fungi: openstack-doc-tools | 17:44 |
| AJaeger_ | fungi: version 0.26 | 17:44 |
| asselin__ | clarkb, yes, ofcourse, rereview, and then readd :) | 17:44 |
| zaro | morning | 17:44 |
| nibalizer | jeblair: no, clarkb and I haven't circled around on that yet | 17:44 |
| fungi | AJaeger_: thanks, starting on that now | 17:44 |
| AJaeger_ | thanks a lot, fungi! | 17:44 |
| anteaya | morning zaro | 17:44 |
| jeblair | nibalizer: i was just about to review changes that i think might conflict with it and i thought it might be nice not to merge those. :) | 17:45 |
| nibalizer | clarkb: got time today to lay out your plan? | 17:45 |
| nibalizer | jeblair: i appreciate that | 17:45 |
| clarkb | nibalizer: I thought I wrote it down somewhere, basic idea was move each logical chunk of the file at a time | 17:45 |
| clarkb | nibalizer: to reduce the number of variables that can break | 17:46 |
| clarkb | and since that file is already organized into chunks it should be simple to find those boundaries | 17:46 |
| *** vhoward has left #openstack-infra | 17:46 | |
| jeblair | clarkb: do you think that will reduce the overall risk? or does that just mean we have 10 chances to break everything? :) | 17:46 |
| *** alexpilotti has joined #openstack-infra | 17:46 | |
| pc_m | fungi: Overall, we're talking about 5 files and 3 test files. | 17:46 |
| *** gyee has joined #openstack-infra | 17:46 | |
| *** Sukhdev has joined #openstack-infra | 17:47 | |
| clarkb | jeblair: I think it confines the risk, last time we did this every single host was affected, but in this case we can make the centos changes and make sure the git mirror is happy before moving on and making the afs changes and so on | 17:47 |
| *** freerunner has quit IRC | 17:47 | |
| nibalizer | https://review.openstack.org/#/c/171359/ is the review | 17:47 |
| jeblair | clarkb, nibalizer: (this will be so much nicer when we have high-level functional testing, but we have what we have now :|) | 17:47 |
| clarkb | jeblair: also I can trigger jobs and zaro's test case of offline node, submit job, online node is working. So I think I am about as ready on my change as I will be | 17:47 |
| jeblair | clarkb: re gearman: cool | 17:48 |
| *** Swami has joined #openstack-infra | 17:48 | |
| *** craigbr has quit IRC | 17:48 | |
| nibalizer | so one thing to note is that from the perspective of doing it, and I know gerrit doesn't show it that way, is that its a 10 line change :) | 17:48 |
| clarkb | nibalizer: sort of, | 17:48 |
| clarkb | nibalizer: you have to keep in mind that if you have a stray include somewhere then its a 300 line change | 17:48 |
| nibalizer | what has happened is 10 lines from one file has moved into another file, then the file was renamed to the first file | 17:49 |
| *** freerunner has joined #openstack-infra | 17:49 | |
| clarkb | nibalizer: yes but names matter in puppet | 17:49 |
| nibalizer | so this file is not very big http://git.openstack.org/cgit/openstack-infra/system-config/tree/modules/openstack_project/manifests/server.pp | 17:50 |
| nibalizer | the english version of the change is to put that exim resource (behind a feature flag) into template | 17:50 |
| *** alexpilotti_ has joined #openstack-infra | 17:50 | |
| clarkb | but do so with the server name | 17:50 |
| *** e0ne has joined #openstack-infra | 17:50 | |
| nibalizer | so one idea would be to do moving exim behind a feature flag in one change, then just switch servers one at a time to use template directly instead of server | 17:51 |
| nibalizer | then server gets deleted ? | 17:51 |
| *** alexpilotti has quit IRC | 17:51 | |
| *** alexpilotti_ is now known as alexpilotti | 17:51 | |
| zaro | clarkb: i have tried with jjb and looks good | 17:52 |
| jeblair | nibalizer: that sounds good, and i think would reduce overall risk | 17:52 |
| jeblair | (at the expense of probably a bit more infra-root time) | 17:53 |
| clarkb | and considering we flattened base into template that makes sense to me | 17:53 |
| *** ZZelle has quit IRC | 17:53 | |
| clarkb | zaro: thanks | 17:53 |
| *** kgiusti has quit IRC | 17:53 | |
| *** ZZelle has joined #openstack-infra | 17:53 | |
| jeblair | (i'd probably still rename template->server or something when all is completed; template still means "nodepool template" to me) | 17:54 |
| clarkb | zaro: https://review.openstack.org/#/c/180371/ is a followup change to make function registration quicker, but it doesn't work quite right yet. I left a paste with the geard logs showing it unregistering some functions that should remain registered (I think) | 17:54 |
| fungi | mordred: lifeless: we have a new pbr challenge i think... AJaeger_ pushed a "0.26" tag to openstack/openstack-doc-tools which caused pbr to generate wheel and sdist files with 0.26.0 in their names, but our pypi upload script expects that to match the tag | 17:55 |
| clarkb | fungi: I think we need to enforce semver's three tuple in zuul layout now | 17:55 |
| fungi | curious as to what you think is the best workaround for that | 17:55 |
| jeblair | nibalizer, clarkb: do we have a plan? | 17:55 |
| clarkb | jeblair: nibalizer ya that proposal sounds good to me | 17:56 |
| *** cody-somerville has quit IRC | 17:57 | |
| *** tim_o has quit IRC | 17:57 | |
| AJaeger_ | fungi: It wasn't me that actually pushed the tag ;) but I asked for it, so I'm still guilty ;) And that's how we did it for some time. | 17:57 |
| AJaeger_ | fungi, should I ask for a 0.26.0 tag to get pushed instead? | 17:58 |
| AJaeger_ | Or do we need 0.26.1 now? | 17:58 |
| pc_m | fungi: So it looks like all the files have an original committer of dougwig, as he created the *aaS repos, the second version are all owned by me. | 17:58 |
| *** signed8bit has joined #openstack-infra | 17:58 | |
| fungi | AJaeger_: yep, not at all anyone's fault, this is an unanticipated interaction between our release automation tooling and the latest pbr release | 17:58 |
| mordred | fungi, clarkb: I think enforcing the 3 tuple is a good idea | 17:59 |
| fungi | AJaeger_: i _think_ a 0.26.0 tag will work and replace the existing tarballs on tarballs.o.o and upload normally | 17:59 |
| AJaeger_ | fungi: ok, will ask loquacities and annegentle for it. | 17:59 |
| clarkb | should check it before pushing it though | 17:59 |
| clarkb | do a `python setup.py sdist` and check the output | 17:59 |
| jeblair | nibalizer: if that sounds good to you and you think we can get those merged by end of next week, i'll procedural-2 changes that touch server.pp and template.pp; need your ack first though. | 18:00 |
| fungi | clarkb: AJaeger_: right. and even pip install the resulting tarball into a fresh virtualenv for a sanity check | 18:00 |
| nibalizer | jeblair: works for me | 18:01 |
| nibalizer | ill spin those patches up today | 18:01 |
| jeblair | (ooh -- new gertty feature idea: apply the same review to a set of changes) | 18:01 |
| nibalizer | jeblair: call it 'the hammer' | 18:01 |
| fungi | clarkb: mordred: i'll spin up a straw man change to the (pre-)release regex for layout.yaml now | 18:01 |
| nibalizer | only when minus reviews tho | 18:01 |
| crinkle | jeblair: the stackforge modules would use that :) | 18:01 |
| mordred | fungi: woot | 18:02 |
| AJaeger_ | fungi, clarkb: So, installing pbr 0.11 first? | 18:03 |
| clarkb | AJaeger_: if you use a fresh venv ou should get that version | 18:04 |
| fungi | AJaeger_: tox -e venv python setup.py sdist | 18:04 |
| clarkb | locally it works if I tag 0.26.0 and sdist and pip install . | 18:04 |
| fungi | AJaeger_: that's how the job is building the tarball | 18:04 |
| AJaeger_ | clarkb: yeah, that's what I did as well... | 18:05 |
| AJaeger_ | fungi, clarkb: Thanks | 18:05 |
| fungi | AJaeger_: git clean -dfx first or add -r to the tox invocation to make sure you get latest pbr | 18:05 |
| fungi | AJaeger_: you can also double-check after by running .tox/venv/bin/pip list | 18:05 |
| *** ZZelle_ has joined #openstack-infra | 18:06 | |
| jeblair | nibalizer: amusingly i -2d another of your changes :) | 18:06 |
| pc_m | fungi: I did git log of all the files and there is a total of 17 unique commits for all the files. | 18:06 |
| AJaeger_ | thanks, fungi. Looks fine: "openstack-doc-tools (0.26.0, /home/aj/Software/vcs/OpenStack/openstack/openstack-doc-tools)" | 18:07 |
| jeblair | crinkle: what's the use case for the stackforge modules? | 18:07 |
| pleia2 | cinerama: ok, so I figured out what it's doing, it does actually go back to including all 40 languages when you check "Inherit languages from project settings." again (see the top of the screen after nagivating away from that tab in the UI), but when you uncheck it again, it preserves the shorter list, I wouldn't have expected this behavior (I'd expect it to go back to listing all 40) | 18:07 |
| clarkb | asselin__: done, the delta was small so it was easy :) | 18:07 |
| anteaya | pc_m: would you consider adding the git log as a readme file and copying the code as new files to the stackforge repo? | 18:08 |
| asselin__ | clarkb, \0/ | 18:08 |
| AJaeger_ | clarkb, fungi : So that looks sane - thanks! | 18:08 |
| crinkle | jeblair: sometimes we have to emergency pin a bunch of gems across 20 modules, so getting to +2 on a bunch of patches would be neat | 18:08 |
| openstackgerrit | Clint 'SpamapS' Byrum proposed openstack-infra/shade: Fix delete_server when wait=True https://review.openstack.org/180659 | 18:08 |
| anteaya | pc_m: it seems like a lot of work to preserve the history of 17 commits | 18:08 |
| pc_m | fungi: I'm wondering if it is best to just copy as new files and reference the old repo shas | 18:08 |
| pc_m | anteaya: Yeah, seems like it. | 18:08 |
| *** freerunner has quit IRC | 18:08 | |
| anteaya | pc_m: will save you a ton of work and pain | 18:09 |
| fungi | pc_m: pragmatism ftw! | 18:09 |
| jeblair | crinkle: ah cool. i think it should be doable; i'll put it on the backlog | 18:09 |
| cinerama | pleia2: oh weird | 18:09 |
| pc_m | anteaya: Would it be better to put those commits into a README, or include them in the commit message? | 18:09 |
| clarkb | jeblair: do you want me to approve my gearman-plugin change before or after running it on say jenkins07? | 18:09 |
| *** markmcclain has quit IRC | 18:09 | |
| clarkb | jeblair: also would you like to rereview prior? | 18:09 |
| *** doug-fish has left #openstack-infra | 18:09 | |
| *** signed8bit has quit IRC | 18:09 | |
| SpamapS | Shrews: https://review.openstack.org/180659 is the fix for delete_server | 18:10 |
| SpamapS | mordred: ^ | 18:10 |
| jeblair | clarkb: was just about to go re-review; i think it's fine if you want to test it on 07 now. | 18:10 |
| pleia2 | cinerama: indeed, I'll still submit the bug, they may say it's expected behavior (having to add back list of default languages is easier than remembering+culling the list again if you unchecked accidentally?) but it's not what I expected | 18:10 |
| cinerama | pleia2: ok. so not a "bug" per se, but Shenanigans(tm) | 18:11 |
| *** signed8bit has joined #openstack-infra | 18:11 | |
| pleia2 | cinerama: yeah, at the very least they should document it | 18:11 |
| *** freerunner has joined #openstack-infra | 18:11 | |
| *** reed has quit IRC | 18:11 | |
| *** eharney has joined #openstack-infra | 18:12 | |
| *** ryanpetrello_ has quit IRC | 18:12 | |
| clarkb | ok putting jenkins07 in shutdown mode now, unless. pleia2 you look busy, but any interest in doing a jenkins plugin upgrade from a source build? | 18:12 |
| clarkb | pleia2: I can wait a bit too if you are but need to finish other things first | 18:12 |
| *** f1ller is now known as filler | 18:12 | |
| *** _nadya_ has quit IRC | 18:13 | |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/project-config: Only support PBR>=0.11.0 semver release tags https://review.openstack.org/180660 | 18:13 |
| fungi | clarkb: mordred: lifeless: ^ | 18:13 |
| cinerama | pleia2: i'm uploading another thing now, going to see how the proposal goes | 18:13 |
| cinerama | pleia2: oh CRAP. can you fix that tickbox or whatever? it's not pulling all the locales now | 18:14 |
| *** Sukhdev has quit IRC | 18:14 | |
| cinerama | pleia2: that is the magic of the xml creation | 18:14 |
| *** reed has joined #openstack-infra | 18:14 | |
| fungi | clarkb: mordred: lifeless: while the regular expressions there check out (based on manual testing expected and unexpected strings), i'm unconvinced that assuming all projects must support version tags as if they're using pbr (even when they're not) should be necessary to take advantage of our pre-release and release pipelines | 18:14 |
| *** e0ne_ has joined #openstack-infra | 18:15 | |
| *** e0ne has quit IRC | 18:15 | |
| cinerama | pleia2: if we can't pull down all the locales reliably in the xml we have to explicitly list them all on the command line for push/pull and this is a thing that i Do Not Want | 18:15 |
| pc_m | anteaya: fungi: Thanks for all the useful advice. I think I have a handle on how to do this. To get full history of these files, we'd have to go way back to the original neutron repo, so it is probably not worth it. | 18:15 |
| mordred | fungi: that's a good point | 18:15 |
| jeblair | fungi: especially for non-python projects? | 18:15 |
| clarkb | fungi: mordred except this is more "use semver" than "use pbr" | 18:15 |
| fungi | jeblair: especially for any projects | 18:15 |
| clarkb | and it seems reasonable to say use semver | 18:15 |
| clarkb | fungi: I do have a comment or two as soon as I can finish verifying one thing | 18:16 |
| fungi | clarkb: well, "use pbr's pep-440+semver mutant hybrid concept of versioning) | 18:16 |
| clarkb | sure, but that only affects the pre release pipeline | 18:17 |
| jeblair | clarkb: lgtm will let you aprv on your schedule | 18:17 |
| clarkb | release is pure semver three tuple | 18:17 |
| fungi | true enough | 18:17 |
| *** e0ne has joined #openstack-infra | 18:17 | |
| pleia2 | cinerama: in my project? "Inherit languages from project settings." is selected | 18:17 |
| pleia2 | cinerama: it thinks it has 40 languages to translate | 18:18 |
| cinerama | pleia2: i just created a new project and now the file is stuffed for some reason | 18:18 |
| cinerama | this makes me sad | 18:18 |
| pleia2 | ugh | 18:18 |
| clarkb | fungi: see comment | 18:18 |
| *** ryanpetrello_ has joined #openstack-infra | 18:18 | |
| pleia2 | clarkb: I may be tied up for a bit, have this and then a couple other things on my plate today | 18:19 |
| *** freerunner has quit IRC | 18:19 | |
| AJaeger_ | pleia2, cinerama: We better speed up - transifex has several "bad gateway" errors today ;/ hope they fix it... | 18:19 |
| cinerama | pleia2: actually i might have made a subtle mistake | 18:19 |
| clarkb | pleia2: ok, I will likely move forward on this, but ping if you end up with free time and are interested | 18:19 |
| pleia2 | cinerama: this is why we need to restrict language+project stuff to admins :) | 18:19 |
| *** e0ne_ has quit IRC | 18:20 | |
| clarkb | 07 is in shutdown mode | 18:20 |
| *** sdake has joined #openstack-infra | 18:20 | |
| fungi | clarkb: good point. i need to double-check the actual pre-release normalization pbr's doing | 18:20 |
| *** mrmartin has joined #openstack-infra | 18:21 | |
| cinerama | pleia2: yep, was my bug. carry on with the...other bugs :) | 18:21 |
| openstackgerrit | Merged openstack-infra/puppet-openstackci: Create a log server like http://logs.openstack.org https://review.openstack.org/167425 | 18:21 |
| fungi | clarkb: based on http://git.openstack.org/cgit/openstack-dev/pbr/tree/pbr/tests/test_version.py#n30 i _think_ you're wrong, though i do need to remove the [0-9]* prefix since pbr is going to strip it | 18:22 |
| pleia2 | cinerama: phew | 18:22 |
| *** freerunner has joined #openstack-infra | 18:22 | |
| AJaeger_ | fungi, clarkb : We have 0.26.0 now - thanks! https://pypi.python.org/pypi/openstack-doc-tools/ | 18:23 |
| zaro | clarkb: commented on change 179988 will take a look at the follow on change. | 18:23 |
| clarkb | fungi: hrm time to read the pep440 normalization rules I think | 18:23 |
| *** sdake_ has joined #openstack-infra | 18:23 | |
| *** sdake__ has quit IRC | 18:24 | |
| cinerama | [WARN] Skipping file glance/locale/pt_BR/LC_MESSAGES/glance-log-info.po; no locale entry found in zanata.xml | 18:24 |
| cinerama | [WARN] Skipping file glance/locale/en_GB/LC_MESSAGES/glance-log-info.po; no locale entry found in zanata.xml | 18:24 |
| cinerama | AARGH | 18:24 |
| fungi | clarkb: yeah, it looks like it will only generate pre-release versions matching ^([0-9]+\.){3}(a|b|rc)[0-9]+$ | 18:24 |
| cinerama | this is why we put the mapping | 18:24 |
| clarkb | zaro: ya the node label thing should be unrelated to my change since it will just do registerFunctions if they have been updated by some external event | 18:24 |
| clarkb | fungi: it being pbr? | 18:24 |
| fungi | clarkb: yep, though i'm fine reworking that regex to be a visual superset of the release regex rather than a logical one | 18:24 |
| clarkb | fungi: see https://www.python.org/dev/peps/pep-0440/#pre-release-separators | 18:25 |
| clarkb | fungi: I think pbr is wrong if it adds the . :/ | 18:26 |
| cinerama | pleia2: ok i really don't get this one: [WARN] Skipping file glance/locale/fr/LC_MESSAGES/glance-log-error.po; no locale entry found in zanata.xml | 18:26 |
| openstackgerrit | Merged openstack-infra/system-config: Use common log server implementation https://review.openstack.org/167426 | 18:26 |
| cinerama | it's like, right there | 18:26 |
| anteaya | pc_m: welcome glad you have a way forward, having the commits in a commit message would make for a very large commit message | 18:26 |
| *** sdake has quit IRC | 18:26 | |
| fungi | clarkb: i'm in agreement based on "The normal form for this is without a separator." | 18:27 |
| clarkb | fungi: I am running an sdist build of nova (should've picked a smaller repo) | 18:27 |
| *** baoli_ has quit IRC | 18:27 | |
| cinerama | crap and now it's doing the thing where it decides it doesn't like the .ini file | 18:27 |
| *** SumitNaiksatam_ has joined #openstack-infra | 18:28 | |
| asselin__ | clarkb, jeblair so is logs.openstack.org now using puppet-openstack ci? | 18:28 |
| jeblair | asselin__: did the changes merge? | 18:29 |
| asselin__ | yes | 18:29 |
| clarkb | will have to wait for the next puppet run | 18:29 |
| clarkb | but it should at that point | 18:29 |
| adrian_otto | is there a Gerrit admin around who can delete some duplicate branches for me? | 18:30 |
| *** SumitNaiksatam has quit IRC | 18:30 | |
| *** SumitNaiksatam_ is now known as SumitNaiksatam | 18:30 | |
| clarkb | we only update puppet modules at the beginning of each run for both system-config and its deps | 18:30 |
| clarkb | adrian_otto: sure | 18:30 |
| clarkb | adrian_otto: what branches on which projects? | 18:30 |
| adrian_otto | magnum and python-magnumclient both have a kilo-stable | 18:30 |
| pleia2 | cinerama: hm, weird | 18:30 |
| adrian_otto | we want to keep stable/kilo and delete kilo-stable | 18:31 |
| adrian_otto | clarkb: make sense? | 18:31 |
| clarkb | adrian_otto: yup | 18:31 |
| clarkb | adrian_otto: all done | 18:32 |
| *** mtanino has joined #openstack-infra | 18:32 | |
| cinerama | [DEBUG] Loading configuration from the path /home/jenkins/.config/zanata.ini | 18:32 |
| cinerama | [ERROR] Execution failed: | 18:33 |
| cinerama | org.zanata.client.exceptions.ConfigException: Username must be specified | 18:33 |
| cinerama | the username is in the @*$#% zanata.ini | 18:33 |
| *** yamahata has joined #openstack-infra | 18:33 | |
| *** dannywilson has joined #openstack-infra | 18:34 | |
| * mordred hands cinerama a fluffy alapaca of happiness | 18:34 | |
| *** gyee has quit IRC | 18:37 | |
| pleia2 | software is horrible | 18:39 |
| cinerama | mordred, you know i've never been real clear on the difference between alpacas and llamas. are alpacas better? | 18:39 |
| nibalizer | pleia2: not exactly 100% of it... | 18:39 |
| nibalizer | just almost all of it | 18:39 |
| pleia2 | alpacas are smaller | 18:39 |
| mordred | cinerama: I think alpacas also have nicer hair | 18:40 |
| *** hashar has joined #openstack-infra | 18:40 | |
| jeblair | llamas hold more water in their fur if you wanted to throw a wet one | 18:41 |
| clarkb | fungi: ok more info pbr made a 2015.1.0.0b1 so its normalized but != the tag | 18:41 |
| fungi | mordred: emus are cool too. and they don't have any hair at all. though not in the way that worstcat has no hair | 18:41 |
| mordred | emus make strange noises | 18:41 |
| anteaya | would I want to throw a wet llama though | 18:41 |
| anteaya | my back says no | 18:42 |
| jeblair | mordred makes strange noises | 18:42 |
| fungi | clarkb: okay, so needs to be ^([0-9]+\.){3}0(a|b|rc)[0-9]+$ to match pbr 0.11.0 behavior | 18:42 |
| *** doug-fish has joined #openstack-infra | 18:42 | |
| mordred | jeblair: fungi says that ^^ and you think I make weird noises? | 18:42 |
| jeblair | mordred: i've seen your perl | 18:42 |
| *** mattfarina has quit IRC | 18:42 | |
| *** markmcclain has joined #openstack-infra | 18:42 | |
| clarkb | fungi: I think instead of {3}0 you need {3}[0-9]+ | 18:42 |
| fungi | heh... your perl is showing | 18:42 |
| fungi | clarkb: pbr lets the leader on the alphabetic component be nonzero? | 18:43 |
| clarkb | fungi: because you *could* decide to have a .2b1? maybe? | 18:43 |
| clarkb | fungi: so many questions :) clearly we need simpler versioning | 18:43 |
| clarkb | I propose that all versions exist on a natural number line | 18:44 |
| fungi | clarkb: not a multidimensional array? | 18:44 |
| clarkb | nope, a single dimension of infinitely countable (the coutable is important here I think) values is what I want | 18:45 |
| lifeless | fungi: whats https://review.openstack.org/#/c/180660/ about ? | 18:46 |
| fungi | clarkb: i vote for a real number line instead. so i can begin versioning my releases with transcendentals | 18:46 |
| clarkb | fungi: anyways commented with what I have discovered so others can chime in there | 18:46 |
| fungi | lifeless: a 0.26 tag was pushed to a pbr-using project | 18:46 |
| fungi | lifeless: our tarball and wheel jobs of course ended up with 0.26.0 in the filenames | 18:46 |
| lifeless | asselin__: well, we haven't spoken to ttx yet | 18:47 |
| fungi | lifeless: then our pypi upload job went to download the 0.26 tarball and wheel, which dne | 18:47 |
| lifeless | asselin__: There is a potential reason to keep the version clamps in servers, and until we've had that discussion I wouldn't advocate wholesale removal | 18:47 |
| clarkb | its a simplicity thing, our tags should match the versions we want because scripts and humans all have to understand them | 18:47 |
| *** gokrokve has quit IRC | 18:47 | |
| *** openstackgerrit has quit IRC | 18:47 | |
| *** changbl has quit IRC | 18:47 | |
| *** markvoelker has quit IRC | 18:47 | |
| *** Longgeek has quit IRC | 18:47 | |
| *** jtomasek has quit IRC | 18:47 | |
| *** smccully has quit IRC | 18:47 | |
| *** russellb has quit IRC | 18:48 | |
| *** trey has quit IRC | 18:48 | |
| *** gema has quit IRC | 18:48 | |
| *** adreznec has quit IRC | 18:48 | |
| *** mfink- has quit IRC | 18:48 | |
| *** SotK has quit IRC | 18:48 | |
| *** tsufiev has quit IRC | 18:48 | |
| *** mrda has quit IRC | 18:48 | |
| *** ianw has quit IRC | 18:48 | |
| *** yamahata has quit IRC | 18:48 | |
| *** mrmartin has quit IRC | 18:48 | |
| *** e0ne has quit IRC | 18:48 | |
| *** ZZelle has quit IRC | 18:48 | |
| *** whoops has quit IRC | 18:48 | |
| *** ajmiller has quit IRC | 18:48 | |
| *** adrian_otto has quit IRC | 18:48 | |
| *** sc` has quit IRC | 18:48 | |
| *** mmedvede has quit IRC | 18:48 | |
| *** jogo has quit IRC | 18:48 | |
| *** rkukura has quit IRC | 18:48 | |
| *** zaneb has quit IRC | 18:48 | |
| *** mkoderer has quit IRC | 18:48 | |
| *** morganfainberg has quit IRC | 18:48 | |
| *** gibi has quit IRC | 18:48 | |
| *** asilenkov has quit IRC | 18:48 | |
| *** jesusaurus has quit IRC | 18:48 | |
| *** plol has quit IRC | 18:48 | |
| *** pothole has quit IRC | 18:48 | |
| *** vipul has quit IRC | 18:48 | |
| *** devananda has quit IRC | 18:48 | |
| *** mriedem has quit IRC | 18:48 | |
| *** niska has quit IRC | 18:48 | |
| *** cloudnull has quit IRC | 18:48 | |
| *** hogepodge has quit IRC | 18:48 | |
| *** raginbajin has quit IRC | 18:48 | |
| *** briancline has quit IRC | 18:48 | |
| clarkb | as soon as I have to think about what the mapping is, trouble results (note that I am doing that now) | 18:48 |
| SpamapS | dohhhh | 18:48 |
| asselin__ | lifeless, ok I see...please -1 then if you haven't already..... | 18:48 |
| lifeless | asselin__: the audit I did was for conflicting versions, not the presence of versions | 18:48 |
| *** niska has joined #openstack-infra | 18:48 | |
| SpamapS | Shrews: good feedback on my delete_server fixes, posted an updated patch | 18:48 |
| *** yamahata has joined #openstack-infra | 18:49 | |
| *** mrmartin has joined #openstack-infra | 18:49 | |
| *** e0ne has joined #openstack-infra | 18:49 | |
| *** ZZelle has joined #openstack-infra | 18:49 | |
| *** whoops has joined #openstack-infra | 18:49 | |
| *** ajmiller has joined #openstack-infra | 18:49 | |
| *** adrian_otto has joined #openstack-infra | 18:49 | |
| *** mriedem has joined #openstack-infra | 18:49 | |
| *** sc` has joined #openstack-infra | 18:49 | |
| *** mmedvede has joined #openstack-infra | 18:49 | |
| *** cloudnull has joined #openstack-infra | 18:49 | |
| *** zaneb has joined #openstack-infra | 18:49 | |
| *** jogo has joined #openstack-infra | 18:49 | |
| *** rkukura has joined #openstack-infra | 18:49 | |
| *** hogepodge has joined #openstack-infra | 18:49 | |
| *** devananda has joined #openstack-infra | 18:49 | |
| *** vipul has joined #openstack-infra | 18:49 | |
| *** pothole has joined #openstack-infra | 18:49 | |
| *** plol has joined #openstack-infra | 18:49 | |
| *** jesusaurus has joined #openstack-infra | 18:49 | |
| *** asilenkov has joined #openstack-infra | 18:49 | |
| *** gibi has joined #openstack-infra | 18:49 | |
| *** morganfainberg has joined #openstack-infra | 18:49 | |
| *** mkoderer has joined #openstack-infra | 18:49 | |
| *** raginbajin has joined #openstack-infra | 18:49 | |
| *** briancline has joined #openstack-infra | 18:49 | |
| *** gokrokve has joined #openstack-infra | 18:49 | |
| *** openstackgerrit has joined #openstack-infra | 18:49 | |
| *** changbl has joined #openstack-infra | 18:49 | |
| *** markvoelker has joined #openstack-infra | 18:49 | |
| *** Longgeek has joined #openstack-infra | 18:49 | |
| *** jtomasek has joined #openstack-infra | 18:49 | |
| *** smccully has joined #openstack-infra | 18:49 | |
| *** russellb has joined #openstack-infra | 18:49 | |
| *** trey has joined #openstack-infra | 18:49 | |
| *** gema has joined #openstack-infra | 18:49 | |
| *** adreznec has joined #openstack-infra | 18:49 | |
| *** mfink- has joined #openstack-infra | 18:49 | |
| *** SotK has joined #openstack-infra | 18:49 | |
| *** tsufiev has joined #openstack-infra | 18:49 | |
| *** mrda has joined #openstack-infra | 18:49 | |
| *** ianw has joined #openstack-infra | 18:49 | |
| lifeless | clarkb: fungi: ok, ack. | 18:50 |
| *** mattfarina has joined #openstack-infra | 18:50 | |
| fungi | lifeless: there are of course several potential places we can tackle that impedance mismatch | 18:50 |
| fungi | but all the ones i can think of result in some distasteful assumption or another | 18:51 |
| lifeless | one of the things we've wanted for ages | 18:51 |
| *** mattfarina has quit IRC | 18:51 | |
| lifeless | is 'run tests on releasing' | 18:51 |
| lifeless | if there was such a thing, we could make the tag based on the pbr output after running the tests | 18:52 |
| fungi | yep. getting some shim between pushing the tag and having the tag end up in the official repo and providing a feedback loop to tests run in that shim has been the challenge | 18:52 |
| clarkb | lifeless: how would you trigger a release in that situation? I think we had some assumptions that test on releasing would be triggered by reviewable tags in gerrit | 18:52 |
| clarkb | lifeless: we could make gerrit -1/-2 in that situation if the tag was wrong, but we would have to propose a new tag | 18:53 |
| lifeless | so I don't know all the bits limitations - I hesitate to propose implementation | 18:53 |
| lifeless | but in broad strokes we want a human to say 'please release version <loose string>', a robot to verify 'this release won't explode the world', and then to make a canonicalised tag. | 18:54 |
| *** ajmiller has quit IRC | 18:54 | |
| lifeless | jenkins jobs via zuul are the obvious middle step | 18:55 |
| *** ajmiller has joined #openstack-infra | 18:55 | |
| fungi | i can think of ways to work around the lack of actual tag review in gerrit, such as having a separate tags repo where tags are uploaded as files/blobs in new commits, then tested by applying into the actual target repo and returning test results to the review on that change | 18:55 |
| fungi | but that's pretty ugly and hard for anyone besides us to comprehend | 18:55 |
| clarkb | ya I mostly wanted to not overengineer a distant future today, when the problem we have is evident as of yesterday | 18:56 |
| reed | which version of puppet do you guys use? | 18:56 |
| lifeless | k | 18:56 |
| clarkb | reed: latest 3.X, I think 3.7.something | 18:56 |
| reed | ACK | 18:57 |
| *** mattfarina has joined #openstack-infra | 19:00 | |
| lifeless | the spec for pbr had a CLI that would report the version | 19:00 |
| lifeless | we have the CLI now | 19:00 |
| lifeless | but nothing much behind it | 19:01 |
| *** craigbr has joined #openstack-infra | 19:01 | |
| *** soren has joined #openstack-infra | 19:01 | |
| *** hashar has quit IRC | 19:02 | |
| *** freerunner has quit IRC | 19:03 | |
| *** Sukhdev has joined #openstack-infra | 19:03 | |
| fungi | lifeless: related to that discussion, clarkb and i read pep-440 to indicate that 1.2.3a4 should be already normalized, rather than normalize to 1.2.3.0a4 | 19:03 |
| fungi | how did pbr arrive at the latter? | 19:04 |
| *** jtriley_ has joined #openstack-infra | 19:04 | |
| fungi | (if you recall) | 19:04 |
| *** annegentle has joined #openstack-infra | 19:06 | |
| *** freerunner has joined #openstack-infra | 19:06 | |
| *** wayneeseguin has quit IRC | 19:06 | |
| *** gordc has joined #openstack-infra | 19:06 | |
| *** jtriley has quit IRC | 19:06 | |
| clarkb | waiting on one more job to finish on jenkins07 before I can apply the gearman plugin update there. Going to grab lunch while I wait | 19:07 |
| *** gyee has joined #openstack-infra | 19:07 | |
| *** achanda has quit IRC | 19:07 | |
| *** ivar-laz_ has quit IRC | 19:07 | |
| clarkb | asselin__: http://puppetboard.openstack.org/report/static.openstack.org/dba07e303ea584fee3a4bfb34d90966e5dc04c6b | 19:08 |
| clarkb | asselin__: other than that I think it looks good | 19:08 |
| *** ivar-lazzaro has joined #openstack-infra | 19:08 | |
| asselin__ | clarkb, ok cool! will look into that... | 19:08 |
| *** freerunner has quit IRC | 19:09 | |
| fungi | clarkb: keep in mind that puppet is still disabled on all the jenkins masters | 19:10 |
| lifeless | fungi: we wanted to keep the semver component strictly semver | 19:10 |
| lifeless | fungi: we're in a half-breed situation basically | 19:11 |
| fungi | clarkb: i'm about done confirming that jjb updates are complete and no-op on rerunning so i can reenable | 19:11 |
| *** hashar has joined #openstack-infra | 19:11 | |
| fungi | lifeless: okay, makes sense. to me the details of fine-tuning that regex are interesting, but academic until we determine whether we even want to limit all projects to pbr-centric normalized versions in tags | 19:12 |
| pleia2 | clarkb: ok, have some time now if we're still playing with jenkins plugins | 19:12 |
| lifeless | we can of course change pbr more | 19:12 |
| lifeless | as long as it accepts anything it ever output | 19:12 |
| clarkb | pleia2 waiting for jobs to finish so grabbing lunch | 19:12 |
| clarkb | fungi noted, shouldnt be an issue | 19:12 |
| clarkb | fungi (jenkins that is) | 19:13 |
| pleia2 | clarkb: lunch is a good idea, sync up after :) | 19:13 |
| openstackgerrit | melanie witt proposed openstack-infra/elastic-recheck: Update query for libvirt socket hangup bug 1451506 https://review.openstack.org/180684 | 19:13 |
| openstack | bug 1451506 in OpenStack Compute (nova) "spawn failed with "libvirtError: internal error: received hangup / error event on socket" in the gate" [Undecided,Confirmed] https://launchpad.net/bugs/1451506 | 19:13 |
| fungi | lifeless: clarkb: mordred: jeblair: what do you think of having pbr-pre-release and pbr-release pipelines with these strict regexes, and then relax the pre-release and release pipeline regexes further? | 19:13 |
| *** hashar has quit IRC | 19:13 | |
| *** amitgandhinz has quit IRC | 19:14 | |
| *** kgiusti has joined #openstack-infra | 19:14 | |
| *** hashar has joined #openstack-infra | 19:14 | |
| mordred | fungi: how about call it "semver-release"? | 19:14 |
| *** boris-42 has joined #openstack-infra | 19:14 | |
| fungi | pbr-pep-440-semver-pre-release | 19:14 |
| *** craigbr has quit IRC | 19:14 | |
| mordred | because non-pbr could still desire to have the same rules enforced | 19:14 |
| *** zul has quit IRC | 19:14 | |
| * fungi grins | 19:14 | |
| mordred | ++ | 19:14 |
| *** tonytan4ever has quit IRC | 19:15 | |
| *** zul has joined #openstack-infra | 19:16 | |
| *** salv-orl_ has joined #openstack-infra | 19:16 | |
| *** isviridov_away is now known as isviridov | 19:18 | |
| *** yamahata has quit IRC | 19:18 | |
| *** mrmartin has quit IRC | 19:18 | |
| *** e0ne has quit IRC | 19:18 | |
| *** ZZelle has quit IRC | 19:18 | |
| *** whoops has quit IRC | 19:18 | |
| *** adrian_otto has quit IRC | 19:18 | |
| *** sc` has quit IRC | 19:18 | |
| *** mmedvede has quit IRC | 19:18 | |
| *** jogo has quit IRC | 19:18 | |
| *** rkukura has quit IRC | 19:18 | |
| *** zaneb has quit IRC | 19:18 | |
| *** mkoderer has quit IRC | 19:18 | |
| *** morganfainberg has quit IRC | 19:18 | |
| *** gibi has quit IRC | 19:18 | |
| *** asilenkov has quit IRC | 19:18 | |
| *** jesusaurus has quit IRC | 19:18 | |
| *** plol has quit IRC | 19:18 | |
| *** pothole has quit IRC | 19:18 | |
| *** vipul has quit IRC | 19:18 | |
| *** devananda has quit IRC | 19:18 | |
| *** jtriley_ has quit IRC | 19:18 | |
| *** annegentle has quit IRC | 19:19 | |
| *** yamahata has joined #openstack-infra | 19:19 | |
| *** mrmartin has joined #openstack-infra | 19:19 | |
| *** e0ne has joined #openstack-infra | 19:19 | |
| *** ZZelle has joined #openstack-infra | 19:19 | |
| *** whoops has joined #openstack-infra | 19:19 | |
| *** adrian_otto has joined #openstack-infra | 19:19 | |
| *** sc` has joined #openstack-infra | 19:19 | |
| *** mmedvede has joined #openstack-infra | 19:19 | |
| *** zaneb has joined #openstack-infra | 19:19 | |
| *** jogo has joined #openstack-infra | 19:19 | |
| *** rkukura has joined #openstack-infra | 19:19 | |
| *** devananda has joined #openstack-infra | 19:19 | |
| *** vipul has joined #openstack-infra | 19:19 | |
| *** pothole has joined #openstack-infra | 19:19 | |
| *** plol has joined #openstack-infra | 19:19 | |
| *** jesusaurus has joined #openstack-infra | 19:19 | |
| *** asilenkov has joined #openstack-infra | 19:19 | |
| *** gibi has joined #openstack-infra | 19:19 | |
| *** morganfainberg has joined #openstack-infra | 19:19 | |
| *** mkoderer has joined #openstack-infra | 19:19 | |
| *** zaneb has quit IRC | 19:19 | |
| *** zbitter has joined #openstack-infra | 19:19 | |
| *** mrmartin has quit IRC | 19:19 | |
| *** annegentle has joined #openstack-infra | 19:19 | |
| *** salv-orlando has quit IRC | 19:19 | |
| *** mrmartin has joined #openstack-infra | 19:19 | |
| *** morganfainberg has quit IRC | 19:20 | |
| *** jtriley has joined #openstack-infra | 19:21 | |
| *** morganfainberg has joined #openstack-infra | 19:21 | |
| *** zbitter is now known as zaneb | 19:21 | |
| *** Somay has joined #openstack-infra | 19:23 | |
| *** otter768 has joined #openstack-infra | 19:23 | |
| *** tonytan4ever has joined #openstack-infra | 19:24 | |
| *** ociuhandu has quit IRC | 19:25 | |
| mordred | Shrews, SpamapS, clarkb, fungi, jeblair: I'm about to push the shade tag ... any last second concerns? | 19:27 |
| fungi | none from me | 19:27 |
| *** otter768 has quit IRC | 19:28 | |
| *** coolsvap|afk has quit IRC | 19:28 | |
| *** cody-somerville has joined #openstack-infra | 19:31 | |
| *** coolsvap|afk has joined #openstack-infra | 19:31 | |
| jeblair | mordred: nak | 19:32 |
| *** uberjay has joined #openstack-infra | 19:32 | |
| *** jtriley has quit IRC | 19:34 | |
| uberjay | i just asked this in #openstack-cinder, but it really belongs over here: I appear to have some unfortunate timing. i'm in the middle of setting up the log server for our os-ci, and ~1hr ago the log server puppet configuration was cleaned up/refactored in infra/system-config (https://github.com/openstack-infra/system-config/commit/518c54f841d6db15398a09d1251841f03015b6cf), which causes the os-ext-testing logserver setup to fail because of missing | 19:35 |
| uberjay | templates: http://paste.openstack.org/show/215509/ | 19:35 |
| *** jtriley has joined #openstack-infra | 19:35 | |
| *** sdake has joined #openstack-infra | 19:36 | |
| jeblair | uberjay: we didn't write the os-ext-testing module or design any of our systems for it, but we understand the need and utility of it. so we're in the process of refactoring our modules into a puppet-openstackci module that can be reliably used to run a 3rd party ci system | 19:36 |
| jeblair | uberjay: http://specs.openstack.org/openstack-infra/infra-specs/specs/openstackci.html | 19:37 |
| jeblair | uberjay: we'd love your help on that; i'm not sure if it is in a state where it's ready for you to use yet (we now use it to run our log server, but i'm not certain how easy it would be for you to do so at this point). it's early days. | 19:38 |
| jeblair | uberjay: if you're not up for all of that right now, you might want to try pinning modules versions to specific commits. lots of stuff is going to continue to change over the next few months | 19:39 |
| uberjay | jeblair: yeah, I assumed that's what happened. hm. thanks for the spec link, this is helpful. | 19:39 |
| *** sdake_ has quit IRC | 19:39 | |
| *** akshai has quit IRC | 19:39 | |
| jeblair | uberjay: asselin__'s repo might be helpful here too; i'm not sure | 19:40 |
| uberjay | jeblair: ok, yeah, unfortunately I don't have the time to help with early-days testing for this -- I wish I did. :) | 19:40 |
| uberjay | jeblair: ah, I suspect that's what I'm using! | 19:40 |
| jeblair | uberjay: no worries, you're welcome to join whenever it gets usable enough for you :) | 19:41 |
| fungi | fwiw asselin__ is also helping drive the openstack-ci puppet module work, so may have some additional suggestions | 19:41 |
| *** Somay has quit IRC | 19:42 | |
| uberjay | fungi: that's a great point. i just realized the same person was responsible for the openstackci-puppet repo. | 19:43 |
| fungi | sdague: were you planning to backport 180549 to stable/kilo as well? | 19:43 |
| *** craigbr has joined #openstack-infra | 19:43 | |
| fungi | just curious mostly | 19:43 |
| uberjay | asselin__: if you're around, do you have any suggestions on how to approach the divergence with os-ext-testing & the infrastructure modules? | 19:43 |
| *** sdake_ has joined #openstack-infra | 19:44 | |
| *** ajo has quit IRC | 19:44 | |
| uberjay | asselin__: in the meantime, I could hold the system-config/project-config modules to specific commits, or just manually configure the logserver. | 19:45 |
| *** sdake has quit IRC | 19:47 | |
| *** shashankhegde has quit IRC | 19:51 | |
| *** _nadya_ has joined #openstack-infra | 19:51 | |
| *** baoli has joined #openstack-infra | 19:52 | |
| *** prometheanfire has joined #openstack-infra | 19:53 | |
| *** craigbr has quit IRC | 19:53 | |
| prometheanfire | why does review.openstack have such bad ssh config settings (macs and ciphers at the least, some better kex options would be nice too) | 19:53 |
| mordred | prometheanfire: it's not a real ssh server | 19:54 |
| prometheanfire | :( | 19:55 |
| *** Rockyg has joined #openstack-infra | 19:55 | |
| jeblair | prometheanfire: actually, it might be able to be improved upon. you might take a look at https://review.openstack.org/Documentation/config-gerrit.html and propose a patch to http://git.openstack.org/cgit/openstack-infra/puppet-gerrit/ if you see an opportunity for improvement | 19:57 |
| jeblair | prometheanfire: also, we're planning a gerrit upgrade on saturday which will include a new version of the built-in sshd | 19:58 |
| prometheanfire | that's nice :D | 19:59 |
| *** armax has quit IRC | 20:00 | |
| fungi | we'll hopefully be on mina-sshd 0.14 once the upgrade window completes | 20:00 |
| clarkb | pleia2: I am back and ready when you are | 20:00 |
| pleia2 | clarkb: yep, I'm ready | 20:00 |
| clarkb | pleia2: first thing is to double check https://jenkins07.openstack.org/ is not running any jobs (it seems done) | 20:01 |
| *** prometheanfire has left #openstack-infra | 20:01 | |
| pleia2 | yep | 20:01 |
| jeblair | nibalizer: do you want me to merge 171369 or should it wait for the other stuff from earlier? | 20:01 |
| clarkb | pleia2: then we have two options for installing the plugin, we can either use the Jenkins admin gui to install it while jenkins is still up, or we can stop jenkins then manually install the plugin. | 20:01 |
| fungi | huh. kde's getting interesting. how did i not know about krita? | 20:02 |
| clarkb | pleia2: I tend to use the second option because it gives me control over the process and we don't ahve the weirdness of an installed but not fully running plugin going on | 20:02 |
| pleia2 | clarkb: to confirm, I know no jobs are running because the build systems are Idle, right? | 20:02 |
| clarkb | pleia2: correct | 20:02 |
| pleia2 | perfect | 20:02 |
| pleia2 | clarkb: +1 to second way, installing plugins via web guis is meh | 20:03 |
| jeblair | fungi: oh yeah, i've heard good things about krita but have not used it | 20:03 |
| clarkb | pleia2: in either case we are going to install the build artifact that resulted from https://review.openstack.org/#/c/179988/ 's check test | 20:03 |
| pleia2 | clarkb: ok | 20:03 |
| clarkb | hrm do we not upload a test hpi? I thought we did | 20:03 |
| pleia2 | the gearman-plugin.jar ? | 20:04 |
| fungi | krita's drawing tablet support looks better-implemented than the gimp | 20:04 |
| *** rkukura has quit IRC | 20:04 | |
| clarkb | pleia2: the .hpi or .jpi | 20:04 |
| pleia2 | ah, ok | 20:04 |
| clarkb | http://tarballs.openstack.org/ci/gearman-plugin/?C=M;O=A actually I think it is there? | 20:04 |
| clarkb | or is that jeblairs? | 20:04 |
| *** hashar has quit IRC | 20:04 | |
| clarkb | I think thats jeblairs hrm maybe I should go ahead and approve this so we can get a nice hpi built for us | 20:05 |
| clarkb | pleia2: the other option is to build the hpi youself with `mvn package` | 20:05 |
| * pleia2 nods | 20:05 | |
| clarkb | pleia2: but I prefer getting jenkins to do it so we can all operate off of the same package | 20:05 |
| clarkb | let me approve | 20:05 |
| pleia2 | yeah, I don't want to get into the business of building java packages | 20:06 |
| clarkb | now we wait a few minutes to get the package built and it should show up at the link above | 20:06 |
| *** ildikov has quit IRC | 20:06 | |
| pleia2 | go go gadget gate job | 20:06 |
| fungi | it's "fun" (in the same way that inserting your extremities into a wood chipper might also be considered fun) | 20:06 |
| *** dizquierdo has joined #openstack-infra | 20:07 | |
| pleia2 | (best to make sure you have the right version wood chipper) | 20:07 |
| clarkb | there was a time I would randomly try kde instead of gnome2, but then their indexer thing happened and killed my machine and I switched to lightweight desktops with xmonad as the window manager | 20:09 |
| openstackgerrit | Merged openstack-infra/gearman-plugin: Fix race between adding job and registering https://review.openstack.org/179988 | 20:09 |
| *** freerunner has joined #openstack-infra | 20:09 | |
| fungi | clarkb: we're not cd'ing that ^ right? | 20:09 |
| clarkb | fungi: we are not | 20:09 |
| clarkb | fungi: we don't really have a good way to control jenkins plugin versions automagically | 20:09 |
| jeblair | best to just get rid of jenkins | 20:10 |
| fungi | just making sure turning puppet back on for the jenkins masters here in a few moments won't cause a problem | 20:10 |
| clarkb | fungi: it shouldn't | 20:10 |
| clarkb | fungi: the jenkins puppet plugin management is write once | 20:10 |
| fungi | ahh, that's right, we have to do the plugin upgrade dance in jenkins anyway | 20:10 |
| clarkb | fungi: it doesn't upgrade anything | 20:10 |
| fungi | yep. remembering now | 20:10 |
| *** notnownikki has quit IRC | 20:10 | |
| *** jcoufal has joined #openstack-infra | 20:11 | |
| *** jcoufal has quit IRC | 20:11 | |
| fungi | jeblair: perhaps soon, all 0 of our jenkins plugins will automatically be at their intended versions | 20:11 |
| jeblair | clarkb, fungi, zaro: i was just looking at 172534 and thought to myself, "I wonder if anyone understands the BC jar situation on gerrit." | 20:11 |
| pleia2 | heh | 20:11 |
| jeblair | because we had that issue with 2.8 and i think we learned something about symlinks but i am not sure what it was and then i flew around on planes | 20:12 |
| clarkb | jeblair: puppet was not properly cleaning things up, I wrote a change that *should* fix that | 20:12 |
| asselin__ | uberjay, yes, you can pin your system config | 20:12 |
| fungi | jeblair: oh, right, so... gerrit seems to provide its own when we unpack the war. 172534 could be overcomplicating this? | 20:13 |
| asselin__ | uberjay, I'l update my repo to pull in the latest | 20:13 |
| clarkb | jeblair: https://review.openstack.org/#/c/178251/ | 20:13 |
| openstackgerrit | sebastian marcet proposed openstack-infra/openstackid: Migration of resource server https://review.openstack.org/178854 | 20:13 |
| openstackgerrit | sebastian marcet proposed openstack-infra/openstackid-resources: OpenstackId resource server https://review.openstack.org/178853 | 20:13 |
| *** gyee has quit IRC | 20:13 | |
| jeblair | fungi: ah, yeah, maybe 534 is unecessary? maybe we just stop tidying? | 20:13 |
| uberjay | asselin__: what's the correct way to pin a repo? | 20:13 |
| jeblair | clarkb: ? | 20:13 |
| *** lennyb has joined #openstack-infra | 20:14 | |
| clarkb | jeblair: basically we have to run the tidy after installing gerrit and initializing it but before starting the service | 20:14 |
| clarkb | jeblair: 178251 should do that properly now | 20:14 |
| *** yamahata has quit IRC | 20:14 | |
| jeblair | (in other words: for 2.8 we need to tidy correctly, for 2.10 perhaps we don't install libs and don't tidy; just let gerrit do its thing) | 20:14 |
| fungi | jeblair: clarkb well, for the libs that get unpacked from the gerrit war though, we could just drop the symlinks and use those instead? | 20:14 |
| fungi | yeah, possibly | 20:15 |
| clarkb | oh right because 2.10 wants the newer versions | 20:15 |
| clarkb | ya I think so | 20:15 |
| *** asselin__ is now known as asselin | 20:15 | |
| *** AJaeger_ has quit IRC | 20:15 | |
| fungi | since 2.8 _would_ work with distro-provided bc we blew away teh one from the war and symlinked. 2.10 won't work with the one from the system, so we're working around our workaround | 20:15 |
| clarkb | pleia2: ok http://tarballs.openstack.org/ci/gearman-plugin/gearman-plugin-0.1.1.6.0314ab1.hpi is available now | 20:15 |
| pleia2 | cool | 20:15 |
| jeblair | clarkb: i think 178251 has errors | 20:16 |
| pleia2 | wget to jenkins07? | 20:16 |
| clarkb | pleia2: so next step is hop on jenkins07 and stash that somewhere (in your homedir) | 20:16 |
| clarkb | pleia2: yup | 20:16 |
| pleia2 | k | 20:16 |
| pleia2 | done | 20:16 |
| clarkb | jeblair: ah yup, fixing | 20:16 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/subunit2sql: WIP: Add CLI tool to graph aggregate failure counts for tests https://review.openstack.org/178039 | 20:16 |
| jogo | I ran recheck on a patch that had a +W but wasn't in the gate | 20:17 |
| *** mattfarina has quit IRC | 20:17 | |
| jogo | and it went through check and just stopped and didn't go into gate | 20:17 |
| clarkb | pleia2: now if you ls -l /var/lib/jenkins/plugins | grep gearman you will see the existing plugin | 20:17 |
| mtreinish | jogo: dependency? | 20:17 |
| fungi | jogo: did it already have a +1 when you left a recheck comment? | 20:17 |
| jogo | mtreinish: no | 20:17 |
| clarkb | pleia2: should be one dir, then >=1 [hj]pi files | 20:17 |
| jogo | fungi: see https://review.openstack.org/#/c/177222/7 | 20:17 |
| fungi | jogo: if so, gerrit didn't add a new +1 | 20:17 |
| openstackgerrit | Alexander Yip proposed openstack-infra/project-config: Enable congress and tempest in devstack in congress-dsvm-api test https://review.openstack.org/180726 | 20:17 |
| pleia2 | clarkb: yep -rw-r--r-- 1 jenkins nogroup 762K Oct 22 2014 gearman-plugin.hpi | 20:18 |
| *** shashankhegde has joined #openstack-infra | 20:18 | |
| clarkb | pleia2: what we are going to do is, stop jenkins, rm the plugin dir, move the existing .hpi/jpi aside, copy new one in, set perms to jenkins:nogroup, start jenkins | 20:18 |
| asselin | uberjay, just comment out theses lines: https://github.com/rasselin/os-ext-testing/blob/master/puppet/install_master.sh#L36-L44 | 20:18 |
| fungi | jogo: note the lack of "Verified+1" in the new gerrit comment | 20:18 |
| asselin | uberjay, and manually pick what you want | 20:18 |
| pleia2 | clarkb: easy enough, shall I? | 20:18 |
| fungi | jogo: you need to get that change reapproved | 20:18 |
| clarkb | pleia2: oh and don't forget to double check jenkins actually stopped after stopping it | 20:18 |
| *** HeOS has joined #openstack-infra | 20:18 | |
| pleia2 | clarkb: right, thanks | 20:18 |
| clarkb | pleia2: yup go for it, let me know if anything pop up | 20:18 |
| *** Krinkle is now known as Krinkle|detached | 20:19 | |
| jogo | fungi: ahh thanks wasn't sure | 20:19 |
| nibalizer | jeblair: 171369 can go | 20:19 |
| uberjay | asselin: ah, ok. i'd done that, but thought there may be a more clever way. ;) | 20:19 |
| fungi | jogo: either approve it yourself, or get sdague to remove and readd his approval, or whatever | 20:19 |
| *** mdenny has joined #openstack-infra | 20:19 | |
| jogo | fungi: just put a +W of my own there | 20:19 |
| jogo | fungi: a +W is sufficient I don't need to +2 it as well right? | 20:19 |
| fungi | right | 20:19 |
| jeblair | nibalizer: done | 20:20 |
| nibalizer | \o/ | 20:20 |
| pleia2 | clarkb: there is a gearman-plugin and gearman-plugin.hpi - anything to do with the non .hpi one? | 20:20 |
| jeblair | zaro: see comments in https://review.openstack.org/172534 | 20:20 |
| fungi | jogo: as long as there's at least one code review +2 already present | 20:20 |
| clarkb | pleia2: thats a dir right? you can delete it | 20:20 |
| *** dkranz has quit IRC | 20:20 | |
| jogo | fungi: perfect | 20:20 |
| clarkb | pleia2: aiui the hpi file gets unpacked into that dir somehow and I don't trust it to delete files if they go away across versions | 20:20 |
| clarkb | pleia2: once you start jenkins you should see a new dir with that ame | 20:21 |
| jeblair | clarkb, pleia2: process to stop jenkins: /etc/init.d/jenkins stop; killall jenkins; killall -9 jenkins; ps aux|grep jenkins; kill kill kill; reboot | 20:21 |
| pleia2 | clarkb: yeah, it is, thanks | 20:21 |
| clarkb | jeblair: maybe even nova delete; nova boot :) | 20:21 |
| openstackgerrit | Clark Boylan proposed openstack-infra/puppet-gerrit: Run lib tidy after plugin install, before start https://review.openstack.org/178251 | 20:22 |
| clarkb | jeblair: ^ that may not be useful for very long but for completeness that should make puppet happy | 20:22 |
| fungi | hah | 20:22 |
| fungi | i usually stop, then kill -1, then -7, then -9, then sometimes have to start down the same path for the parent pid | 20:22 |
| fungi | _most_ of the time it dies on the -7 sigsegv | 20:23 |
| clarkb | fungi: I usually stop then go straight to -9 | 20:23 |
| fungi | and the parent then terminates | 20:23 |
| jeblair | clarkb: yeah, i think it's worth getting in now | 20:23 |
| fungi | sometimes i even get lucky and stop works as intended | 20:23 |
| jeblair | clarkb: +3; it's in zuul's hands now | 20:23 |
| openstackgerrit | sebastian marcet proposed openstack-infra/openstackid-resources: OpenstackId resource server https://review.openstack.org/178853 | 20:23 |
| openstackgerrit | sebastian marcet proposed openstack-infra/openstackid: Migration of resource server https://review.openstack.org/178854 | 20:24 |
| lennyb | Hi All, is there a way to trigger zuul with disabled Gerrit account ? | 20:24 |
| *** zz_ja has quit IRC | 20:25 | |
| pleia2 | clarkb: ok, jenkins07 is coming back up | 20:25 |
| *** hemnafk is now known as hemna | 20:26 | |
| clarkb | pleia2: looks like jenkins is coming back up, once its up we will want ot make sure its running jobs and we can double check the plugin version in the manage plugins window | 20:26 |
| pleia2 | and already building some things | 20:26 |
| jeblair | lennyb: i would say intentionally no | 20:26 |
| * pleia2 nods | 20:26 | |
| *** e0ne has quit IRC | 20:26 | |
| *** mattfarina has joined #openstack-infra | 20:26 | |
| *** mattfarina has quit IRC | 20:26 | |
| lennyb | jeblair: thanks. | 20:26 |
| clarkb | pleia2: https://jenkins07.openstack.org/pluginManager/installed is the shortcut to the plugin page | 20:27 |
| clarkb | pleia2: or you can do manage jenkins -> manage plugins -> installed | 20:27 |
| pleia2 | clarkb: ah cool, I was clicking around | 20:27 |
| clarkb | pleia2: version lgtm | 20:27 |
| *** zz_ja has joined #openstack-infra | 20:27 | |
| pleia2 | lyz is missing the Overall/Administer permission | 20:28 |
| clarkb | pleia2: now its just a matter of keeping track of jobs and making sure the server doesn't go haywire, I can keep an eye on it if you want to move onto other stuff | 20:28 |
| clarkb | pleia2: oh, hrm let me check that | 20:28 |
| pleia2 | so maybe we did find something I still need to be added to! | 20:28 |
| fungi | pleia2: did you make sure to check the admins group checkbox when you did the openid authentication, or did you just click through? | 20:28 |
| clarkb | pleia2: did you check the box with openid to send the group info? | 20:29 |
| *** freerunner has quit IRC | 20:29 | |
| clarkb | ya that | 20:29 |
| openstackgerrit | Merged openstack-infra/system-config: Pull o_p::server invocation out of afs roles https://review.openstack.org/171369 | 20:29 |
| *** smarcet has joined #openstack-infra | 20:29 | |
| jeblair | nibalizer: you see fbo is doing some work on your base refactor spec too, right? | 20:29 |
| pleia2 | fungi: doooh | 20:29 |
| pleia2 | fungi: I don't read login screens! :) | 20:29 |
| *** emagana has quit IRC | 20:29 | |
| pleia2 | much better, that was it | 20:29 |
| fungi | pleia2: don't feel bad. i once spent half an hour confused because i did the same thing | 20:29 |
| *** freerunner has joined #openstack-infra | 20:30 | |
| fungi | necessary learning pain | 20:30 |
| *** ivar-lazzaro has quit IRC | 20:30 | |
| pleia2 | 0.1.1.6.0314ab1 yay | 20:30 |
| smarcet | fungi: hello | 20:30 |
| *** baoli has quit IRC | 20:30 | |
| *** ivar-lazzaro has joined #openstack-infra | 20:31 | |
| fungi | smarcet: hi there | 20:31 |
| smarcet | fungi: was wondering how should approve these changes https://review.openstack.org/#/c/178853 and https://review.openstack.org/#/c/178854 | 20:31 |
| *** baoli has joined #openstack-infra | 20:31 | |
| *** rbradfor has quit IRC | 20:31 | |
| smarcet | fungi: sorry who should | 20:31 |
| clarkb | hrm https://jenkins07.openstack.org/job/gate-ironic-python27/1587/console not sure if that is related or not | 20:31 |
| *** lennyb has left #openstack-infra | 20:32 | |
| *** ildikov has joined #openstack-infra | 20:32 | |
| smarcet | fungi: should i just give them +2? or should i ask to someone else to review them? | 20:32 |
| fungi | smarcet: since you're probably the only developer on that project, i guess you should for now | 20:32 |
| smarcet | fungi: roger doger thx :) | 20:32 |
| pleia2 | clarkb: hm :\ | 20:32 |
| fungi | smarcet: once those projects gain regular reviewers, that's time to revisit that workflow | 20:32 |
| clarkb | pleia2: actually that is a requested job abort I think zuul did that | 20:33 |
| clarkb | pleia2: so we may be fine, still watching though | 20:33 |
| pleia2 | clarkb: phew | 20:33 |
| * pleia2 nods | 20:33 | |
| fungi | smarcet: if there are other devs at tipit familiar with that codebase, then we should probably add them to the core review groups for them so that they can look over changes like that | 20:33 |
| clarkb | pleia2: the RequestAbortedException is what jenkins does when zuul says it no longer needs a job (maybe an earlier test failed or somethign) | 20:34 |
| *** e0ne has joined #openstack-infra | 20:34 | |
| pleia2 | clarkb: I see | 20:34 |
| clarkb | https://jenkins07.openstack.org/job/gate-nova-pep8/11582/console and https://jenkins07.openstack.org/job/gate-nova-docs/11507/console were both successful | 20:34 |
| pleia2 | yep, just looked at the latter, great | 20:34 |
| fungi | smarcet: looks like mrmartin can also +2/approve changes to those repos right now | 20:35 |
| smarcet | fungi: yeah marton has +2 rites | 20:35 |
| *** radez is now known as radez_g0n3 | 20:35 | |
| smarcet | should i ask to him? | 20:35 |
| mrmartin | I can | 20:36 |
| mrmartin | but not now if you don't mind, it is a bit late here | 20:36 |
| clarkb | pleia2: and https://jenkins07.openstack.org/job/gate-poppy-python34/91/console ran to completion and appears to have properly failed | 20:36 |
| mrmartin | smarcet: I've added this to my backlog, and will check tomorrow | 20:36 |
| clarkb | pleia2: so I think we are good so far | 20:36 |
| *** Swami has quit IRC | 20:37 | |
| pleia2 | clarkb: great | 20:37 |
| smarcet | mmartin: sure np | 20:37 |
| smarcet | i will add u are reviewer | 20:37 |
| nibalizer | jeblair: no i don't see that.. link? | 20:37 |
| mrmartin | smarcet: thanks! | 20:38 |
| *** achanda has joined #openstack-infra | 20:38 | |
| pleia2 | clarkb: so once this is all good, then we do it on all the jenkinses? | 20:39 |
| clarkb | pleia2: yup | 20:39 |
| clarkb | pleia2: we can do two or three at a time too to speed things up | 20:39 |
| * pleia2 nods | 20:40 | |
| clarkb | pleia2: probably going to let this sit in place until tomorrow though so we can check for any leaked nodes (that shouldn't happen anymore) | 20:40 |
| pleia2 | clarkb: wfm | 20:40 |
| jeblair | nibalizer: https://review.openstack.org/#/q/owner:%22Fabien+Boucher%22+status:open+topic:downstream-puppet,n,z | 20:40 |
| pleia2 | I'm around all morning tomorrow, after noon gets dicey because lunch meeting + getting to the valley by 5pm for an event | 20:40 |
| openstackgerrit | sebastian marcet proposed openstack-infra/openstackid: Migration of resource server https://review.openstack.org/178854 | 20:40 |
| clarkb | pleia2: noted, we can get an early start | 20:40 |
| clarkb | jeblair: anything you want us to check as far as work statuses go? | 20:41 |
| nibalizer | jeblair: ah i thought you meant an update to the spec itself | 20:41 |
| clarkb | jeblair: tcpdump maybe? | 20:41 |
| nibalizer | fbo: thanks! | 20:41 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/project-config: puppet: run beaker on both trusty & centos7 https://review.openstack.org/175434 | 20:41 |
| jeblair | clarkb: nah, i believe it. :) | 20:41 |
| clarkb | pleia2: the rollback process is basically the same too, stop jenkins, really stop it, put old hpi back in place, start jenkins | 20:42 |
| yolanda | jeblair, i answered to some of your reviews | 20:42 |
| yolanda | i need more feedback on that | 20:42 |
| jeblair | yolanda: which one(s)? | 20:42 |
| jeblair | yolanda: (i still haven't gotten to the bottom of the list :) | 20:42 |
| pleia2 | clarkb: easy enough, and timestamps make figuring out which .bak it is simple enough | 20:43 |
| *** unicell has joined #openstack-infra | 20:43 | |
| yolanda | https://review.openstack.org/#/c/161663/ | 20:43 |
| yolanda | https://review.openstack.org/#/c/171526/ | 20:43 |
| yolanda | these 2 | 20:44 |
| yolanda | jeblair, thx for the approval on some, really appreciated | 20:44 |
| *** unicell has quit IRC | 20:45 | |
| *** unicell has joined #openstack-infra | 20:45 | |
| clarkb | asselin: any progress on that error? I am willing to helpout now that the jenkins thing is done | 20:45 |
| *** achanda has quit IRC | 20:45 | |
| *** e0ne has quit IRC | 20:45 | |
| jeblair | yolanda: regarding 161663, i don't think my suggestion contradicts any of the advice you have previously received. my thought is that the puppet-gerrit module should know nothing about jeepyb because it is possible to run gerrit without jeepyb. | 20:46 |
| pabelanger | Good news everybody, grafyaml appears to be working! So far, you can use a simple yaml to create a dashboard, with some rows. | 20:46 |
| pabelanger | with the help of python-voluptuous its actually very simple to setup the grafana schema | 20:47 |
| jeblair | yolanda: so i'm asking is it possible to remove manifests/remotes from puppet-gerrit entirely and move it to openstack_project/gerrit for now? (and later into openstackci) | 20:47 |
| yolanda | jeblair, ok , your suggestion makes sense | 20:47 |
| *** moshele has joined #openstack-infra | 20:48 | |
| pabelanger | I have to run now, but wanted to give a simple update. | 20:48 |
| pabelanger | chat in the morning | 20:48 |
| *** pabelanger has quit IRC | 20:48 | |
| jeblair | pabelanger: w00t, thanks! | 20:48 |
| jogo | moshele: o/ I think you need to ask here | 20:48 |
| yolanda | i'll add the change to remove from there, and move to openstack_project using that jeepyb manage_projects class | 20:49 |
| jeblair | yolanda: (and also when you do that, i think you could do the inline suggestion about the log file path too) | 20:49 |
| moshele | jogo: do you know who? | 20:49 |
| jogo | moshele: just ask in general | 20:49 |
| openstackgerrit | Merged openstack-infra/puppet-gerrit: Run lib tidy after plugin install, before start https://review.openstack.org/178251 | 20:49 |
| jeblair | we're a collective | 20:49 |
| jeblair | yolanda: re 171526, answer is "yes". :) | 20:50 |
| *** ayoung has joined #openstack-infra | 20:50 | |
| openstackgerrit | Merged openstack-infra/system-config: Bump stankevich-python version https://review.openstack.org/174820 | 20:50 |
| yolanda | jeblair, about logfile, do you want to send output of cronjob and from notify into same logfile? | 20:50 |
| openstackgerrit | Merged openstack-infra/system-config: Bump stankevich-python version https://review.openstack.org/174820 | 20:50 |
| yolanda | or i'm not following you? | 20:51 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/subunit2sql: Add db api method to get uuid from test_id https://review.openstack.org/180741 | 20:51 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/subunit2sql: Add db api method to get per status counts for a test https://review.openstack.org/180742 | 20:51 |
| moshele | can you enable the Mellanox CI account? all the jenkins jobs are stoped and we need it to debug the issue in Our CI | 20:51 |
| jeblair | yolanda: no; i'm just saying the cron entry can hardcode the logfile and the notify can hardcode its logfile, and the consumer of the module doesn't need to specify that | 20:51 |
| yolanda | i'm ok with that | 20:51 |
| jeblair | moshele: have you cahnged it to a configuration where it will not leave comments in gerrit while you debug the issue? | 20:52 |
| yolanda | so it sounds like a plan, thx | 20:52 |
| jeblair | moshele: s/cahnged/changed/ | 20:52 |
| yolanda | really, i'm going to start working at nights, it's much easier :) | 20:52 |
| *** unicell has quit IRC | 20:52 | |
| jeblair | yolanda: heh, i usually review earlier, i'm just not caught up after traveling yet | 20:52 |
| yolanda | jeblair, for the other holded changes, i agree with you, i hope that this lock is released soon | 20:53 |
| moshele | jeblair: all the nodes are down and yes it configured not to leave comments | 20:53 |
| *** _nadya_ has quit IRC | 20:53 | |
| yolanda | jeblair, not also for the reviews, but for the opportunities to talk in chat, is so helpful | 20:53 |
| jeblair | moshele: what's the username? | 20:53 |
| *** _nadya_ has joined #openstack-infra | 20:53 | |
| *** unicell has joined #openstack-infra | 20:54 | |
| clarkb | pleia2: I am also working on a second change to the gearman plugin that we may want to bundle into the large scale plugin upgrades | 20:54 |
| *** unicell has quit IRC | 20:55 | |
| clarkb | I should start working on figuring out why it doesn't do what I expect | 20:55 |
| *** samueldmq has quit IRC | 20:55 | |
| moshele | jelair: Gerrit Account: Mellanox CI <mlnx-openstack-ci@dev.mellanox.co.il> | 20:55 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/project-config: puppet: run beaker on both trusty & centos7 https://review.openstack.org/175434 | 20:55 |
| EmilienM | nibalizer: ^^^^^^^^ I think this patch is ready for review, I've added Centos7 support for beaker | 20:56 |
| *** aysyd has quit IRC | 20:56 | |
| pleia2 | clarkb: cool | 20:56 |
| clarkb | zaro: jeblair do you know if there was a specific reason to always reset abilities and reregister everything? was it just simpler or is there something I should be afraid of there | 20:57 |
| openstackgerrit | Merged openstack-infra/puppet-logstash: Add missing vhost_name https://review.openstack.org/176631 | 20:57 |
| *** _nadya_ has quit IRC | 20:58 | |
| jeblair | moshele: done | 20:58 |
| moshele | jeblar: thanks | 20:58 |
| jeblair | clarkb: i think programming was hard and we only had like 500 jobs at the time so why bother? :) | 20:59 |
| jeblair | clarkb: i just had one thought you should watch out for -- there might be cases where a node wants to remove all jobs; you should detect that and use RESET instead of CANT_DO there | 21:00 |
| clarkb | jeblair: good point | 21:00 |
| clarkb | I think that is easy to chek for, just look for an empty new list | 21:00 |
| jeblair | ya | 21:00 |
| *** unicell has joined #openstack-infra | 21:01 | |
| *** emagana has joined #openstack-infra | 21:01 | |
| fungi | Number of jobs generated: 5556 | 21:01 |
| fungi | order of magnitude in just a couple years | 21:02 |
| jeblair | i like that there's space for another digit there | 21:02 |
| fungi | indeed | 21:02 |
| fungi | converse moore's law. 10x increase in 2 years | 21:02 |
| fungi | i hereby name thee "cruft's law of accumulation" | 21:03 |
| jeblair | (i hope we have zuul3 before we have the fleet of jenkins-job-builder machines whose only purpose is to continually update the 50000 jenkins jobs across our 30 jenkins masters) | 21:03 |
| *** Sukhdev has quit IRC | 21:03 | |
| *** gokrokve has quit IRC | 21:03 | |
| *** gokrokve has joined #openstack-infra | 21:04 | |
| openstackgerrit | Stephen Gordon proposed openstack-infra/project-config: Add openstack-nfv to gerritbot https://review.openstack.org/180744 | 21:04 |
| *** gokrokve has quit IRC | 21:05 | |
| *** gokrokve has joined #openstack-infra | 21:05 | |
| fungi | so even with the jenkins masters caught up on jjb updates from less than 24 hours prior, a full update with today's puppet applied doesn't complete within the exec timeout | 21:05 |
| clarkb | so it looks like we do setFunctions multiple times, the first time we include build:foobar:master then the second time we exclude it hence the CANT_DOs for the :master job | 21:05 |
| *** moshele has left #openstack-infra | 21:05 | |
| *** shashankhegde has quit IRC | 21:05 | |
| fungi | i think our template expansions are just getting unwieldy | 21:05 |
| clarkb | I want to say that this behavior must exist in the old code too, will need to also test with a real label | 21:05 |
| pleia2 | oof | 21:05 |
| jeblair | fungi: hrm, that's weird; i would expect it to take < 5 mins | 21:05 |
| *** emagana has quit IRC | 21:06 | |
| fungi | Info: /etc/jenkins_jobs/config: Scheduling refresh of Exec[jenkins_jobs_update] | 21:06 |
| fungi | Error: /Stage[main]/Jenkins::Job_builder/Exec[jenkins_jobs_update]: Failed to call refresh: Command exceeded timeout | 21:06 |
| fungi | that was doing puppet agent --test on jenkins02 | 21:06 |
| jeblair | clarkb: oh, seems like :master is not something we ever want | 21:06 |
| clarkb | jeblair: ya for our use case I think its fine | 21:06 |
| clarkb | but this may be a general case bug | 21:07 |
| fungi | it looks like this template expansion is courtesy of a change switching the nova api version in a bunch of devstack-gate job defs | 21:07 |
| jeblair | clarkb: okay, as long as the general case isn't too hard :) | 21:07 |
| clarkb | jeblair: well if its preexisting I am not going to fix it here :) | 21:07 |
| *** ajmiller has quit IRC | 21:07 | |
| clarkb | testing that next | 21:08 |
| jeblair | fungi: oh. so maybe time to just bump the timeout? | 21:08 |
| *** gokrokve has quit IRC | 21:08 | |
| *** gokrokve_ has joined #openstack-infra | 21:08 | |
| *** bswartz has quit IRC | 21:08 | |
| fungi | jeblair: maybe. i'm seeing how much longer the catch-up takes after that | 21:09 |
| jeblair | fungi: it only happens on subscription, so we only catchup on the next change :/ | 21:09 |
| fungi | to get some idea of what a reasonable timeout bump might look like | 21:09 |
| jeblair | fungi: ah, yeah. makes sense. | 21:09 |
| fungi | righth, catching up by manually running jjb again, not reapplying from puppey | 21:10 |
| fungi | s/y$/t/ | 21:10 |
| jeblair | you're talking weird again | 21:10 |
| clarkb | jeblair: but pretty sure my change makes function registrations N * slaves instead of N * N * slaves where N is number of jobs updated so should make a huge difference | 21:11 |
| *** rbradfor has joined #openstack-infra | 21:11 | |
| fungi | that would also explain why job updates take so much longer when we're busy. i assumed it was load on the jenkins masters, but sounds like it's a multiplication due to number of slaves | 21:12 |
| *** cody-somerville has quit IRC | 21:12 | |
| fungi | jeblair: additional wall clock time was 7m20s | 21:12 |
| jeblair | clarkb: and that's just initial ones; should make updates M*slaves where M is number of added and removed jobs rather than N*slaves. | 21:12 |
| fungi | current timeout is 10m? if so, should probably bump to 20 | 21:12 |
| jeblair | fungi: sgtm | 21:12 |
| fungi | on the way | 21:13 |
| openstackgerrit | Merged openstack-infra/system-config: Don't hardcode names on certs https://review.openstack.org/179067 | 21:13 |
| *** tiswanso has quit IRC | 21:13 | |
| *** Swami has joined #openstack-infra | 21:13 | |
| *** sdake has joined #openstack-infra | 21:14 | |
| *** ldnunes has quit IRC | 21:14 | |
| *** smarcet has quit IRC | 21:15 | |
| *** zz_ja has quit IRC | 21:15 | |
| clarkb | jeblair: confirmed for some reason we were already removing build:foo:master after first registering it | 21:16 |
| clarkb | jeblair: so I will just update to optimize for the remove all jobs case | 21:16 |
| jeblair | clarkb: neat! | 21:16 |
| *** pc_m has quit IRC | 21:16 | |
| *** ociuhandu has joined #openstack-infra | 21:16 | |
| *** dboik has quit IRC | 21:16 | |
| *** zz_ja has joined #openstack-infra | 21:17 | |
| *** sdake_ has quit IRC | 21:18 | |
| openstackgerrit | Merged openstack-infra/puppet-cgit: Manage cgitrc config file https://review.openstack.org/179132 | 21:19 |
| *** adrian_otto has quit IRC | 21:19 | |
| *** freerunner has quit IRC | 21:19 | |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/system-config: Increase JJB timeout to 20min on Jenkins masters https://review.openstack.org/180753 | 21:20 |
| *** maurosr has quit IRC | 21:21 | |
| fungi | not sure whether that ^ is where we want it, but increasing the default timeout in puppet-jenkins seemed less downstream-friendly | 21:21 |
| *** jgrimm_ has quit IRC | 21:22 | |
| *** zz_ja has quit IRC | 21:22 | |
| openstackgerrit | greghaynes proposed openstack-infra/nodepool: WIP: Use shade to get the client objects https://review.openstack.org/168603 | 21:22 |
| jeblair | fungi: i feel like we're in no position to make promises about puppet-jenkins at this point, it's still a huge tangle of openstack specific code. i reckon wherever is fine for now :) | 21:22 |
| mordred | I agree with that | 21:22 |
| jeblair | fungi: (i mean, the real answer is in our openstack-specific hiera yaml file, but we don't have that yet) | 21:22 |
| *** maurosr has joined #openstack-infra | 21:22 | |
| openstackgerrit | greghaynes proposed openstack-infra/nodepool: WIP: Use shade to get the client objects https://review.openstack.org/168603 | 21:23 |
| *** jgrimm has joined #openstack-infra | 21:23 | |
| *** sabeen has joined #openstack-infra | 21:23 | |
| *** Longgeek has quit IRC | 21:23 | |
| *** zz_ja has joined #openstack-infra | 21:23 | |
| *** otter768 has joined #openstack-infra | 21:24 | |
| *** ddieterly has quit IRC | 21:26 | |
| *** esker has quit IRC | 21:28 | |
| *** david-lyle has quit IRC | 21:29 | |
| *** otter768 has quit IRC | 21:29 | |
| *** kgiusti has left #openstack-infra | 21:31 | |
| *** shashankhegde has joined #openstack-infra | 21:31 | |
| *** cdent has quit IRC | 21:33 | |
| openstackgerrit | Clint 'SpamapS' Byrum proposed openstack-infra/infra-manual: Initial infra manual for infra-cloud https://review.openstack.org/180762 | 21:34 |
| SpamapS | jeblair: ^ an initial distilling of the notes we took into something that resembles a framework for building a cloud. | 21:35 |
| SpamapS | VERY high level. | 21:35 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/subunit2sql: Add graph command to show aggregate counts for tests https://review.openstack.org/178039 | 21:35 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/subunit2sql: Add db api method to get per status counts for a test https://review.openstack.org/180742 | 21:35 |
| SpamapS | Every time I descended into the lower levels I realized we didn't actually answer many of those questions. | 21:35 |
| pleia2 | SpamapS: I probably missed the discussion, but why is this doc vs spec? | 21:36 |
| SpamapS | pleia2: I'm fuzzy on the details as it happend a while ago, but mostly because there's nothing to "change". | 21:37 |
| *** craigbr has joined #openstack-infra | 21:38 | |
| SpamapS | pleia2: and also because we'd just copy the content of the spec directly to the manual anyway. | 21:38 |
| SpamapS | So it's sort of like.. lets just document this as it should be, and then build on top of it from there. | 21:38 |
| pleia2 | I see, thanks | 21:38 |
| SpamapS | pleia2: as you can see, I'm not 100% married to the "just start in the manual" idea. :) | 21:39 |
| SpamapS | more of a 2nd-date-this-looks-interesting stage right now :) | 21:39 |
| pleia2 | SpamapS: so when you say "nothing to change" - will we be using the tripleo framework for managing a cloud, so that work is already pretty much done? | 21:40 |
| SpamapS | pleia2: as it says, no TripleO | 21:40 |
| *** bhunter71 has quit IRC | 21:40 | |
| SpamapS | pleia2: or rather, it doesn't say TripleO, it says Puppet. :) | 21:41 |
| pleia2 | SpamapS yeah, but I mean... we don't have puppet configs ourselves in -infra to manage an openstack cloud | 21:41 |
| *** asahlin is now known as asahlin_afk | 21:41 | |
| pleia2 | so it seems like a big change | 21:41 |
| SpamapS | pleia2: we have some in stackforge. :) | 21:41 |
| pleia2 | yeah, but you still need to glue them together, and it's not trivial | 21:41 |
| pleia2 | puppet-openstack is not a thing anymore, so you've got all your puppet-nova puppet-neutron etc bits there | 21:42 |
| SpamapS | pleia2: the idea being to make it work _as close_ to all the other infra things we have... just happens to be on top of an Ironic-all-in-one box. | 21:42 |
| SpamapS | pleia2: yes, the glueing things together is where infra comes in. :) | 21:42 |
| pleia2 | SpamapS: and that's why I'd vote for a spec | 21:42 |
| pleia2 | plus specs make it much easier for newcomers to look at one place for our big projects (I think running a cloud is a big project) | 21:43 |
| uberjay | is it possible to have nodepool not automatically delete an instance after it's no longer needed? I need to go dig a little deeper and see why some tests are failing, and the logs aren't illuminating enough. | 21:43 |
| SpamapS | pleia2: it's so big, it would be like, 10 specs. :) | 21:43 |
| pleia2 | SpamapS: indeed! | 21:43 |
| *** baoli has quit IRC | 21:43 | |
| SpamapS | pleia2: which is another reason it may be good to start with a not-spec, and then have specs for all the things. | 21:43 |
| SpamapS | or not, we could also have one big spec that spawns "babies" | 21:44 |
| pleia2 | SpamapS: I guess I have no objection to manual docs, but since I haven't yet seen a spec, I was confused about the order of operations (and a lot of data is very similar) | 21:44 |
| *** baoli has joined #openstack-infra | 21:44 | |
| greghaynes | SpamapS: oh no, heat convergence again | 21:44 |
| *** ajmiller has joined #openstack-infra | 21:44 | |
| SpamapS | greghaynes: I think Heat convergence is a cool idea... because lifeless and I came up with it. ;) | 21:44 |
| greghaynes | I just mean all the specs | 21:44 |
| SpamapS | Oh sorry | 21:45 |
| greghaynes | the specs that refer to sub specs | 21:45 |
| SpamapS | I thought maybe something was happening. :) | 21:45 |
| lifeless | SPECS | 21:45 |
| SpamapS | yes, heat convergence was at 5 specs when last I poked at it. | 21:45 |
| *** btully has quit IRC | 21:45 | |
| SpamapS | Because really, it was refactoring 90% of Heat.... | 21:45 |
| pleia2 | SpamapS: anyway, it's fine, just wondering :) | 21:45 |
| SpamapS | pleia2: thank you for asking. I'm wondering myself. :) | 21:45 |
| *** achanda has joined #openstack-infra | 21:46 | |
| greghaynes | well be talking about this at summit, maybe add that question to the etherpad? | 21:46 |
| SpamapS | is there already a slot for htis? | 21:46 |
| SpamapS | this | 21:46 |
| *** alexpilotti has quit IRC | 21:46 | |
| pleia2 | mordred added a placeholder description for it during our meeting (under fishbowl) | 21:47 |
| greghaynes | I think the coalescing into slots hasnt happened, but its on the summit etherpad | 21:47 |
| openstackgerrit | Clark Boylan proposed openstack-infra/gearman-plugin: Register the diff of functions https://review.openstack.org/180371 | 21:47 |
| *** packet has quit IRC | 21:47 | |
| clarkb | jeblair: zaro ^ that appears to work | 21:47 |
| clarkb | though I haven't tested with any labels yet, I need to setup a slave in my testbed | 21:47 |
| clarkb | uberjay: you can "hold" the node | 21:48 |
| clarkb | uberjay: do a nodepool hold $nodeid | 21:49 |
| *** jtomasek has quit IRC | 21:49 | |
| *** packet has joined #openstack-infra | 21:49 | |
| uberjay | clarkb: ah, so hold the node, then manually run my job, and it will stick around after failure? | 21:49 |
| clarkb | uberjay: ya, a held node should not be deleted | 21:50 |
| clarkb | uberjay: so when you are done with it you will want to nodepool delete $nodeid otherwise it won'y get cleaned up | 21:50 |
| uberjay | clarkb: excellent, thanks! i'll give it a shot | 21:50 |
| ianw | AJaeger: why abandon https://review.openstack.org/#/c/180309/ ? i think the template of the regex is a good idea | 21:50 |
| ianw | (maybe you just didn't want to rebase?) | 21:50 |
| *** achanda has quit IRC | 21:51 | |
| *** craigbr has quit IRC | 21:52 | |
| *** sdake has quit IRC | 21:53 | |
| *** Krinkle|detached is now known as Krinkle | 21:53 | |
| *** craigbr has joined #openstack-infra | 21:53 | |
| *** sdake has joined #openstack-infra | 21:53 | |
| *** sabeen has quit IRC | 21:54 | |
| *** baoli has quit IRC | 21:54 | |
| *** baoli has joined #openstack-infra | 21:55 | |
| *** mrmartin has quit IRC | 21:56 | |
| *** stevemar has quit IRC | 21:57 | |
| *** ajmiller_ has joined #openstack-infra | 21:58 | |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/project-config: Initial integration tests for storyboard https://review.openstack.org/156044 | 21:58 |
| *** zz_ja has quit IRC | 21:58 | |
| *** Krinkle is now known as Krinkle|detached | 21:58 | |
| clarkb | SpamapS: commented on that fwiw | 21:59 |
| *** maurosr has quit IRC | 21:59 | |
| openstackgerrit | Matthew Treinish proposed openstack-infra/subunit2sql: Add db api method to get per status counts for a test https://review.openstack.org/180742 | 22:01 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/subunit2sql: Add graph command to show aggregate counts for tests https://review.openstack.org/178039 | 22:01 |
| *** achanda has joined #openstack-infra | 22:01 | |
| *** ajmiller has quit IRC | 22:02 | |
| openstackgerrit | Merged openstack-infra/elastic-recheck: Update query for libvirt socket hangup bug 1451506 https://review.openstack.org/180684 | 22:02 |
| openstack | bug 1451506 in OpenStack Compute (nova) "spawn failed with "libvirtError: internal error: received hangup / error event on socket" in the gate" [Undecided,Confirmed] https://launchpad.net/bugs/1451506 | 22:02 |
| *** achanda has quit IRC | 22:02 | |
| *** Kennan2 has quit IRC | 22:02 | |
| *** achanda has joined #openstack-infra | 22:03 | |
| *** Kennan has joined #openstack-infra | 22:03 | |
| *** maurosr has joined #openstack-infra | 22:03 | |
| *** zz_ja has joined #openstack-infra | 22:06 | |
| *** jgrimm has quit IRC | 22:07 | |
| *** gordc has quit IRC | 22:07 | |
| openstackgerrit | Matthew Treinish proposed openstack-infra/subunit2sql: Add graph command to show aggregate counts for tests https://review.openstack.org/178039 | 22:07 |
| clarkb | pleia2: 07 is still looking good, so we should probably expect to do the others tomorrow | 22:07 |
| pleia2 | clarkb: great | 22:07 |
| *** jtriley has quit IRC | 22:07 | |
| *** zz_ja has quit IRC | 22:07 | |
| *** zz_ja has joined #openstack-infra | 22:07 | |
| clarkb | ok time to make a second container and see if I can make jenkins talk to you | 22:08 |
| clarkb | er s/you/it/ | 22:08 |
| *** dkranz has joined #openstack-infra | 22:09 | |
| *** jgrimm has joined #openstack-infra | 22:09 | |
| *** zz_ja has quit IRC | 22:10 | |
| *** maurosr has quit IRC | 22:12 | |
| *** ociuhandu has quit IRC | 22:14 | |
| *** maurosr has joined #openstack-infra | 22:14 | |
| *** zz_ja has joined #openstack-infra | 22:15 | |
| *** sdake has quit IRC | 22:17 | |
| krotscheck | clarkb,sdague,AJaeger: https://review.openstack.org/#/c/156044/ is a rebase with +2A that was blocked on a rebaseable merge conflict. Could one of you three drop the +A on it again? | 22:17 |
| *** sdake has joined #openstack-infra | 22:17 | |
| *** sdake has quit IRC | 22:17 | |
| clarkb | from my local jenkins trying to connect to a slave http://paste.openstack.org/show/215538/ (I can fix it, just wow) | 22:17 |
| *** sdake has joined #openstack-infra | 22:17 | |
| openstackgerrit | Steve Leon proposed openstack-infra/project-config: Add devstack gate job for Cue project https://review.openstack.org/180774 | 22:18 |
| *** adrian_otto has joined #openstack-infra | 22:18 | |
| fungi | i've currently got jenkins05 updating now. should be done with the last couple shortly | 22:18 |
| clarkb | krotscheck: done | 22:18 |
| fungi | jjb updating that is | 22:18 |
| krotscheck | clarkb: Thanks :) | 22:19 |
| fungi | puppet is reenabled on jenkins.o.o and 01-05 so far | 22:19 |
| *** packet has quit IRC | 22:19 | |
| *** dprince has quit IRC | 22:20 | |
| *** isviridov is now known as isviridov_away | 22:21 | |
| *** gokrokve_ has quit IRC | 22:23 | |
| clarkb | jeblair: zaro ok I have a slave attached to my master and its running jobs with label expressions so that looks good | 22:23 |
| jeblair | SpamapS, pleia2: so the idea was that most of what we eventually needed was detailed documentation for the system we're building; it seemed like we could just create the documentation, and treat that as the spec. | 22:24 |
| pleia2 | jeblair: ok, fair enough | 22:25 |
| jeblair | SpamapS: however, i think i may have miscommunicated that a bit, as i had intended that it go into system-config/docs | 22:25 |
| jeblair | SpamapS: which where http://ci.openstack.org/ is built from, and has our operational documentation, as opposed to infra-manual, which is "the user manual for the system". infra-cloud isn't user facing in the usual sense for us :) | 22:26 |
| jeblair | pleia2: anyway, in general i'm a fan of the specs process, and we might end up with one or more here; i thought this might end up being a special case though. we'll see? | 22:27 |
| pleia2 | jeblair: yep, we'll see :) | 22:27 |
| jogo | haha https://www.openstack.org/brand/interop/ uses github mirror not git.o.o | 22:28 |
| pleia2 | google rank of github (better than ours) makes it continue to be a tough sell | 22:28 |
| jeblair | wow, if only this were an open source project so we could change that | 22:28 |
| *** dizquierdo has quit IRC | 22:29 | |
| jeblair | hogepodge: since i can't submit a patch to do so, would you please change that? ^ | 22:29 |
| *** tonytan4ever has quit IRC | 22:30 | |
| *** DinaBelova has quit IRC | 22:30 | |
| *** tonytan4ever has joined #openstack-infra | 22:31 | |
| *** DinaBelova has joined #openstack-infra | 22:31 | |
| *** tonytan4ever has quit IRC | 22:31 | |
| *** tonytan4ever has joined #openstack-infra | 22:31 | |
| jogo | jeblair: even better they say " The list of required capabilities (with must-pass tests) and designated code sections are published on github and summarized below." | 22:31 |
| openstackgerrit | Merged openstack-infra/project-config: Initial integration tests for storyboard https://review.openstack.org/156044 | 22:31 |
| clarkb | jeblair: zaro and i have confirmed taht we RESET_ABILITIES instead of sending CANT_DO if we are told to unregister all functions | 22:32 |
| hogepodge | jogo yeah. thanks | 22:32 |
| pleia2 | \o/ | 22:32 |
| mordred | SpamapS: I added some comments to the doc patch - the patch itself shoudl be super easy to migrate to the system-config repo fwiw | 22:32 |
| jeblair | hogepodge: equivalent link: https://git.openstack.org/cgit/openstack/defcore/tree/2015.04.json | 22:32 |
| jogo | hogepodge: thanks | 22:33 |
| *** whoops has quit IRC | 22:33 | |
| jeblair | (i actually really like the openstack logo and repo destription up there) | 22:33 |
| hogepodge | jeblair: curious, why is one link worse than the other? | 22:34 |
| hogepodge | jogo ^ | 22:34 |
| jeblair | hogepodge: well, we put a lot of work into making a nice git server for openstack. it has openstack branding, it runs open source software, it's maintined by the community, and it's more reliable than github | 22:34 |
| hogepodge | jeblair: +1 | 22:35 |
| hogepodge | jeblair: thanks | 22:35 |
| jeblair | np | 22:35 |
| jeblair | hogepodge: pleia2 did the lion's share of the work, but we're all very proud of it :) | 22:35 |
| hogepodge | jeblair: I'll be sure to update links wherever else I find them. TIL | 22:36 |
| anteaya | yay pleia2 | 22:36 |
| pleia2 | hogepodge: that's great, thanks | 22:36 |
| jogo | jeblair: yeah our link looks good | 22:37 |
| *** annegentle has quit IRC | 22:38 | |
| *** sarob has quit IRC | 22:38 | |
| *** salv-orl_ has quit IRC | 22:38 | |
| openstackgerrit | Spencer Krum proposed openstack-infra/system-config: Add the ability for template to manage exim https://review.openstack.org/180782 | 22:39 |
| fungi | hogepodge: also, those third-party hosting sites have a tendency to close up and disappear over time (codehaus, gitorious, sourceforge, google code, et cetera) | 22:39 |
| jogo | jeblair: I am reading https://www.openstack.org/brand/ and having trouble seeing where the use of https://git.openstack.org/static/openstack.png in https://git.openstack.org/cgit/ is allowed | 22:39 |
| fungi | hogepodge: pretty sure git.openstack.org will be around as long as openstack itself | 22:39 |
| *** salv-orlando has joined #openstack-infra | 22:40 | |
| jeblair | jogo: note the "openstack.org" at the end of "git.openstack.org" :) | 22:40 |
| clarkb | jeblair: even if it wasn't the trademark policy explicitly allows the logo for non commercial community use | 22:40 |
| jogo | clarkb: its not that simple anymore | 22:41 |
| clarkb | pretty sure openstackgitmirror.foo.com could do the same | 22:41 |
| clarkb | as long as they don't host ads | 22:41 |
| *** baoli has quit IRC | 22:41 | |
| zaro | jeblair, clarkb, fungi, pleia2 : this just merged upstream and i think we need it for gerrit upgrade: https://review.openstack.org/#/c/180781 | 22:41 |
| *** baoli has joined #openstack-infra | 22:42 | |
| zaro | clarkb: have been roaming, will take a look at your gearman-plugin change now | 22:42 |
| fungi | zaro: i read through the bug report that's supposedly addressing and it had something to do with problems which only manifested when anonymous access is disallowed | 22:42 |
| jogo | You must keep sufficient space around our Brand Assets so they appear clean and uncluttered. | 22:43 |
| jogo | https://www.openstack.org/brand/openstack-logo/logo-download/ | 22:43 |
| uberjay | asselin: totally worked, thanks a lot -- I really appreciate it! | 22:43 |
| jogo | I don't think we can have the blue background :) | 22:43 |
| zaro | fungi: i must have missed that, re-reading again. | 22:43 |
| jeblair | jogo: the blue background was from the openstack website :) | 22:43 |
| *** dkranz has quit IRC | 22:44 | |
| hogepodge | jogo: jeblair: pleia2: fungi: all git links updated. | 22:44 |
| jogo | jeblair: that appears to contradict the latest usage guidelines | 22:44 |
| openstackgerrit | Spencer Krum proposed openstack-infra/system-config: Move afs servers to using o_p::template https://review.openstack.org/180783 | 22:44 |
| jogo | hogepodge: another question on https://www.openstack.org/brand/ try clicking the primary openstack logo | 22:44 |
| jogo | it goes to an email addr | 22:45 |
| *** annegentle has joined #openstack-infra | 22:45 | |
| fungi | zaro: https://code.google.com/p/gerrit/issues/detail?id=2296 supposedly we have that bug in our version? | 22:45 |
| nibalizer | jeblair: clarkb can you look at 180783 and 180782 to see if that looks like what we discussed in channel? | 22:45 |
| *** emagana has joined #openstack-infra | 22:45 | |
| jogo | 'Use of the primary logo is limited to educational and community building activities, for example educational presentations about OpenStack that are not attached to a commercial product or brand, or a blog post to talk about your involvement in the OpenStack community, not tied to a commercial product or brand' | 22:46 |
| zaro | fungi: yeah, our version on jgit 3.7.0, fix for that was to update to 3.7.1 | 22:46 |
| jogo | if I say I work for HP in a slide deck can I have the OpenStack logo as well? | 22:46 |
| jogo | mordred: ^ | 22:47 |
| zaro | fungi: and the one i just pushed up is a fix to that update | 22:47 |
| fungi | zaro: i'm not opposed to adding more bugfixes to that branch and test again before we upgrade, but there are likely to be more that come up after we upgrade too so we'll need to plan for some future point release updates anyway | 22:47 |
| hogepodge | jogo: I'll ask. I'm sure the e-mail creates a ticket, but it is weird to have it as a header. | 22:48 |
| jogo | hogepodge: yeah | 22:48 |
| *** adrian_otto has quit IRC | 22:48 | |
| jogo | hogepodge: I am find it amusing how confusing the trademark policy is for community use | 22:48 |
| zaro | fungi: are you proposing to wait and see? | 22:48 |
| hogepodge | jogo I think it's being reviewed right now, and hopefully will improve | 22:50 |
| mtreinish | jogo: heh, are you pointing out the issues with the openstack.org site? I remember when we went through that and submitted feedback forms on everything | 22:50 |
| *** mrunge has quit IRC | 22:50 | |
| mtreinish | I guess nothing changed | 22:50 |
| jogo | hogepodge: for example see my quote and question above. | 22:50 |
| jogo | mtreinish: oh I forgot about that, I found typos in the bylaws | 22:51 |
| jogo | hogepodge: glad to see it being cleaned up | 22:51 |
| fungi | zaro: only if you don't think there's time to test it and confirm that we're still in good shape to upgrade after that patch which just merged today is added | 22:51 |
| hogepodge | jogo typos? | 22:51 |
| fungi | jogo: hogepodge: the restricted use logos url mentioned at https://www.openstack.org/brand/openstack-logo/ is also 404 | 22:52 |
| mordred | jogo: well, I'm not sure that's an infraqusetion really ... but I use openstack logos on my slides all the time | 22:52 |
| hogepodge | I'd totally be game for posting issues here https://etherpad.openstack.org/p/openstack-org-bug-reports | 22:52 |
| hogepodge | and can follow up on them | 22:52 |
| fungi | hogepodge: or just convince people to open lp bugs for openstack-org | 22:53 |
| hogepodge | I know it's not the greatest bug tracker in the world. Or we could just create something in launchpad or watever the preferred bug tool is | 22:53 |
| Rockyg | hogepodge, jogo: and all the freakin' typefaces.....UGGGGGGLY | 22:53 |
| mtreinish | fungi: is it tracked on lp? | 22:53 |
| hogepodge | fungi that | 22:53 |
| fungi | mtreinish: yes | 22:53 |
| *** craigbr has quit IRC | 22:53 | |
| jogo | hogepodge: the bylaws at one point had bad links in them etc. | 22:53 |
| fungi | all things our community would just fix... once the www site is being automatically deployed from public source in gerrit | 22:54 |
| *** Krinkle|detached is now known as Krinkle | 22:54 | |
| Rockyg | fungi: ++ | 22:54 |
| jogo | mordred: no its not infra, asking you as a BoD member | 22:55 |
| mordred | jogo: ah! gotcha | 22:55 |
| mordred | jogo: so, I believe the intent is taht openstack developers should be able to talk about openstack without fear of reprisal from the openstack foundation | 22:55 |
| Rockyg | Uhoh, time for some political doublespeak! | 22:55 |
| mtreinish | fungi: maybe that should be somewhere on the site then. It's not at all obvious, and at one point I didn't think that was the case | 22:55 |
| jogo | mordred: I hope so, but that isn't what it says. If I say I work for HP that is partially commercial by HP | 22:55 |
| jogo | HP gets commercial value out of that ... I think | 22:56 |
| mordred | I do not believe that I agree with that | 22:56 |
| mordred | I believe if you were talking about HP and HP products that would be correct | 22:56 |
| zaro | fungi: reading again it seems like the upgrade to jgit 3.7.1 was a mistake but instead of going back to 3.7.0 they went to 4.0.0 | 22:56 |
| *** bswartz has joined #openstack-infra | 22:56 | |
| mordred | I believe mentioning that HP employs you is a statement of fact that does not obviate the fact that you are an openstack developer | 22:56 |
| jogo | you getting in front of a room of people and saying impressive things and saying I work for HP is good PR for HP no? | 22:56 |
| zaro | fungi: so yeah, maybe just better to stay with 3.7.0 for now. | 22:56 |
| fungi | mtreinish: https://wiki.openstack.org/wiki/Website | 22:56 |
| jogo | mordred: hopefully, just pointing out the phrasing is open to interpretation | 22:57 |
| mordred | jogo: sure. but you are not standing up on stage talking about how awesome hp is and using openstack logos to do it | 22:57 |
| fungi | mtreinish: i agree, the www site lacks the usual "how to report a problem with this site" links | 22:57 |
| *** armax has joined #openstack-infra | 22:57 | |
| jogo | mordred: I bet someone somewhere has, but this is just a fun tangent | 22:58 |
| jogo | fungi: ++ | 22:58 |
| *** SumitNaiksatam has quit IRC | 22:58 | |
| mordred | jogo: :) | 22:58 |
| mtreinish | fungi: heh, yeah :) | 22:58 |
| fungi | mtreinish: might be another good thing to put in a bug report ;) | 22:58 |
| jogo | . The framework for the www.openstack.org site is maintained at https://github.com/toddmorey/openstack-org, and patches are welcome | 22:58 |
| *** bknudson has quit IRC | 22:58 | |
| jogo | lol openstack.org uses github | 22:58 |
| fungi | jogo: also last updated 4 years ago | 22:59 |
| jogo | oh that is 4 years old | 22:59 |
| jogo | yeah ... | 22:59 |
| reed | jogo, I think that information is old | 22:59 |
| fungi | i think that wiki article is a little moldy around the edges | 22:59 |
| reed | fungi, which one? | 22:59 |
| reed | Website? | 22:59 |
| fungi | reed: https://wiki.openstack.org/wiki/Website | 22:59 |
| fungi | yeah | 22:59 |
| reed | lets tag it as old | 23:00 |
| * reed tags | 23:00 | |
| jeblair | https://git.openstack.org/cgit/openstack-infra/openstackweb/ | 23:00 |
| fungi | however i can confirm that the e-mail address for todd and the bug tracker in lp are still correct and active | 23:00 |
| jeblair | hasn't been touched in 4 months | 23:00 |
| reed | https://github.com/OpenStackweb/ is newer | 23:01 |
| fungi | and yes, openstack-infra/openstackweb once we get the go-ahead from tipit that they're ready to start using that, at which point i'll refresh it from https://github.com/OpenStackweb/openstack-org | 23:01 |
| reed | or maybe not | 23:01 |
| pleia2 | I think our git.o.o was last updated when clarkb was working on https://review.openstack.org/#/c/141571/ | 23:01 |
| *** packet has joined #openstack-infra | 23:01 | |
| jeblair | reed: wait... so further activity is happening on the github repo? | 23:02 |
| reed | https://github.com/OpenStackweb/openstack-org it is newer | 23:02 |
| fungi | pleia2: right, i don't want to keep pushing into gerrit for them until they confirm they're going to start using it | 23:02 |
| pleia2 | fungi: yep | 23:02 |
| jeblair | reed: why not gerrit? | 23:02 |
| openstackgerrit | melanie witt proposed openstack-infra/elastic-recheck: Add query for libvirt cannot recv data bug 1452485 https://review.openstack.org/180787 | 23:02 |
| openstack | bug 1452485 in OpenStack Compute (nova) "Error from libvirt "Cannot recv data: Connection reset by peer"" [Undecided,New] https://launchpad.net/bugs/1452485 | 23:02 |
| reed | jeblair, i'm as surprised as you are | 23:02 |
| * fungi is less surprised, more just frustrated | 23:02 | |
| *** annegentle has quit IRC | 23:02 | |
| jeblair | reed: okay, so we just need to decide on a day, we'll push the latest into gerrit, and then they can use that. | 23:03 |
| pleia2 | I hold out hope we're still inching to the finish line | 23:03 |
| reed | the positive thing is that the repo is public | 23:03 |
| jeblair | reed: can you help coordinate that? | 23:03 |
| hogepodge | one should always assume a wiki article is moldy. Wiki is moldy by design | 23:03 |
| reed | jeblair, I can ask, that's all I can do at the moment | 23:03 |
| *** dustins has quit IRC | 23:03 | |
| *** samueldmq has joined #openstack-infra | 23:04 | |
| jeblair | reed: okay, i need someone at the foundation to take responsibility for this | 23:04 |
| *** armax has quit IRC | 23:04 | |
| fungi | i think ttx plans to walk that gauntlet | 23:04 |
| reed | jeblair, it's something we need to discuss with Thierry | 23:04 |
| jogo | jeblair: btw I like the favicon from infra way better then openstack.org | 23:05 |
| jogo | reed: ^ | 23:05 |
| jeblair | reed: ok | 23:05 |
| jogo | the .org has a ugly white box around it | 23:05 |
| *** maurosr has quit IRC | 23:05 | |
| fungi | jogo: i think that's logo wonks getting pedantic about allowed background colors | 23:05 |
| jogo | fungi: yup! | 23:06 |
| fungi | transparent background on a logo means you have no control over the background color! onoes! | 23:06 |
| pleia2 | heh | 23:06 |
| jogo | fungi: logo wonks would disapprove of https://review.openstack.org/ | 23:06 |
| jeblair | ttx: let's work on setting a date to switch the openstack-org workflow to gerrit. we don't need to do anything else, just get them to submit new changes through gerrit. they can self-approve with no testing and continue deploying the way they are now. just a switch to gerrit is all. | 23:06 |
| jhesketh | Morning | 23:07 |
| *** zz_ja has quit IRC | 23:07 | |
| jeblair | ttx: we'll set a date to make that change, we will push the latest code into gerrit, and then they'll stop using the github repo | 23:07 |
| fungi | (amd in fact some of the tipit staff already do this for openstackid now) | 23:07 |
| *** maurosr has joined #openstack-infra | 23:08 | |
| *** zz_ja has joined #openstack-infra | 23:09 | |
| jogo | great bug https://bugs.launchpad.net/openstack-org/+bug/1315240 | 23:09 |
| openstack | Launchpad bug 1315240 in openstack-org "http://www.openstack.org/community/ congratulates jenkins frequently" [Low,Confirmed] | 23:09 |
| jeblair | hehe :) | 23:10 |
| fungi | that's no bug | 23:10 |
| fungi | well, okay, it _should_ be congratulating zuul | 23:10 |
| fungi | or the infra team | 23:10 |
| *** baoli has quit IRC | 23:10 | |
| jeblair | i hear the openstack proposal bot is a top contributor. a veritable _machine_. | 23:10 |
| anteaya | ha ha ha | 23:11 |
| fungi | indeed. i've had to disqualify her from several elections already | 23:11 |
| fungi | on mere technicalities, mind you | 23:11 |
| jogo | fungi: even better since it uses some launchpad feed things like https://launchpad.net/skylable get picked up | 23:12 |
| mtreinish | fungi: that feels like discrimination against non-human people | 23:12 |
| jogo | since openstack org is an open group | 23:12 |
| fungi | mtreinish: brushing up on your asimov i see | 23:12 |
| *** ondergetekende has quit IRC | 23:13 | |
| fungi | ("the bicentennial man" is one of my favorite explorations of personal rights) | 23:13 |
| openstackgerrit | melanie witt proposed openstack-infra/elastic-recheck: Add query for libvirt cannot recv data bug 1252947 https://review.openstack.org/180787 | 23:13 |
| openstack | bug 1252947 in OpenStack Compute (nova) "libvirtError: Cannot recv data: Connection reset by peer" [Undecided,Confirmed] https://launchpad.net/bugs/1252947 | 23:13 |
| *** adrian_otto has joined #openstack-infra | 23:13 | |
| mtreinish | fungi: heh, man it's been a while since I read any asimov. Probably too long | 23:14 |
| zaro | jeblair: about https://review.openstack.org/#/c/172534/ | 23:15 |
| mtreinish | yeah, "The Bicentennial Man" is great | 23:15 |
| zaro | jeblair: the bouncy castle jars are not packaged into the war | 23:15 |
| fungi | zaro: what was unpacking it then? | 23:15 |
| *** ondergetekende has joined #openstack-infra | 23:15 | |
| zaro | jeblair: it's like the mysql lib, it's downloaded | 23:16 |
| fungi | zaro: oh--the gerrit init exec downloads it? | 23:16 |
| zaro | i believe that's due to licencing | 23:16 |
| *** hemna is now known as hemnafk | 23:16 | |
| zaro | fungi: yep, so it's problematic | 23:16 |
| jeblair | fungi, zaro: ah, yeah, the gerrit init must have done it then... | 23:16 |
| fungi | wow. double-ick | 23:16 |
| jeblair | fungi, zaro: so we should probably either rely on that, or find a way to disable that and do what zaro was doing in his patch | 23:17 |
| jogo | haha the community page is pointing to bzr | 23:17 |
| *** panda has quit IRC | 23:17 | |
| zaro | relying on that is problematic i think | 23:17 |
| zaro | because it may download different versions on updates to gerrit | 23:17 |
| anteaya | jogo: while your are at it can you do something about SEO for things | 23:17 |
| jogo | https://code.launchpad.net/nova | 23:18 |
| anteaya | because I would like folks to find our docs easier | 23:18 |
| fungi | jogo: maybe you should stop looking at that website. it's going to just keep disappointing you | 23:18 |
| zaro | then we would be left with different versions the file system | 23:18 |
| *** panda has joined #openstack-infra | 23:18 | |
| jogo | fungi: I am having fun laughing at how bad it is | 23:18 |
| fungi | zaro: great point. the filename is variable, so we'd end up having to clean any out prior to running init again | 23:18 |
| anteaya | jogo: as long as you are enjoying yourself | 23:19 |
| jeblair | jogo: i think it's a good website i'd like to be able to help maintain :) | 23:19 |
| zaro | jeblair: not sure if it's diableable, but deleting the jars is basically acomplishing the same thing | 23:19 |
| jogo | jeblair: ++, the rough edges just need some polish | 23:20 |
| *** weshay has quit IRC | 23:22 | |
| zaro | jeblair: don't think it's possible to disable downloading of those libs, at least there's no option to disable for 'init' command | 23:23 |
| jeblair | zaro: i agree | 23:24 |
| jeblair | zaro: okay, so you're proposing to keep "tidying" the gerrit-downloaded jars and use our downloaded jars as in your change | 23:24 |
| *** packet has quit IRC | 23:24 | |
| *** zz_dimtruck is now known as dimtruck | 23:24 | |
| zaro | jeblair: yes, unless you think there's a better way. | 23:24 |
| openstackgerrit | melanie witt proposed openstack-infra/elastic-recheck: Add query for libvirt cannot recv data bug 1252947 https://review.openstack.org/180787 | 23:25 |
| openstack | bug 1252947 in OpenStack Compute (nova) "libvirtError: Cannot recv data: Connection reset by peer" [Undecided,Confirmed] https://launchpad.net/bugs/1252947 | 23:25 |
| *** otter768 has joined #openstack-infra | 23:25 | |
| jeblair | zaro: no, that makes sense. and it looks like your change is still compatible with the tidy change because you use different names for the jars you download | 23:26 |
| jeblair | zaro: so i think you're change is what we need then | 23:26 |
| jeblair | fungi: sound right to you too? | 23:26 |
| fungi | jeblair: yeah, i'm good with it | 23:26 |
| jeblair | fungi, zaro: oh, one more thing -- do we want to put the new stuff in a conditional based on gerrit version? | 23:27 |
| jeblair | so the puppet-gerrit module can continue (at least for a while as long as someone cares) to be used for 2.8? | 23:27 |
| anteaya | it would be clearer what is going on | 23:27 |
| anteaya | and gerrit downloaded jars aren't triggered in any automated way, are they? | 23:28 |
| fungi | i guess we would do that with an additional class parameter? or a comparison on a variable which is expanded as part of the war filename? | 23:28 |
| openstackgerrit | melanie witt proposed openstack-infra/elastic-recheck: Add context "libvirt" back into query for bug 1451506 https://review.openstack.org/180793 | 23:29 |
| openstack | bug 1451506 in OpenStack Compute (nova) "spawn failed with "libvirtError: internal error: received hangup / error event on socket" in the gate" [Undecided,Confirmed] https://launchpad.net/bugs/1451506 | 23:29 |
| zaro | that's doable, you can get version info from war with --version | 23:29 |
| fungi | zaro: well, i'm wondering how to feed that back into a puppet variable to switch on in the manifest though | 23:30 |
| zaro | jeblair: i'll give it a try. | 23:30 |
| zaro | fungi: that's probably how i would imagine doing it | 23:30 |
| anteaya | it executes if contactstore == true | 23:30 |
| zaro | fungi: is that puppet possible? | 23:30 |
| *** otter768 has quit IRC | 23:31 | |
| anteaya | when does contactstore == true? | 23:31 |
| *** nelsnelson has quit IRC | 23:31 | |
| fungi | zaro: it's not something i've ever tried, which is why i point out that it's an interesting mechanism to be considering for that purpose | 23:31 |
| *** craigbr has joined #openstack-infra | 23:31 | |
| fungi | anteaya: it's passed in as a class parameter | 23:32 |
| zaro | anteaya: it's always true for our gerrit | 23:32 |
| anteaya | thanks | 23:32 |
| *** Rockyg has quit IRC | 23:32 | |
| *** rfolco has quit IRC | 23:34 | |
| zaro | alright, i'll look into making it work for both version, seems like it would be useful for others. | 23:34 |
| SpamapS | jeblair: you did not miscommunicate. I'm just unfamiliar with the layout of infra still. :) Moving the patch over. | 23:34 |
| *** dims_ has joined #openstack-infra | 23:36 | |
| *** soren has quit IRC | 23:37 | |
| *** dims has quit IRC | 23:38 | |
| *** dimtruck is now known as zz_dimtruck | 23:39 | |
| openstackgerrit | Clint 'SpamapS' Byrum proposed openstack-infra/system-config: Initial manual for infra-cloud https://review.openstack.org/180796 | 23:39 |
| SpamapS | hm thats weird I'd swear I kept the change-id | 23:42 |
| *** sarob has joined #openstack-infra | 23:42 | |
| SpamapS | Before I thrash a bit... thoughts? Better to just make it a new review, or keep the change-id? | 23:42 |
| clarkb | its new either way | 23:43 |
| anteaya | it is the same change-id, it is a new repo | 23:45 |
| anteaya | so gerrit makes a new review number since the repo is different | 23:46 |
| SpamapS | got it | 23:46 |
| SpamapS | won't worry about it then | 23:46 |
| anteaya | you linked in comments so the history can be tracked | 23:46 |
| *** david-lyle has joined #openstack-infra | 23:47 | |
| *** mmedvede_ has joined #openstack-infra | 23:47 | |
| jesusaurus | has infra had any problems with gerrit replication causing the web ui and ssh api to be unresponsive? | 23:47 |
| fungi | SpamapS: yeah, you can see it's the same change id by going to https://review.openstack.org/#/q/Icb35adf70fa98e64dbbe0464b95af3b5de51a980,n,z | 23:48 |
| fungi | jesusaurus: what gerrit release? | 23:48 |
| fungi | jesusaurus: if 2.10.3 then sounds familiar to https://groups.google.com/forum/#!msg/repo-discuss/Kv4bWLESbQ4/-oSNbuTQwkUJ | 23:49 |
| jesusaurus | fungi: no, I'm stil on an ancient 2.4 | 23:50 |
| fungi | jesusaurus: nope, doesn't sound familiar then | 23:50 |
| jesusaurus | hrm, okay | 23:51 |
| *** _nadya_ has joined #openstack-infra | 23:54 | |
| *** ayoung has quit IRC | 23:55 | |
| *** signed8bit is now known as signed8bit_ZZZzz | 23:56 | |
| *** _nadya_ has quit IRC | 23:59 | |
| *** ZZelle_ has quit IRC | 23:59 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!
