| cathy_ | jeblair: Can we be added as owner of this spec since we are the authors of this spec and answer all review comments and posting all these version updates | 00:00 |
|---|---|---|
| *** heyongli has quit IRC | 00:00 | |
| *** mattymo has joined #openstack-infra | 00:00 | |
| *** mattymo has quit IRC | 00:00 | |
| cathy_ | jesusaurus: I just added +1 to the workflow. Could you check if that is OK now? How long does it take for the merge to be completed? | 00:01 |
| jesusaurus | clarkb: I don't think that's needed, the change addresses my need to put the logstash indexer and the log_processor worker on different hosts | 00:01 |
| *** dingyichen has joined #openstack-infra | 00:01 | |
| clarkb | jesusaurus: out of curiousity what prompted that need? the indexer is the resource hog so colocating shouldn't be a major issue (its why I did it) | 00:01 |
| *** salv-orlando has joined #openstack-infra | 00:01 | |
| *** dannywilson has quit IRC | 00:02 | |
| pleia2 | cathy_: depends on how many tests are run on it, you can check status of any change by going to http://status.openstack.org/zuul/ and searching for the change number, in your case: 192933 | 00:02 |
| *** mattymo has joined #openstack-infra | 00:02 | |
| *** mattymo has quit IRC | 00:02 | |
| *** openstack has joined #openstack-infra | 00:05 | |
| cathy_ | pleia2: yes, I have addressed his comments and also he participated in today's IRC meeting and he is Ok with merging this in his email to me. | 00:05 |
| *** yamamoto has joined #openstack-infra | 00:06 | |
| *** salv-orlando has quit IRC | 00:06 | |
| *** mattymo has joined #openstack-infra | 00:06 | |
| *** mattymo has quit IRC | 00:06 | |
| cathy_ | pleia2: Do you see anything missing in https://review.openstack.org/#/c/192933/ for it to be merged from process perspective? I did not know that I need to approve the work flow before. | 00:07 |
| pleia2 | cathy_: see the bottom of the patch, he left messages 5 minutes ago | 00:07 |
| pleia2 | cathy_: he does not agree this is ready to merge | 00:07 |
| pleia2 | cathy_: this isn't really an infra thing, you just need to work with your other core reviewers on this | 00:07 |
| *** heyongli has quit IRC | 00:07 | |
| cathy_ | Ok, I see | 00:08 |
| pleia2 | :) | 00:08 |
| cathy_ | let me take a look at his latest comment. | 00:08 |
| *** mattymo has joined #openstack-infra | 00:09 | |
| *** mattymo has quit IRC | 00:09 | |
| clarkb | asselin: it looks like your key to me | 00:09 |
| clarkb | asselin: wha tam I missing? | 00:09 |
| openstackgerrit | Merged openstack-infra/system-config: Note to escape literal $ in invite2summit template https://review.openstack.org/200517 | 00:09 |
| openstackgerrit | Merged openstack-infra/puppet-project_config: Add a common data directory for job workers https://review.openstack.org/200694 | 00:09 |
| openstackgerrit | Merged openstack-infra/system-config: Adjust Zanata admins https://review.openstack.org/201808 | 00:09 |
| openstackgerrit | Merged openstack-infra/system-config: Add more sites to our SSL cert expiration checker https://review.openstack.org/202757 | 00:09 |
| *** mayurig has quit IRC | 00:09 | |
| asselin | clarkb, not sure what I missing either....started about 2 days ago....? | 00:09 |
| clarkb | asselin: I don't understand your question is what I am saying | 00:10 |
| clarkb | the key matches wht you provide | 00:10 |
| clarkb | but you say its not installing your key? | 00:10 |
| *** mattymo has joined #openstack-infra | 00:11 | |
| asselin | clarkb, you are right | 00:11 |
| *** mattymo has quit IRC | 00:11 | |
| *** heyongli has joined #openstack-infra | 00:11 | |
| * asselin needs to take a break | 00:11 | |
| *** tsekiyam_ has joined #openstack-infra | 00:11 | |
| *** hdd has quit IRC | 00:12 | |
| *** boris-42 has quit IRC | 00:12 | |
| cathy_ | armax: are you here? Sorry I did not get the "HTML in visible" issue. So does it mean I need to commit the index file to add the "api"? | 00:12 |
| *** mattymo has joined #openstack-infra | 00:13 | |
| *** mattymo has quit IRC | 00:13 | |
| armax | cathy_: yes | 00:13 |
| cathy_ | armax: OK, will do right now. | 00:13 |
| reed | has anybody seen this before: shade.exc.OpenStackCloudException: Error fetching image list: Error in connecting to glance: No module named cryptography.hazmat.bindings.openssl.binding | 00:13 |
| armax | cathy_: look here: http://docs-draft.openstack.org/33/192933/12/check/gate-networking-sfc-docs/1dbf6b3//doc/build/html/ | 00:13 |
| armax | cathy_: there’s no href rendered into the html page, so the doc won’t be visible | 00:14 |
| clarkb | reed: that looks like your openssl install isn't quite right, try pip install -U pyopenssl and make sure you have the dev headers for its deps installed | 00:14 |
| armax | cathy_: look at the patchset I posted in the review page | 00:14 |
| armax | cathy_: that’s your cue | 00:14 |
| *** esmute has joined #openstack-infra | 00:14 | |
| reed | clarkb, that's what I suspected... thanks | 00:14 |
| *** mattymo has joined #openstack-infra | 00:15 | |
| *** mattymo has quit IRC | 00:15 | |
| *** tsekiyama has quit IRC | 00:15 | |
| *** mtanino has quit IRC | 00:15 | |
| *** Sukhdev has quit IRC | 00:16 | |
| *** tsekiyam_ has quit IRC | 00:16 | |
| reed | clarkb, the weird thing is that this is a basically stock Ubuntu 15.04, i haven't messed up with it (yet) :) | 00:16 |
| *** FallenPegasus has quit IRC | 00:17 | |
| *** mattymo has joined #openstack-infra | 00:17 | |
| *** mattymo has quit IRC | 00:17 | |
| cathy_ | armax: done. Could you take a look and see if everything is OK. Thanks for the pointer! | 00:17 |
| reed | and in fact, Requirement already up-to-date: pyopenssl in /usr/local/lib/python2.7/dist-packages | 00:17 |
| *** bhunter71 has joined #openstack-infra | 00:17 | |
| armax | cathy_: it looks good, other than that, there were couple of other suggestions made in PS12 | 00:17 |
| *** achanda has quit IRC | 00:17 | |
| armax | cathy_: you may want to look at those too | 00:17 |
| *** rlandy has joined #openstack-infra | 00:18 | |
| cathy_ | armax: I have addressed them. I don't think they impact the merge | 00:18 |
| armax | cathy_: well, you haven’t | 00:18 |
| *** dalgaaf has quit IRC | 00:18 | |
| clarkb | reed try doing the upgrade with cryptographt | 00:18 |
| cathy_ | armax: I have replied to all comments | 00:19 |
| *** mattymo has joined #openstack-infra | 00:19 | |
| *** mattymo has quit IRC | 00:19 | |
| *** sputnik13 has quit IRC | 00:19 | |
| armax | cathy_: agreed, but it’s important to capture those comments or the outcome of the discussion back into the doc | 00:19 |
| armax | cathy_: referring to the review history can be problematic especially when many people are trying to be on the same page | 00:19 |
| armax | cathy_: also look at jeblair’s suggestion about the right use of the co-authored-by tag | 00:20 |
| armax | cathy_: that way, the ownership is correctly assigned to you and louis | 00:21 |
| reed | clarkb, that solved it, thanks | 00:21 |
| *** mattymo has joined #openstack-infra | 00:21 | |
| *** mattymo has quit IRC | 00:21 | |
| cathy_ | OK, I will correct the co-author tag. and update the singular to plural. and upload a new version. Anything else? Let me take a look at my replies again | 00:21 |
| cathy_ | armax: Ok, For the parameter one, you think I should add my comments to that section to make it more clear? | 00:23 |
| *** mattymo has joined #openstack-infra | 00:23 | |
| *** mattymo has quit IRC | 00:23 | |
| armax | cathy_: that would be very helpful, I think | 00:23 |
| armax | cathy_: your response was spot on, but I wouldn’t want to see that type of detail missing on the doc, at least in my opinion | 00:23 |
| armax | cathy_: granted we can iterate, it sounds like that’s the type of detail we can capture now rather than later | 00:24 |
| *** Swami has quit IRC | 00:25 | |
| *** mattymo has joined #openstack-infra | 00:25 | |
| *** mattymo has quit IRC | 00:25 | |
| *** hogepodge has joined #openstack-infra | 00:25 | |
| cathy_ | armax: OK, I will add my response to your comment on parameter, correct the singular, clarify ipv4 and ipv6 are both supported. Anything else? I really like to get all into this latest version so that it can be merged and I can concentrate on writing the design document:-) | 00:26 |
| *** mattymo has joined #openstack-infra | 00:27 | |
| *** mattymo has quit IRC | 00:27 | |
| armax | cathy_: nothing from me, but usually your best bet is to multi-task! You should not serialize tasks when it comes to open source! | 00:27 |
| armax | cathy_: for the things that make sense multi-task of course; it looks like you have enough to get you going, and imo this doc is ready to roll | 00:28 |
| armax | cathy_: I haven’t looked closely at the other -1's | 00:28 |
| *** davideagnello has joined #openstack-infra | 00:28 | |
| *** esmute has quit IRC | 00:28 | |
| *** madhu_ak has quit IRC | 00:29 | |
| *** mattymo has joined #openstack-infra | 00:29 | |
| *** mattymo has quit IRC | 00:29 | |
| *** sdake has joined #openstack-infra | 00:30 | |
| *** mattymo has joined #openstack-infra | 00:31 | |
| *** mattymo has quit IRC | 00:31 | |
| *** woodster_ has quit IRC | 00:32 | |
| *** madhu_ak has joined #openstack-infra | 00:32 | |
| cathy_ | armax: OK, the new version will be uploaded soon. I am multitasking:-) Just do not want to be side tracked by this merging issue and get emails from people asking this:-) | 00:32 |
| armax | cathy_: you’re not alone, we all share the same pain | 00:33 |
| *** mattymo has joined #openstack-infra | 00:33 | |
| *** mattymo has quit IRC | 00:33 | |
| cathy_ | armax: I have to multitasking since I have work in other open source and standard organizations:-) | 00:33 |
| cathy_ | armax: understand! | 00:33 |
| armax | cathy_: again, you’re not alone | 00:33 |
| armax | cathy_: you’re doing great, keep it up! | 00:34 |
| cathy_ | armax: Thanks! I will post the new version soon. Please approve it if you see OK. | 00:34 |
| armax | ok | 00:34 |
| *** mattymo has joined #openstack-infra | 00:35 | |
| *** mattymo has quit IRC | 00:35 | |
| *** madhu_ak_ has joined #openstack-infra | 00:35 | |
| *** btully has quit IRC | 00:35 | |
| *** achanda has joined #openstack-infra | 00:36 | |
| *** sigmavirus24 is now known as sigmavirus24_awa | 00:37 | |
| *** madhu_ak has quit IRC | 00:37 | |
| *** mattymo has joined #openstack-infra | 00:37 | |
| *** mattymo has quit IRC | 00:37 | |
| *** esmute has joined #openstack-infra | 00:37 | |
| *** ZZelle_ has quit IRC | 00:38 | |
| *** SumitNaiksatam has quit IRC | 00:39 | |
| *** mattymo has joined #openstack-infra | 00:39 | |
| *** mattymo has quit IRC | 00:39 | |
| *** SumitNaiksatam has joined #openstack-infra | 00:39 | |
| *** esmute has quit IRC | 00:40 | |
| *** mattymo has joined #openstack-infra | 00:41 | |
| *** mattymo has quit IRC | 00:41 | |
| *** mattymo has joined #openstack-infra | 00:43 | |
| *** mattymo has quit IRC | 00:43 | |
| *** rlandy has quit IRC | 00:45 | |
| reed | clarkb, where can I find sample code using shade? | 00:45 |
| *** mattymo has joined #openstack-infra | 00:45 | |
| *** mattymo has quit IRC | 00:45 | |
| *** laughterwym has joined #openstack-infra | 00:45 | |
| reed | like somewhere where I can see how create_server works :) | 00:45 |
| *** rlandy has joined #openstack-infra | 00:45 | |
| *** SumitNaiksatam has quit IRC | 00:46 | |
| *** mattymo has joined #openstack-infra | 00:47 | |
| *** mattymo has quit IRC | 00:47 | |
| *** arxcruz has quit IRC | 00:48 | |
| cathy_ | armax: new version uploaded. Could you check to see if it is OK? I will wait for your reply here:-) Thanks. | 00:48 |
| *** mattymo has joined #openstack-infra | 00:49 | |
| *** mattymo has quit IRC | 00:49 | |
| clarkb | reed ansible uses it | 00:49 |
| * reed lookoing at https://github.com/emonty/ansible-modules-core/blob/devel/cloud/openstack/os_server.py | 00:50 | |
| armax | cathy_: looks good | 00:50 |
| *** oomichi has joined #openstack-infra | 00:50 | |
| cathy_ | cool. Could you give it +2? I will give it too. Thanks | 00:50 |
| *** mattymo has joined #openstack-infra | 00:51 | |
| *** mattymo has quit IRC | 00:51 | |
| cathy_ | armax: the co-author syntax also looks correct, right? | 00:52 |
| armax | cathy_: one final nit | 00:52 |
| armax | no | 00:52 |
| armax | pe | 00:52 |
| cathy_ | armax: ? what is "pe"? | 00:53 |
| armax | nope | 00:53 |
| armax | cathy_: I’ll fix it | 00:53 |
| *** mattymo has joined #openstack-infra | 00:53 | |
| *** mattymo has quit IRC | 00:53 | |
| cathy_ | armax: great. Thanks! | 00:53 |
| armax | cathy_: there | 00:54 |
| openstackgerrit | Merged openstack-infra/system-config: Add subnode logs to logstash https://review.openstack.org/176499 | 00:54 |
| cathy_ | armax: could you clarify what is "pe"? I would like to learn:-) | 00:54 |
| *** otter768 has joined #openstack-infra | 00:55 | |
| armax | cathy_: it was me typing nope to fast | 00:55 |
| armax | cathy_: it turned out ‘no’ and ‘pe' | 00:55 |
| armax | cathy_: nothing else | 00:55 |
| *** mattymo has joined #openstack-infra | 00:55 | |
| *** mattymo has quit IRC | 00:55 | |
| *** earlephilhower has quit IRC | 00:55 | |
| cathy_ | armax: see your change. Got it. Thanks for all the help! | 00:56 |
| *** zz_dimtruck is now known as dimtruck | 00:56 | |
| *** achanda has quit IRC | 00:56 | |
| cathy_ | armax: Have you approved it? I have not seen that yet. | 00:57 |
| *** mattymo has joined #openstack-infra | 00:57 | |
| *** mattymo has quit IRC | 00:57 | |
| *** sabeen1 has joined #openstack-infra | 00:58 | |
| *** madhu_ak_ has quit IRC | 00:59 | |
| *** otter768 has quit IRC | 00:59 | |
| *** mattymo has joined #openstack-infra | 00:59 | |
| *** mattymo has quit IRC | 00:59 | |
| *** rm_work is now known as rm_work|away | 01:00 | |
| *** mattymo has joined #openstack-infra | 01:01 | |
| *** mattymo has quit IRC | 01:01 | |
| *** pvaneck has quit IRC | 01:02 | |
| cathy_ | armax: Hi, are you there? | 01:02 |
| mordred | reed: http://paste.openstack.org/show/382315 | 01:02 |
| mordred | reed: local test script of mine | 01:03 |
| *** mattymo has joined #openstack-infra | 01:03 | |
| *** mattymo has quit IRC | 01:03 | |
| reed | thanks mordred, exactly what I was looking for | 01:03 |
| *** mattymo has joined #openstack-infra | 01:05 | |
| *** mattymo has quit IRC | 01:05 | |
| reed | uuuuuu | 01:06 |
| reed | it works | 01:06 |
| *** cathy_ has quit IRC | 01:06 | |
| reed | mordred, I feel that by end of Monday I'll have a patch up for shade on http://developer.openstack.org/firstapp-libcloud/getting_started.html ... at worst a WIP :) | 01:06 |
| *** hdd has joined #openstack-infra | 01:06 | |
| *** SumitNaiksatam has joined #openstack-infra | 01:07 | |
| *** SumitNaiksatam has quit IRC | 01:07 | |
| *** achanda has joined #openstack-infra | 01:07 | |
| *** mattymo has joined #openstack-infra | 01:07 | |
| *** mattymo has quit IRC | 01:07 | |
| *** jasondotstar has quit IRC | 01:08 | |
| *** mattymo has joined #openstack-infra | 01:09 | |
| *** mattymo has quit IRC | 01:09 | |
| *** alexsyip has quit IRC | 01:10 | |
| Shrews | reed: I hear that mordred will pay you upwards of $1 plus all the beer you can drink if you add documentation for using the new Ansible modules to that. | 01:11 |
| *** mattymo has joined #openstack-infra | 01:12 | |
| *** mattymo has quit IRC | 01:12 | |
| anteaya | so if cathy_ shows up again she needs to join the -neutron channel and talk to neutron devs as some don't feel she is addressing their concerns | 01:12 |
| reed | Shrews, one step at the time :) | 01:12 |
| anteaya | if anyone is feeling up to redirecting her | 01:12 |
| reed | my dreamhost account now has an instance stuck on delete and I have filled my quota... time to call it a day. At least I quit knowing I have moved forward in my task. #winning | 01:13 |
| *** mattymo has joined #openstack-infra | 01:14 | |
| *** mattymo has quit IRC | 01:14 | |
| reed | cheers folks, tty tomorrow | 01:14 |
| anteaya | night reed | 01:14 |
| *** jamesmcarthur has quit IRC | 01:15 | |
| *** mattymo has joined #openstack-infra | 01:16 | |
| *** mattymo has quit IRC | 01:16 | |
| *** yamamoto has quit IRC | 01:16 | |
| *** fitoduarte has joined #openstack-infra | 01:16 | |
| jroll | anteaya: self-merge, ouch | 01:16 |
| anteaya | no kidding | 01:17 |
| *** mattymo has joined #openstack-infra | 01:18 | |
| *** mattymo has quit IRC | 01:18 | |
| *** mattymo has joined #openstack-infra | 01:20 | |
| *** mattymo has quit IRC | 01:20 | |
| *** esmute has joined #openstack-infra | 01:20 | |
| *** ivar-laz_ has joined #openstack-infra | 01:21 | |
| *** esmute has quit IRC | 01:21 | |
| *** mattymo has joined #openstack-infra | 01:22 | |
| *** mattymo has quit IRC | 01:22 | |
| *** mattymo has joined #openstack-infra | 01:24 | |
| *** mattymo has quit IRC | 01:24 | |
| *** ivar-lazzaro has quit IRC | 01:24 | |
| *** otter768 has joined #openstack-infra | 01:25 | |
| *** woodster_ has joined #openstack-infra | 01:25 | |
| *** ivar-laz_ has quit IRC | 01:25 | |
| *** mattymo has joined #openstack-infra | 01:26 | |
| *** mattymo has quit IRC | 01:26 | |
| *** mattymo has joined #openstack-infra | 01:28 | |
| *** mattymo has quit IRC | 01:28 | |
| *** mattymo has joined #openstack-infra | 01:30 | |
| *** mattymo has quit IRC | 01:30 | |
| *** mattymo has joined #openstack-infra | 01:32 | |
| *** mattymo has quit IRC | 01:32 | |
| *** mattymo has joined #openstack-infra | 01:34 | |
| *** mattymo has quit IRC | 01:34 | |
| *** afazekas has quit IRC | 01:34 | |
| openstackgerrit | Merged openstack-infra/devstack-gate: Unmount the ephemeral drive before formatting it https://review.openstack.org/202745 | 01:35 |
| *** adduarte has joined #openstack-infra | 01:36 | |
| *** mattymo has joined #openstack-infra | 01:36 | |
| *** mattymo has quit IRC | 01:36 | |
| *** patrickeast has quit IRC | 01:38 | |
| *** mattymo has joined #openstack-infra | 01:38 | |
| *** mattymo has quit IRC | 01:38 | |
| *** ddieterl_ has joined #openstack-infra | 01:39 | |
| *** fitoduarte has quit IRC | 01:39 | |
| *** mattymo has joined #openstack-infra | 01:40 | |
| *** mattymo has quit IRC | 01:40 | |
| *** puranamr has joined #openstack-infra | 01:41 | |
| *** puranamr has quit IRC | 01:42 | |
| *** mattymo has joined #openstack-infra | 01:42 | |
| *** mattymo has quit IRC | 01:42 | |
| *** ddieterly has quit IRC | 01:43 | |
| *** mattymo has joined #openstack-infra | 01:44 | |
| *** mattymo has quit IRC | 01:44 | |
| *** bdemers has quit IRC | 01:45 | |
| *** mattymo has joined #openstack-infra | 01:46 | |
| *** mattymo has quit IRC | 01:46 | |
| *** bdemers has joined #openstack-infra | 01:46 | |
| *** mattymo has joined #openstack-infra | 01:48 | |
| *** puranamr has joined #openstack-infra | 01:50 | |
| *** bdemers has quit IRC | 01:51 | |
| *** puranamr has quit IRC | 01:52 | |
| *** bdemers has joined #openstack-infra | 01:52 | |
| *** otter768 has quit IRC | 01:53 | |
| clarkb | I have reenabled config drive on devstack-trusty hpcloud nodes and its looking good so far | 01:57 |
| *** bdemers has quit IRC | 01:57 | |
| *** puranamr has joined #openstack-infra | 01:58 | |
| anteaya | yay? | 01:58 |
| *** puranamr has quit IRC | 01:59 | |
| clarkb | ya this should get multiode working in hpcloud | 01:59 |
| anteaya | oh yay! | 02:00 |
| anteaya | \o/ | 02:00 |
| *** rlandy has quit IRC | 02:01 | |
| *** thorst has joined #openstack-infra | 02:01 | |
| *** thorst has quit IRC | 02:06 | |
| *** ddieterl_ has quit IRC | 02:10 | |
| *** puranamr has joined #openstack-infra | 02:14 | |
| *** Sherif has quit IRC | 02:14 | |
| *** ddieterly has joined #openstack-infra | 02:14 | |
| *** macjack has left #openstack-infra | 02:15 | |
| *** bdemers has joined #openstack-infra | 02:18 | |
| *** yamamoto has joined #openstack-infra | 02:23 | |
| *** achanda has quit IRC | 02:23 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/project-config: Check indenting of jenkins/jobs/projects.yaml https://review.openstack.org/202851 | 02:23 |
| *** ayoung has joined #openstack-infra | 02:27 | |
| anteaya | ianw: thanks | 02:28 |
| *** dims_ has quit IRC | 02:31 | |
| *** _ody has joined #openstack-infra | 02:32 | |
| *** spzala has quit IRC | 02:33 | |
| *** tonytan4ever has quit IRC | 02:33 | |
| *** yamahata has joined #openstack-infra | 02:35 | |
| *** signed8bit_ZZZzz has quit IRC | 02:36 | |
| *** _ody has quit IRC | 02:37 | |
| *** chlong-afk has quit IRC | 02:37 | |
| *** jogo has quit IRC | 02:38 | |
| *** chlong has joined #openstack-infra | 02:42 | |
| *** ayoung has quit IRC | 02:45 | |
| *** larainema has quit IRC | 02:45 | |
| *** larainema has joined #openstack-infra | 02:46 | |
| *** puranamr has quit IRC | 02:47 | |
| *** puranamr has joined #openstack-infra | 03:07 | |
| *** vnod has quit IRC | 03:08 | |
| *** mayurig has joined #openstack-infra | 03:10 | |
| *** puranamr has quit IRC | 03:12 | |
| *** boris-42 has joined #openstack-infra | 03:14 | |
| *** mayurig has quit IRC | 03:15 | |
| openstackgerrit | Merged openstack-infra/project-config: Fix failing StoryBoard job https://review.openstack.org/202182 | 03:16 |
| *** julim has quit IRC | 03:18 | |
| *** Kennan has quit IRC | 03:20 | |
| *** Kennan has joined #openstack-infra | 03:20 | |
| *** achanda has joined #openstack-infra | 03:23 | |
| *** sdake has quit IRC | 03:23 | |
| *** ajmiller__ is now known as ajmiller | 03:26 | |
| *** achanda has quit IRC | 03:28 | |
| *** puranamr has joined #openstack-infra | 03:29 | |
| ianw | greghaynes: any chance of a +a on https://review.openstack.org/#/c/185841 ? | 03:31 |
| ianw | (f22 image building) | 03:31 |
| *** woodster_ has quit IRC | 03:32 | |
| *** dims_ has joined #openstack-infra | 03:32 | |
| *** puranamr has quit IRC | 03:33 | |
| *** larainema has quit IRC | 03:33 | |
| *** sdake has joined #openstack-infra | 03:33 | |
| *** larainema has joined #openstack-infra | 03:34 | |
| *** unicell has quit IRC | 03:34 | |
| *** dims_ has quit IRC | 03:37 | |
| *** annegentle has joined #openstack-infra | 03:38 | |
| *** ildikov has quit IRC | 03:40 | |
| *** tonytan4ever has joined #openstack-infra | 03:40 | |
| openstackgerrit | Timothy R. Chavez proposed openstack-infra/zuul: Report the per-job build wait time to graphite https://review.openstack.org/202867 | 03:44 |
| *** hdd has quit IRC | 03:45 | |
| *** pal has joined #openstack-infra | 03:46 | |
| *** mriedem1 has quit IRC | 03:46 | |
| *** puranamr has joined #openstack-infra | 03:48 | |
| *** tlian has quit IRC | 03:49 | |
| *** jogo has joined #openstack-infra | 03:50 | |
| *** puranamr has quit IRC | 03:51 | |
| *** Sherif has joined #openstack-infra | 03:53 | |
| *** puranamr has joined #openstack-infra | 03:53 | |
| *** jamesmcarthur has joined #openstack-infra | 03:53 | |
| *** Sherif has quit IRC | 03:54 | |
| *** otter768 has joined #openstack-infra | 03:54 | |
| *** coolsvap|away is now known as coolsvap | 03:54 | |
| *** achanda has joined #openstack-infra | 03:55 | |
| *** ddieterl_ has joined #openstack-infra | 03:57 | |
| *** otter768 has quit IRC | 03:58 | |
| *** Sherif has joined #openstack-infra | 03:59 | |
| *** achanda has quit IRC | 04:00 | |
| *** erlon has quit IRC | 04:00 | |
| *** geoffarnold has joined #openstack-infra | 04:00 | |
| *** puranamr has quit IRC | 04:00 | |
| *** ddieterly has quit IRC | 04:01 | |
| *** puranamr has joined #openstack-infra | 04:01 | |
| *** jerryz has quit IRC | 04:01 | |
| *** bdemers has quit IRC | 04:04 | |
| *** bdemers has joined #openstack-infra | 04:04 | |
| *** pal has quit IRC | 04:04 | |
| *** rlandy has joined #openstack-infra | 04:06 | |
| *** jamesmcarthur has quit IRC | 04:07 | |
| *** baoli_ has joined #openstack-infra | 04:08 | |
| *** baoli_ has quit IRC | 04:08 | |
| *** puranamr has quit IRC | 04:09 | |
| *** geoffarnold has quit IRC | 04:09 | |
| *** annegentle has quit IRC | 04:11 | |
| *** baoli has quit IRC | 04:11 | |
| *** rlandy has quit IRC | 04:11 | |
| *** yamamoto has quit IRC | 04:13 | |
| *** unicell has joined #openstack-infra | 04:22 | |
| *** bdemers has quit IRC | 04:22 | |
| *** bdemers has joined #openstack-infra | 04:23 | |
| *** yamamoto has joined #openstack-infra | 04:23 | |
| openstackgerrit | Pete Zaitcev proposed openstack-infra/git-review: Update test requirements for Python 2.x https://review.openstack.org/200860 | 04:24 |
| *** unicell has quit IRC | 04:26 | |
| *** unicell has joined #openstack-infra | 04:27 | |
| *** thedodd has joined #openstack-infra | 04:33 | |
| *** _ody has joined #openstack-infra | 04:33 | |
| *** harlowja_at_home has joined #openstack-infra | 04:35 | |
| *** sputnik13 has joined #openstack-infra | 04:36 | |
| *** _ody has quit IRC | 04:38 | |
| *** FallenPegasus has joined #openstack-infra | 04:47 | |
| *** esmute has joined #openstack-infra | 04:50 | |
| *** Sherif has quit IRC | 04:54 | |
| *** sdake has quit IRC | 04:56 | |
| *** tonytan4ever has quit IRC | 04:56 | |
| *** Sherif has joined #openstack-infra | 05:01 | |
| openstackgerrit | Merged openstack-infra/project-config: Check indenting of jenkins/jobs/projects.yaml https://review.openstack.org/202851 | 05:04 |
| *** ParsectiX has joined #openstack-infra | 05:05 | |
| *** dannywilson has joined #openstack-infra | 05:06 | |
| *** ddieterl_ has quit IRC | 05:06 | |
| *** pal has joined #openstack-infra | 05:08 | |
| StevenK | camunoz: I have a Zanata 3.7.1 server up. I didn't have to make any config changes, are you around to have a poke at it? | 05:09 |
| camunoz | hi StevenK , sure... do you have it @ translate-dev? | 05:11 |
| StevenK | camunoz: I do not, I've set up a test machine | 05:13 |
| *** mrmartin has joined #openstack-infra | 05:13 | |
| camunoz | ah ok... well if you didn't make any config changes, I'm pretty sure I know where to poke to make it break :) | 05:14 |
| StevenK | Excellent | 05:14 |
| camunoz | what's the url? | 05:14 |
| *** yfried has joined #openstack-infra | 05:15 | |
| StevenK | camunoz: I'm just sorting out an ssh key SNAFU | 05:15 |
| camunoz | ok | 05:16 |
| StevenK | camunoz: But https://15.125.83.46/ but it seems to not be Zanata yet | 05:16 |
| *** thedodd has quit IRC | 05:17 | |
| camunoz | StevenK, looks like Zanata to me :) | 05:17 |
| StevenK | I have a Wildfly version 8 page | 05:17 |
| camunoz | ??? I do get Zanata.... do data though | 05:17 |
| StevenK | camunoz: I think I was too quick | 05:18 |
| *** ParsectiX has quit IRC | 05:18 | |
| camunoz | login isn't working either | 05:18 |
| StevenK | Login wasn't working for me last night, it should be talking to openstackid, but I'm lost | 05:19 |
| camunoz | any chance I can ssh to it? | 05:19 |
| StevenK | camunoz: Sure | 05:19 |
| *** dguerri` is now known as dguerri | 05:19 | |
| StevenK | camunoz: Privmsg me a key | 05:19 |
| camunoz | k | 05:19 |
| *** boris-42_ has joined #openstack-infra | 05:20 | |
| *** kaisers2 has joined #openstack-infra | 05:21 | |
| *** ameade_ has joined #openstack-infra | 05:21 | |
| *** tchaypo2 has joined #openstack-infra | 05:21 | |
| *** dkehn_ has joined #openstack-infra | 05:22 | |
| *** Volundr has joined #openstack-infra | 05:22 | |
| *** gildub_ has joined #openstack-infra | 05:22 | |
| *** Daviey_ has joined #openstack-infra | 05:22 | |
| *** bhunter71_2 has joined #openstack-infra | 05:22 | |
| *** samuel-dmq has joined #openstack-infra | 05:23 | |
| *** boris-42 has quit IRC | 05:23 | |
| *** tchaypo1 has quit IRC | 05:23 | |
| *** jbernard has quit IRC | 05:23 | |
| *** ameade has quit IRC | 05:23 | |
| *** dkehn has quit IRC | 05:23 | |
| *** ondergetekende has quit IRC | 05:23 | |
| *** gildub has quit IRC | 05:23 | |
| *** mmedvede has quit IRC | 05:23 | |
| *** whoops has quit IRC | 05:23 | |
| *** mancdaz has quit IRC | 05:23 | |
| *** bhunter71 has quit IRC | 05:23 | |
| *** Daviey has quit IRC | 05:23 | |
| *** samueldmq has quit IRC | 05:23 | |
| *** mmedvede has joined #openstack-infra | 05:24 | |
| *** ameade_ is now known as ameade | 05:24 | |
| *** ondergetekende has joined #openstack-infra | 05:24 | |
| *** boris-42_ is now known as boris-42 | 05:24 | |
| *** heyongli has quit IRC | 05:24 | |
| *** jbernard has joined #openstack-infra | 05:24 | |
| *** mancdaz has joined #openstack-infra | 05:24 | |
| camunoz | StevenK, I'm in | 05:25 |
| StevenK | camunoz: NANO!? | 05:27 |
| StevenK | We are no longer friends. :-P | 05:27 |
| camunoz | don't judge ;) | 05:27 |
| StevenK | Hahahaha | 05:28 |
| *** heyongli has joined #openstack-infra | 05:28 | |
| *** esmute has quit IRC | 05:29 | |
| *** _nadya_ has joined #openstack-infra | 05:31 | |
| *** Sherif has quit IRC | 05:33 | |
| *** shashank_hegde_ has joined #openstack-infra | 05:33 | |
| *** _nadya_ has quit IRC | 05:34 | |
| *** _nadya_ has joined #openstack-infra | 05:34 | |
| *** dannywilson has quit IRC | 05:35 | |
| *** dannywilson has joined #openstack-infra | 05:36 | |
| *** s0nea has quit IRC | 05:37 | |
| *** heyongli has quit IRC | 05:37 | |
| *** heyongli has joined #openstack-infra | 05:38 | |
| *** Sherif has joined #openstack-infra | 05:38 | |
| *** _nadya_ has quit IRC | 05:38 | |
| camunoz | StevenK, looks like openstackid.org is returning some kind of error to Zanata.... is there anything that needs to be configured on that side? | 05:38 |
| *** BharatK has joined #openstack-infra | 05:38 | |
| camunoz | https://15.125.83.46/openid.seam?openid.error=Server+Error&openid.ns=http%3A%2F%2Fspecs.openid.net%2Fauth%2F2.0&openid.return_to=https%3A%2F%2F15.125.83.46%3A443%2Fopenid.seam | 05:39 |
| camunoz | that is the request that Zanata gets from the openid server | 05:39 |
| StevenK | camunoz: TBH, I'm not sure about that side at all :-( | 05:39 |
| *** s0nea has joined #openstack-infra | 05:40 | |
| *** FallenPegasus has quit IRC | 05:41 | |
| *** markvoelker has joined #openstack-infra | 05:41 | |
| *** FallenPegasus has joined #openstack-infra | 05:41 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/puppet-jenkins: Use versioncmp on operatingsystemrelease https://review.openstack.org/202884 | 05:42 |
| *** mrmartin has quit IRC | 05:43 | |
| *** larainema has quit IRC | 05:43 | |
| *** Sukhdev has joined #openstack-infra | 05:43 | |
| StevenK | camunoz: So we can't break it if we can't log in? | 05:43 |
| *** markvoelker_ has joined #openstack-infra | 05:44 | |
| StevenK | camunoz: ... sounds like a win? :-P | 05:44 |
| camunoz | yes, well... at least it's secure :) ... just trying to see if there are any differences in the requests going around between the zanata server and the open id server | 05:45 |
| *** markvoelker has quit IRC | 05:45 | |
| *** esmute has joined #openstack-infra | 05:47 | |
| *** bdemers has quit IRC | 05:47 | |
| *** shashank_hegde_ has quit IRC | 05:47 | |
| camunoz | other than the url, the requests are identical | 05:47 |
| *** bdemers has joined #openstack-infra | 05:48 | |
| camunoz | StevenK, the only thing I can think of is that the url needs to be registered with the openid server, but it's just a guess... we can try changing the open id provider if you'd like | 05:49 |
| *** bhunter71_2 has quit IRC | 05:49 | |
| StevenK | I suspect it may be wanting the request URI to be from *.openstack.org -- but given openstackid required changes for translate-dev, I'm not sure | 05:50 |
| StevenK | camunoz: Sure, or we just turn it off since I was expecting this servers lifetime to be meaursed in hours or days | 05:51 |
| camunoz | StevenK, sure... in any case, the changes needed for it to work are all on the release notes and boil down to standalone.xml changes | 05:52 |
| camunoz | let me know when you want to visit them for the translate-dev instance | 05:52 |
| StevenK | camunoz: I drowned in reading about infinicache | 05:52 |
| StevenK | I think? Infini something | 05:52 |
| camunoz | infinispan :) | 05:52 |
| *** bdemers has quit IRC | 05:53 | |
| StevenK | The release notes were good at pointing out things needed to be changed, but I didn't see "Your config needs *blat*" | 05:53 |
| *** bdemers has joined #openstack-infra | 05:54 | |
| StevenK | inifispan is already in /opt/wildfly/standalone/configuration/standalone.xml , by the way? | 05:54 |
| StevenK | *inifinispan | 05:55 |
| *** otter768 has joined #openstack-infra | 05:55 | |
| camunoz | yes, we just need to add the config snippet to it: http://docs.zanata.org/en/release/user-guide/system-admin/configuration/infinispan/ | 05:55 |
| *** mrunge has joined #openstack-infra | 05:56 | |
| StevenK | <subsystem xmlns="urn:jboss:domain:infinispan:2.0"> | 05:56 |
| *** deepakcs has joined #openstack-infra | 05:56 | |
| StevenK | From the existing standalone.xml on the machine | 05:56 |
| StevenK | http://git.openstack.org/cgit/openstack-infra/puppet-zanata/tree/templates/standalone.xml.erb also has infinispan stuff | 05:57 |
| *** bdemers has quit IRC | 05:58 | |
| *** bdemers has joined #openstack-infra | 05:58 | |
| *** otter768 has quit IRC | 05:59 | |
| *** dims_ has joined #openstack-infra | 06:00 | |
| openstackgerrit | Christophe Vanlancker proposed openstack-infra/jenkins-job-builder: Adding support for Manual Build Trigger https://review.openstack.org/202543 | 06:04 |
| *** esmute has quit IRC | 06:05 | |
| *** dims_ has quit IRC | 06:06 | |
| *** pal has quit IRC | 06:07 | |
| *** ddieterly has joined #openstack-infra | 06:07 | |
| *** ParsectiX has joined #openstack-infra | 06:07 | |
| *** ig0r_ has joined #openstack-infra | 06:08 | |
| *** harlowja_at_home has quit IRC | 06:09 | |
| ParsectiX | Good Morning :) | 06:09 |
| StevenK | camunoz: Does the configuration template look sane? Because we seem to have that section already | 06:10 |
| *** ddieterly has quit IRC | 06:11 | |
| camunoz | you are right... that part is already there | 06:12 |
| camunoz | let me check the other part | 06:12 |
| *** Hal has joined #openstack-infra | 06:13 | |
| *** Hal is now known as Guest2275 | 06:14 | |
| openstackgerrit | Boris Pavlovic proposed openstack-infra/project-config: Add python jobs for fuel-web https://review.openstack.org/202892 | 06:14 |
| camunoz | StevenK, so in the <subsystem xmlns="urn:jboss:domain:mail:2.0"> section... | 06:14 |
| camunoz | that mail-session needs to be configured with an actual email server | 06:14 |
| *** maishsk has quit IRC | 06:15 | |
| camunoz | StevenK, so... something like : <mail-session jndi-name="java:/OtherMailSession" from="javaarm@gmail.com"> | 06:16 |
| camunoz | <smtp-server ssl="true" outbound-socket-binding-ref="mail-smtp-gmail"> | 06:16 |
| camunoz | <login name="javaarm@gmail.com" password="your_password"/> | 06:16 |
| camunoz | </smtp-server> | 06:16 |
| camunoz | </mail-session> | 06:16 |
| camunoz | ugh... sorry about that | 06:16 |
| StevenK | Haha | 06:16 |
| StevenK | camunoz: Pastebin, please? | 06:16 |
| StevenK | camunoz: And include the surrounding lines for context | 06:16 |
| camunoz | is there an openstack pastebin? | 06:17 |
| StevenK | camunoz: Of course! | 06:17 |
| StevenK | camunoz: paste.openstack.org | 06:17 |
| camunoz | thx! | 06:17 |
| *** _nadya_ has joined #openstack-infra | 06:18 | |
| camunoz | StevenK, something like this: http://paste.openstack.org/show/382907/ | 06:19 |
| *** _nadya_ has quit IRC | 06:19 | |
| *** _nadya_ has joined #openstack-infra | 06:20 | |
| StevenK | That seems like it requires SMTP AUTH to send too, no? | 06:20 |
| *** bdemers has quit IRC | 06:21 | |
| *** achanda has joined #openstack-infra | 06:21 | |
| *** _nadya_ has quit IRC | 06:21 | |
| StevenK | I'm not sure about our mail server details, or if we'd want to set one up on translate-dev itself | 06:21 |
| StevenK | Above my paygrade, a little | 06:21 |
| *** _nadya_ has joined #openstack-infra | 06:21 | |
| StevenK | camunoz: But hey, if it's just that and change the URL to be 3.7.1, bonus | 06:21 |
| *** Sherif has quit IRC | 06:22 | |
| *** Guest2275 has quit IRC | 06:22 | |
| *** bdemers has joined #openstack-infra | 06:23 | |
| *** dannywil_ has joined #openstack-infra | 06:24 | |
| *** scheuran has joined #openstack-infra | 06:24 | |
| *** yamamoto has quit IRC | 06:25 | |
| camunoz | I think it's already set up for translate-dev... otherwise people wouldn't be able to sign up... it's just done differently, it's done with jndi variables | 06:25 |
| *** dannywil_ has quit IRC | 06:25 | |
| *** dannywil_ has joined #openstack-infra | 06:26 | |
| StevenK | camunoz: And we do get e-mail from the server ... | 06:26 |
| StevenK | So it's already configured? | 06:26 |
| camunoz | trying to find it in the template you gave me... | 06:27 |
| camunoz | but not there.... by default is uses localhost so that might be why | 06:27 |
| *** dannywilson has quit IRC | 06:27 | |
| camunoz | in that case... there's no need to change anything for your instance | 06:28 |
| *** bdemers has quit IRC | 06:29 | |
| *** bdemers has joined #openstack-infra | 06:29 | |
| camunoz | StevenK, yes, I think the default will work for translate-dev | 06:29 |
| openstackgerrit | daisy-ycguo proposed openstack-infra/system-config: Add eavesdrop to #openstack-translation channel https://review.openstack.org/201663 | 06:30 |
| StevenK | camunoz: \o/ | 06:30 |
| *** _ody has joined #openstack-infra | 06:34 | |
| *** Sherif has joined #openstack-infra | 06:36 | |
| camunoz | StevenK , I'm out for the week.... shoot me an email if you hit any roadblocks. have a good weekend | 06:36 |
| StevenK | camunoz: You too, thanks! | 06:36 |
| *** camunoz has quit IRC | 06:36 | |
| *** Sukhdev has quit IRC | 06:37 | |
| openstackgerrit | Ken'ichi Ohmichi proposed openstack-infra/project-config: Don't run dsvm tests on unittest only changes https://review.openstack.org/202898 | 06:37 |
| StevenK | yes | 06:37 |
| StevenK | Blah, that was meant for git review | 06:38 |
| openstackgerrit | Steve Kowalik proposed openstack-infra/puppet-zanata: Move to Zanata 3.7.1 https://review.openstack.org/202899 | 06:38 |
| *** _ody has quit IRC | 06:39 | |
| *** achanda has quit IRC | 06:39 | |
| *** pal has joined #openstack-infra | 06:39 | |
| *** dguerri is now known as dguerri` | 06:40 | |
| *** Sherif has quit IRC | 06:41 | |
| *** pal has quit IRC | 06:41 | |
| *** pal has joined #openstack-infra | 06:43 | |
| *** achanda has joined #openstack-infra | 06:45 | |
| *** ig0r_ has quit IRC | 06:45 | |
| *** jyuso1 has quit IRC | 06:48 | |
| *** ig0r_ has joined #openstack-infra | 06:50 | |
| *** Sherif has joined #openstack-infra | 06:55 | |
| *** yamamoto has joined #openstack-infra | 06:55 | |
| *** Sherif has quit IRC | 06:58 | |
| *** dimtruck is now known as zz_dimtruck | 06:58 | |
| openstackgerrit | Ken'ichi Ohmichi proposed openstack-infra/project-config: Don't run dsvm tests on unittest only changes https://review.openstack.org/202898 | 07:00 |
| *** s0nea has quit IRC | 07:00 | |
| *** s0nea has joined #openstack-infra | 07:02 | |
| AJaeger | Hi lifeless, I have a question about update.py in requirements repo. Are you around? | 07:03 |
| *** sgordon has quit IRC | 07:03 | |
| lifeless | AJaeger: ish | 07:06 |
| *** afazekas has joined #openstack-infra | 07:06 | |
| AJaeger | lifeless: I'm used to run manually "python update.py ../security-specs" but this fails now. | 07:07 |
| AJaeger | http://paste.openstack.org/show/383011/ | 07:07 |
| AJaeger | Does it work for you? | 07:07 |
| AJaeger | no need to dig into this right now - could you take a note and check later, please? | 07:08 |
| *** bradjones has quit IRC | 07:08 | |
| *** mrmartin has joined #openstack-infra | 07:09 | |
| lifeless | AJaeger: you needa newer setuptools | 07:09 |
| AJaeger | ah... | 07:09 |
| AJaeger | indeed, that helped. Thanks! | 07:10 |
| lifeless | AJaeger: possibly we need to start expressing a versioned dep there but it gets complex | 07:10 |
| lifeless | so for now, shrug | 07:10 |
| *** markus_z has joined #openstack-infra | 07:10 | |
| *** bradjones has joined #openstack-infra | 07:11 | |
| *** bradjones has quit IRC | 07:11 | |
| *** bradjones has joined #openstack-infra | 07:11 | |
| *** boris-42 has quit IRC | 07:12 | |
| *** doude has joined #openstack-infra | 07:12 | |
| *** mrmartin has quit IRC | 07:14 | |
| Kennan | hi lifeless: there? | 07:15 |
| *** devvesa has joined #openstack-infra | 07:15 | |
| lifeless | yes | 07:15 |
| lifeless | ish | 07:15 |
| *** dtantsur|afk is now known as dtantsur | 07:15 | |
| Kennan | we found our jenkins all failed with unexpected new errors | 07:16 |
| Kennan | seems some new change happened in infra package or updates | 07:16 |
| Kennan | http://logs.openstack.org/07/199007/5/check/gate-magnum-python27/3b0a4c7/console.html#_2015-07-17_03_20_43_329 | 07:16 |
| Kennan | could you tell me what happned for that ? | 07:16 |
| Kennan | lifeless: TypeError: <MagicMock name='container_logs()' id='140120748333584'> is not JSON serializable | 07:17 |
| lifeless | Kennan: there is a bug open for that | 07:17 |
| Kennan | not figure out what's the issue | 07:17 |
| lifeless | I believe its an upstream (cPython stdlib) bug too | 07:17 |
| Kennan | could you give me a link for that ? | 07:17 |
| Kennan | what's the bug link? | 07:17 |
| lifeless | https://github.com/testing-cabal/mock/issues/139 | 07:18 |
| Kennan | hi lifeless: it seemed it was opened on Apr , but we hit that today or maybe yesterday | 07:19 |
| Kennan | why not hit that before? | 07:19 |
| lifeless | I don't know. maybe something in your tests has altered | 07:20 |
| *** Ala has joined #openstack-infra | 07:20 | |
| lifeless | but, thats pickle not json, but AFAIK they share __reduce__ | 07:21 |
| Kennan | lifeless: the last patch yesteday https://review.openstack.org/#/c/194514/ it worked | 07:23 |
| *** kutija_ has quit IRC | 07:25 | |
| *** sgordon has joined #openstack-infra | 07:25 | |
| *** kutija has joined #openstack-infra | 07:26 | |
| *** achanda has quit IRC | 07:27 | |
| lifeless | Kennan: well, dunno - sorry | 07:29 |
| *** shardy has joined #openstack-infra | 07:33 | |
| *** fhubik has joined #openstack-infra | 07:34 | |
| *** weshay has quit IRC | 07:35 | |
| *** jlanoux has joined #openstack-infra | 07:37 | |
| *** weshay has joined #openstack-infra | 07:37 | |
| *** salv-orlando has joined #openstack-infra | 07:39 | |
| *** Hal has joined #openstack-infra | 07:40 | |
| *** Hal is now known as Guest38539 | 07:41 | |
| *** achanda has joined #openstack-infra | 07:42 | |
| *** weshay has quit IRC | 07:42 | |
| *** sabeen1 has quit IRC | 07:43 | |
| *** jlanoux has quit IRC | 07:43 | |
| *** jlanoux has joined #openstack-infra | 07:44 | |
| *** Guest38539 has quit IRC | 07:45 | |
| *** ihrachyshka has joined #openstack-infra | 07:45 | |
| *** fhubik has quit IRC | 07:46 | |
| *** oomichi has quit IRC | 07:47 | |
| *** fhubik has joined #openstack-infra | 07:48 | |
| *** dannywil_ has quit IRC | 07:49 | |
| *** dims_ has joined #openstack-infra | 07:49 | |
| *** bexelbie has joined #openstack-infra | 07:50 | |
| *** gildub_ has quit IRC | 07:51 | |
| *** coolsvap is now known as coolsvap|afk | 07:51 | |
| *** pal has quit IRC | 07:52 | |
| *** heyongli has quit IRC | 07:52 | |
| *** crc32 has quit IRC | 07:52 | |
| *** larainema has joined #openstack-infra | 07:53 | |
| *** rossella_s has joined #openstack-infra | 07:54 | |
| *** sputnik13 has quit IRC | 07:55 | |
| *** dguitarbite has joined #openstack-infra | 07:55 | |
| *** dguitarbite is now known as Guest13513 | 07:55 | |
| *** otter768 has joined #openstack-infra | 07:56 | |
| *** dims_ has quit IRC | 07:56 | |
| *** heyongli has joined #openstack-infra | 07:58 | |
| *** kaisers2 has left #openstack-infra | 07:58 | |
| *** otter768 has quit IRC | 08:00 | |
| *** shardy_ has joined #openstack-infra | 08:00 | |
| *** shardy has quit IRC | 08:02 | |
| *** HeOS has quit IRC | 08:02 | |
| *** rm_work|away is now known as rm_work | 08:04 | |
| *** shardy_ has quit IRC | 08:06 | |
| *** shardy has joined #openstack-infra | 08:06 | |
| *** fhubik is now known as fhubik_afk | 08:07 | |
| *** weshay has joined #openstack-infra | 08:07 | |
| *** dizquierdo has joined #openstack-infra | 08:07 | |
| *** hashar has joined #openstack-infra | 08:13 | |
| *** jistr has joined #openstack-infra | 08:15 | |
| *** achanda has quit IRC | 08:16 | |
| *** salv-orlando has quit IRC | 08:17 | |
| *** kushal has quit IRC | 08:18 | |
| *** laughterwym has quit IRC | 08:22 | |
| BobBall | clarkb: thanks! | 08:22 |
| *** gsagie has joined #openstack-infra | 08:22 | |
| *** dtantsur is now known as dtantsur|brb | 08:27 | |
| *** fhubik_afk is now known as fhubik | 08:28 | |
| *** larainema has quit IRC | 08:28 | |
| *** larainema has joined #openstack-infra | 08:29 | |
| *** megm_ has quit IRC | 08:29 | |
| *** megm has joined #openstack-infra | 08:30 | |
| *** pal has joined #openstack-infra | 08:31 | |
| *** armax has quit IRC | 08:33 | |
| *** rcarrillocruz has quit IRC | 08:35 | |
| *** _ody has joined #openstack-infra | 08:35 | |
| *** dizquierdo has quit IRC | 08:35 | |
| *** yamamoto has quit IRC | 08:36 | |
| *** derekh has joined #openstack-infra | 08:37 | |
| *** oomichi has joined #openstack-infra | 08:37 | |
| AJaeger | sdague, ttx: Could you review a requirements change, please? https://review.openstack.org/#/c/201933/ unblocks openstack/kuryr and similiar new repos created | 08:37 |
| *** ddieterly has joined #openstack-infra | 08:38 | |
| *** _ody has quit IRC | 08:40 | |
| *** markvoelker_ has quit IRC | 08:42 | |
| *** ddieterly has quit IRC | 08:43 | |
| *** ociuhandu has joined #openstack-infra | 08:43 | |
| *** maishsk has joined #openstack-infra | 08:44 | |
| *** notnownikki has joined #openstack-infra | 08:47 | |
| *** mhu has quit IRC | 08:53 | |
| *** coolsvap|afk is now known as coolsvap | 08:54 | |
| *** markus_z has quit IRC | 08:56 | |
| *** mhu has joined #openstack-infra | 08:56 | |
| *** ociuhandu has quit IRC | 08:56 | |
| *** pal has quit IRC | 08:57 | |
| *** markvoelker has joined #openstack-infra | 08:57 | |
| *** matbu has left #openstack-infra | 08:58 | |
| ttx | AJaeger: looking | 08:59 |
| *** HeOS has joined #openstack-infra | 08:59 | |
| *** fhubik is now known as fhubik_afk | 09:00 | |
| *** ParsectiX has quit IRC | 09:00 | |
| *** e0ne has joined #openstack-infra | 09:01 | |
| *** markvoelker has quit IRC | 09:02 | |
| *** kushal has joined #openstack-infra | 09:02 | |
| AJaeger | ttx, thanks | 09:04 |
| *** fhubik_afk is now known as fhubik | 09:04 | |
| *** deepakcs has quit IRC | 09:04 | |
| *** ParsectiX has joined #openstack-infra | 09:05 | |
| *** maishsk has quit IRC | 09:06 | |
| *** kushal has quit IRC | 09:06 | |
| *** Hal has joined #openstack-infra | 09:06 | |
| *** Hal is now known as Guest77658 | 09:06 | |
| *** bauzas is now known as bauwser | 09:08 | |
| *** markvoelker has joined #openstack-infra | 09:12 | |
| *** markus_z has joined #openstack-infra | 09:13 | |
| *** Daviey_ is now known as Daviey | 09:14 | |
| *** dims_ has joined #openstack-infra | 09:15 | |
| *** teran_ has quit IRC | 09:15 | |
| *** heyongli has quit IRC | 09:16 | |
| *** markvoelker has quit IRC | 09:17 | |
| *** heyongli has joined #openstack-infra | 09:17 | |
| *** dims_ has quit IRC | 09:20 | |
| *** Sherif has joined #openstack-infra | 09:21 | |
| mescanef | SergeyLukjanov: hey, please review https://review.openstack.org/#/c/202233/ | 09:22 |
| mescanef | :-) | 09:22 |
| *** yamahata has quit IRC | 09:23 | |
| *** kushal has joined #openstack-infra | 09:24 | |
| *** Sherif has quit IRC | 09:26 | |
| *** markvoelker has joined #openstack-infra | 09:26 | |
| *** markvoelker has quit IRC | 09:31 | |
| *** ZZelle has quit IRC | 09:31 | |
| *** Kennan2 has joined #openstack-infra | 09:33 | |
| *** Kennan has quit IRC | 09:33 | |
| *** ddieterly has joined #openstack-infra | 09:39 | |
| *** marzif_ has joined #openstack-infra | 09:39 | |
| *** pal has joined #openstack-infra | 09:39 | |
| *** cbader has quit IRC | 09:40 | |
| *** asselin_ has joined #openstack-infra | 09:40 | |
| *** asselin has quit IRC | 09:40 | |
| *** cbader has joined #openstack-infra | 09:40 | |
| *** akraminsky_ has joined #openstack-infra | 09:40 | |
| *** markvoelker has joined #openstack-infra | 09:41 | |
| *** akraminsky has quit IRC | 09:43 | |
| *** pal has quit IRC | 09:44 | |
| *** ddieterly has quit IRC | 09:44 | |
| *** markvoelker has quit IRC | 09:45 | |
| *** dtantsur|brb is now known as dtantsur | 09:46 | |
| *** jyuso1 has joined #openstack-infra | 09:46 | |
| *** yfried has quit IRC | 09:51 | |
| *** fhubik is now known as fhubik_afk | 09:52 | |
| *** markvoelker has joined #openstack-infra | 09:55 | |
| *** otter768 has joined #openstack-infra | 09:56 | |
| *** markvoelker has quit IRC | 10:00 | |
| *** otter768 has quit IRC | 10:01 | |
| *** dims_ has joined #openstack-infra | 10:05 | |
| *** ParsectiX has quit IRC | 10:05 | |
| *** ParsectiX has joined #openstack-infra | 10:06 | |
| *** deepakcs has joined #openstack-infra | 10:07 | |
| *** markvoelker has joined #openstack-infra | 10:07 | |
| *** mrmartin has joined #openstack-infra | 10:08 | |
| *** teran has joined #openstack-infra | 10:11 | |
| *** markvoelker has quit IRC | 10:12 | |
| *** btully has joined #openstack-infra | 10:14 | |
| *** ParsectiX has quit IRC | 10:15 | |
| *** teran has quit IRC | 10:16 | |
| *** teran has joined #openstack-infra | 10:17 | |
| *** amotoki has joined #openstack-infra | 10:18 | |
| *** rm_work is now known as rm_work|away | 10:19 | |
| *** btully has quit IRC | 10:19 | |
| openstackgerrit | Merged openstack-infra/project-config: Added new SF repository for Fuel LDAP plugin integration https://review.openstack.org/202233 | 10:19 |
| *** jasondotstar has joined #openstack-infra | 10:21 | |
| *** markvoelker has joined #openstack-infra | 10:22 | |
| *** Sherif has joined #openstack-infra | 10:22 | |
| gmann_ | clarkb: for list migration issue - https://review.openstack.org/#/c/202837/ | 10:23 |
| *** Sherif has quit IRC | 10:26 | |
| *** amotoki_ has joined #openstack-infra | 10:28 | |
| *** haypo has joined #openstack-infra | 10:28 | |
| *** Somay has joined #openstack-infra | 10:29 | |
| haypo | hi. i'm coming for https://review.openstack.org/#/c/201042/ "Make gate-heat-python34 check voting and add gate". tox -e py34 pass and Heat developers are ok to make the gate voting. is there any candidate to put a second +2? ;) | 10:29 |
| haypo | i'm asking because usually the python 3 support is quickly broken if the gate doesn't become voting | 10:30 |
| haypo | (it happened 10 days ago in Heat) | 10:30 |
| *** pal has joined #openstack-infra | 10:30 | |
| *** pc_m has joined #openstack-infra | 10:30 | |
| *** amotoki has quit IRC | 10:31 | |
| *** openstackgerrit has quit IRC | 10:31 | |
| *** openstackgerrit has joined #openstack-infra | 10:32 | |
| *** markvoelker has quit IRC | 10:32 | |
| Somay | hashar: If you’re free, can you review the following patches - https://review.openstack.org/#/c/189590/ and https://review.openstack.org/#/c/200115/ | 10:33 |
| *** fhubik_afk is now known as fhubik | 10:33 | |
| *** ParsectiX has joined #openstack-infra | 10:35 | |
| *** gsagie has quit IRC | 10:35 | |
| *** dizquierdo has joined #openstack-infra | 10:35 | |
| *** markvoelker has joined #openstack-infra | 10:37 | |
| *** sdake has joined #openstack-infra | 10:39 | |
| *** ddieterly has joined #openstack-infra | 10:40 | |
| *** teran has quit IRC | 10:40 | |
| *** thorst has joined #openstack-infra | 10:40 | |
| *** markvoelker has quit IRC | 10:42 | |
| *** teran has joined #openstack-infra | 10:43 | |
| Somay | zaro: If you’re free, then could you please review https://review.openstack.org/#/c/191534/ https://review.openstack.org/#/c/189590/ and https://review.openstack.org/#/c/200491/ | 10:44 |
| *** rguillebert has joined #openstack-infra | 10:44 | |
| *** ddieterly has quit IRC | 10:44 | |
| *** thorst has quit IRC | 10:45 | |
| *** asahlin has quit IRC | 10:50 | |
| sdague | haypo: done | 10:51 |
| *** markvoelker has joined #openstack-infra | 10:51 | |
| *** asahlin has joined #openstack-infra | 10:52 | |
| *** ujuc has joined #openstack-infra | 10:53 | |
| haypo | sdague: great :) 5 openstack project without voting py34 gate => only 4 left! (swift, horizon, python-savannaaclient, keystonemiddleware) | 10:56 |
| *** markvoelker has quit IRC | 10:56 | |
| sdague | nice | 10:56 |
| haypo | i have patches for swift & horizon. keystonemiddleware is blocked by python-memcached: i sent a pull request but the maintainer doesn'tr eply :-( | 10:57 |
| openstackgerrit | Merged openstack-infra/project-config: Make gate-heat-python34 check voting and add gate https://review.openstack.org/201042 | 10:57 |
| oomichi | sdague: hi, can you take a look at https://review.openstack.org/#/c/202898/ ? | 10:58 |
| AJaeger | haypo: Great, thanks! | 10:58 |
| oomichi | sdague: we are creating a lot of unit tests for tempest and dsvm tests run every time | 10:59 |
| AJaeger | oomichi: mtreinish should +1 that one | 11:00 |
| oomichi | AJaeger: Ok, I see. thanks for your advice :) | 11:00 |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Decouple server class from nodepool service class https://review.openstack.org/172859 | 11:00 |
| sdague | oomichi: seems good | 11:01 |
| oomichi | sdague: thanks :) | 11:01 |
| haypo | AJaeger: thanks for your cooperation on this topic ;) | 11:02 |
| sdague | though, I was actually surprised that etc changes were already in that list | 11:02 |
| sdague | because those can impact dsvm jobs | 11:02 |
| openstackgerrit | Merged openstack/requirements: Add checks for setup.cfg existence in project dictionary https://review.openstack.org/201933 | 11:03 |
| oomichi | sdague: ah, nice point | 11:03 |
| *** vlaza has joined #openstack-infra | 11:03 | |
| *** s0nea has quit IRC | 11:04 | |
| *** pal has quit IRC | 11:04 | |
| *** zeih has joined #openstack-infra | 11:05 | |
| *** dims_ has quit IRC | 11:06 | |
| haypo | sdague, AJaeger: oh by the way, tim burke of swift asked me to switch replace dnspython with dnspython3 in swift. here is my patch to add dnspython3 to global requirements: https://review.openstack.org/#/c/200416/ | 11:06 |
| *** markvoelker has joined #openstack-infra | 11:06 | |
| openstackgerrit | yolanda.robla proposed openstack-infra/system-config: Decouple server class from nodepool service class https://review.openstack.org/172859 | 11:06 |
| openstackgerrit | Merged openstack-infra/system-config: Add a common data directory for job workers https://review.openstack.org/200698 | 11:06 |
| haypo | sdague, AJaeger: it doesn't look like a new project, dnspython3 has the same author, source and license than dnspython, it more looks like dnspython "compiled" to python 3 (using 2to3 or something like that?) | 11:07 |
| *** s0nea has joined #openstack-infra | 11:07 | |
| haypo | later we might help to have single dnspython code base for python 2 and python 3, but right now it's out of my scope (having a voting py34 gate for swift) | 11:07 |
| openstackgerrit | Ken'ichi Ohmichi proposed openstack-infra/project-config: Make dsvm tests run on etc changes https://review.openstack.org/202980 | 11:07 |
| oomichi | sdague: ^^^ for making dsvm tests unskipped when /etc changes | 11:08 |
| oomichi | i need to go back, see you :) | 11:09 |
| *** oomichi has quit IRC | 11:09 | |
| *** shardy_ has joined #openstack-infra | 11:09 | |
| *** markvoelker has quit IRC | 11:11 | |
| *** shardy has quit IRC | 11:11 | |
| *** teran has quit IRC | 11:13 | |
| *** teran has joined #openstack-infra | 11:13 | |
| *** coolsvap is now known as coolsvap|away | 11:14 | |
| *** teran has quit IRC | 11:14 | |
| *** teran has joined #openstack-infra | 11:14 | |
| *** Guest13513 has quit IRC | 11:14 | |
| *** shardy_ has quit IRC | 11:15 | |
| *** shardy has joined #openstack-infra | 11:15 | |
| *** cdent has joined #openstack-infra | 11:16 | |
| *** fhubik has quit IRC | 11:19 | |
| *** markvoelker has joined #openstack-infra | 11:19 | |
| *** ihrachyshka has quit IRC | 11:20 | |
| *** teran has quit IRC | 11:20 | |
| *** teran has joined #openstack-infra | 11:21 | |
| *** Trozz has quit IRC | 11:22 | |
| *** teran has quit IRC | 11:23 | |
| *** Sherif has joined #openstack-infra | 11:23 | |
| *** Trozz has joined #openstack-infra | 11:23 | |
| *** teran has joined #openstack-infra | 11:23 | |
| *** markvoelker has quit IRC | 11:24 | |
| *** pal has joined #openstack-infra | 11:27 | |
| *** Sherif has quit IRC | 11:27 | |
| openstackgerrit | Chris Jones proposed openstack-infra/glean: Refuse to replace pre-existing interface configs. https://review.openstack.org/202984 | 11:29 |
| *** sdake has quit IRC | 11:32 | |
| *** markvoelker has joined #openstack-infra | 11:32 | |
| *** pal has quit IRC | 11:33 | |
| *** alex_xu is now known as alexus | 11:34 | |
| *** ihrachyshka has joined #openstack-infra | 11:34 | |
| *** ujuc has quit IRC | 11:34 | |
| *** ldnunes has joined #openstack-infra | 11:35 | |
| *** zeih has quit IRC | 11:36 | |
| *** yfried has joined #openstack-infra | 11:37 | |
| *** Somay has quit IRC | 11:39 | |
| *** ddieterly has joined #openstack-infra | 11:40 | |
| *** markvoelker has quit IRC | 11:44 | |
| *** ddieterly has quit IRC | 11:45 | |
| openstackgerrit | Adam Coldrick proposed openstack-infra/infra-specs: StoryBoard: Worklists and Boards https://review.openstack.org/202989 | 11:47 |
| *** dprince has joined #openstack-infra | 11:49 | |
| *** ddieterly has joined #openstack-infra | 11:50 | |
| greghaynes | Ng: curious what the motivation for ^ is? | 11:50 |
| *** yfried has quit IRC | 11:51 | |
| Ng | greghaynes: three things, 1) I think it's the right thing to do (operators know best!), 2) there was a TODO asking for it, 3) I wanted to poke around glean's code :D | 11:51 |
| greghaynes | we talked about remembering the config for a givrn instance uuid and not re-asserting state after weve run once forbthat config | 11:52 |
| greghaynes | wondering if that would also fix what youre going for | 11:52 |
| *** yfried has joined #openstack-infra | 11:53 | |
| Ng | greghaynes: I'd be interested to hear mordred's thoughts, but potentially that doesn't fix it - if an operator has loaded a config into an image with the intention of it staying put, even with the one-shot uuid idea, we'd still clobber their file | 11:53 |
| greghaynes | yep | 11:53 |
| Ng | this kind of "which component should win the race" stuff is intractable, someone will always pop up and say "but my distro includes that config file and I don't want it, so please make glean clobber it anyway" ;) | 11:54 |
| *** markvoelker has joined #openstack-infra | 11:55 | |
| *** markus_z has quit IRC | 11:57 | |
| *** otter768 has joined #openstack-infra | 11:57 | |
| *** gmann has joined #openstack-infra | 11:58 | |
| mordred | Ng: what did I do? | 11:58 |
| mordred | has read scrollback ... | 11:59 |
| *** gmann has quit IRC | 11:59 | |
| *** gmann has joined #openstack-infra | 11:59 | |
| mordred | and yeah - I think also, you know, if glean is used to bootstrap config management, and then config management does things, and then the machine is rebooted - we don't want glean clobbering | 11:59 |
| Ng | yes | 11:59 |
| *** markvoelker has quit IRC | 12:00 | |
| *** yfried_ has joined #openstack-infra | 12:01 | |
| *** yfried has quit IRC | 12:01 | |
| mordred | Ng: there are changes to the fixtures which seem unrelated - are those intentional? | 12:01 |
| *** amotoki_ has quit IRC | 12:02 | |
| *** rossella_s has quit IRC | 12:02 | |
| *** otter768 has quit IRC | 12:02 | |
| Ng | mordred: all the eth2 sys entries? | 12:02 |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Decouple openstack_project::server from puppetdb class https://review.openstack.org/172871 | 12:02 |
| Ng | hmm, I didn't actually test it without those present, I figured I'd fully populate the eth2 metadata and just went with it | 12:03 |
| *** dizquierdo has quit IRC | 12:04 | |
| mordred | okie. cool. | 12:04 |
| mordred | yay! | 12:05 |
| *** jasondotstar has quit IRC | 12:06 | |
| *** maishsk_ has joined #openstack-infra | 12:07 | |
| *** markvoelker has joined #openstack-infra | 12:09 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/glean: Support explicitly configured DHCP interfaces https://review.openstack.org/202250 | 12:09 |
| mordred | Ng, greghaynes: ^^ I rebased my thing that added an eth2 on top of Ng's thing | 12:09 |
| *** markvoelker has quit IRC | 12:13 | |
| *** isviridov_away is now known as isviridov | 12:15 | |
| *** markvoelker has joined #openstack-infra | 12:16 | |
| Ng | mordred: hmm, we might need an eth3 for that, since yours is expecting an interface to be written for eth2 and mine is specifically asserting that no such interface is written. This is my fault for not basing mine off the top of yours and adding an eth3, sorry! | 12:16 |
| *** dizquierdo has joined #openstack-infra | 12:16 | |
| Ng | I can rework yours after lunch? | 12:17 |
| *** yfried_ has quit IRC | 12:17 | |
| *** bmwiedemann has left #openstack-infra | 12:18 | |
| mordred | Ng: sure! | 12:20 |
| mordred | Ng: and _duh_ - I'm still first-coffeeing | 12:20 |
| *** dkranz has joined #openstack-infra | 12:20 | |
| *** kushal has quit IRC | 12:21 | |
| *** markvoelker has quit IRC | 12:21 | |
| *** jasondotstar has joined #openstack-infra | 12:21 | |
| *** baoli has joined #openstack-infra | 12:21 | |
| *** EricGonczer_ has joined #openstack-infra | 12:23 | |
| *** asselin has joined #openstack-infra | 12:23 | |
| *** maishsk_ has quit IRC | 12:24 | |
| *** mmedvede_ has joined #openstack-infra | 12:24 | |
| *** Somay has joined #openstack-infra | 12:24 | |
| *** markvoelker has joined #openstack-infra | 12:24 | |
| *** dims_ has joined #openstack-infra | 12:25 | |
| clarkb | mordred cpnfig drive is enabled for devstack trusty now. Will apply it to the other hpcloud dib nodes once I am actually ready to think today | 12:25 |
| *** zeih has joined #openstack-infra | 12:25 | |
| TheJulia | speaking of Glean, when are you folks expecting the next release of it to appear :) | 12:25 |
| clarkb | sdague ^ that has multibode working in hpcloud fro what I can see. Also full tempest passes, can we delete the smoke jovs? | 12:26 |
| mordred | TheJulia: probably as soon as Ng and my patch are landed | 12:26 |
| mordred | clarkb: woot! | 12:26 |
| TheJulia | awesome | 12:27 |
| sdague | clarkb: so, I'd say lets give it a little time, and figure out which multinode jobs we feel comfortable with on the check queue for d-g | 12:27 |
| sdague | because I noticed in this process there are none today | 12:28 |
| clarkb | sdague correct, my original plan was to just delete the single node tests and replace with multinode | 12:28 |
| clarkb | and run the full test suite | 12:28 |
| clarkb | (I still have no idea why the smoke jobs were added) | 12:28 |
| sdague | clarkb: the smoke job was added because live migration was not working reliably enough to make it vote | 12:29 |
| *** aysyd has joined #openstack-infra | 12:30 | |
| clarkb | sdague which took5 minutes of debuggibg and talking to gmann to fix unless I am missi g something | 12:30 |
| sdague | and when I proposed all partial upgrade approaches in grenade be multinode, there was a bit of pushback because of multinode reliabilty | 12:30 |
| *** BharatK has quit IRC | 12:30 | |
| sdague | so jogo pushed a smoke experimental job to just demonstrate that smoke is pretty reliable, what grenade uses, so that was a sane approach | 12:30 |
| sdague | clarkb: well it did after we got through all the hostname issues | 12:31 |
| clarkb | all the hostname issues affect smoke too | 12:31 |
| sdague | sure | 12:31 |
| sdague | but there were timing things here, and perceptions about multinode | 12:31 |
| sdague | anyway, you asked why it showed up | 12:31 |
| *** isviridov is now known as isviridov_away | 12:31 | |
| *** markvoelker has quit IRC | 12:32 | |
| *** thorst has joined #openstack-infra | 12:33 | |
| *** sdake has joined #openstack-infra | 12:33 | |
| clarkb | do those extra jobs provide any value now? | 12:34 |
| *** deepakcs has quit IRC | 12:34 | |
| clarkb | smoke is a subset of full and the actual issues appear to have been related to the infra not tempest test selection. we have fixed the infra for both sets of tests | 12:34 |
| *** thorst has quit IRC | 12:35 | |
| *** ajmiller has quit IRC | 12:35 | |
| openstackgerrit | Andrei V. Ostapenko proposed openstack-infra/project-config: Adds support of external Devstack plugin for MagnetoDB https://review.openstack.org/203004 | 12:35 |
| *** hdd has joined #openstack-infra | 12:35 | |
| *** ajmiller has joined #openstack-infra | 12:35 | |
| *** teran_ has joined #openstack-infra | 12:36 | |
| *** pc_m has quit IRC | 12:36 | |
| *** _ody has joined #openstack-infra | 12:36 | |
| *** pc_m has joined #openstack-infra | 12:36 | |
| *** pal has joined #openstack-infra | 12:37 | |
| *** teran has quit IRC | 12:37 | |
| *** coolsvap|away is now known as coolsvap | 12:37 | |
| *** zeih has quit IRC | 12:37 | |
| mescanef | hi, please add me (mmatuszkowiak@mirantis.com) to the gerrits acl groups regarding node /log-storage([0-1])([1-2])-(msk|srt)\.fuel-infra\.org/ { hiera_include('classes') | 12:38 |
| mescanef | }ops | 12:38 |
| mescanef | :-) | 12:38 |
| mescanef | sorry | 12:39 |
| mescanef | wrong maste | 12:39 |
| greghaynes | TheJulia: what do you think of clint's comment on your glean patch about solving the re-assertion issue in a later patch? | 12:39 |
| *** markvoelker has joined #openstack-infra | 12:39 | |
| mescanef | hi, please add me (mmatuszkowiak@mirantis.com) to the gerrits acl groups regarding https://review.openstack.org/#/c/202233/ | 12:39 |
| mescanef | :-) | 12:39 |
| greghaynes | I might poke at doing that if I get a bit of downtime... | 12:39 |
| *** dingyichen has quit IRC | 12:40 | |
| *** _ody has quit IRC | 12:41 | |
| *** isviridov_away is now known as isviridov | 12:41 | |
| sdague | clarkb: probably not | 12:41 |
| TheJulia | greghaynes: perfectly fine with that, I just pulled my -1 off that review since I don't want to hold you guys up at all. | 12:41 |
| *** eharney has quit IRC | 12:41 | |
| sdague | clarkb: I'm fine with it going away | 12:42 |
| *** ihrachyshka has quit IRC | 12:42 | |
| sdague | I just figured we'd do a job adjustment all at once when we had the grenade stuff ready to go | 12:42 |
| sdague | which is delete grenade-cpu-partial entirely, replace it with grenade-multinode | 12:42 |
| greghaynes | TheJulia: awesome | 12:43 |
| sdague | drop multinode-smoke | 12:43 |
| sdague | get multinode into the d-g check queue | 12:43 |
| sdague | even if -nv | 12:43 |
| *** haypo has left #openstack-infra | 12:43 | |
| sdague | but have it run all the time | 12:43 |
| *** markvoelker has quit IRC | 12:43 | |
| clarkb | sdague ya, though I still think replacing the ezisting nova net and neutron jobs woth multinode would be neat | 12:44 |
| sdague | clarkb: it would be | 12:44 |
| sdague | but, until we get more cloud resources, I think it's a hard trade off | 12:45 |
| sdague | and I also think we probably have to take another pass at debugging tools to make it easier for people to understand what's going on, because it is more complicated | 12:45 |
| openstackgerrit | Andrei V. Ostapenko proposed openstack-infra/project-config: Adds support of external Devstack plugin for MagnetoDB https://review.openstack.org/203004 | 12:46 |
| clarkb | yes though we already run 3 multinode runs per nova patch whoch is the vast volume of work we do | 12:46 |
| *** woodster_ has joined #openstack-infra | 12:46 | |
| TheJulia | greghaynes: the way I look at it, it is infra's tool, I just happen to re-use it because I'm lazy.... and don't like waiting 3 minutes for my test machines to boot. :) | 12:46 |
| pc_m | Hi folks! I have some questions on devstack plugins and grenade plugins that I'm hoping I can seek some guidance/advice on? | 12:46 |
| clarkb | doing the replacement I suggest gets us down to 2 and removes 2 single node jobs | 12:46 |
| *** zeih has joined #openstack-infra | 12:47 | |
| greghaynes | TheJulia: yea, but its a totally valid issue so glad it came up :) | 12:48 |
| mordred | TheJulia: it's everybody's tool, we just provide it a home | 12:48 |
| pc_m | clarkb: sdague: Would either of you have time for some questions? | 12:48 |
| *** isviridov is now known as isviridov_away | 12:48 | |
| TheJulia | :) | 12:48 |
| mordred | greghaynes, TheJulia: I do not see a patch from TheJulia on glean | 12:49 |
| * pc_m or anyone else knowledgable about devstack plugins, grenade plugins, and their relationship to tests | 12:49 | |
| TheJulia | mordred: and feed it some code now and then to keep it happy :) | 12:49 |
| clarkb | pc_m I dont really know much beyobd what the devstack docs say on plugins but typically its best to just ask rather than asking to ask | 12:49 |
| greghaynes | dont feed the projects | 12:49 |
| pc_m | clarkb: ok. here goes... | 12:49 |
| pc_m | I have upstreamed a devstack plugin for VPNaaS repo. Next I tried to remove VPN plugin from the devstacak repo, but realized that a bunch of tests are failing as a result. | 12:50 |
| pc_m | I just pushed up https://review.openstack.org/#/c/202985 as I *think* I need to enable the new devstack plugin in gate_hook.sh. Is that right, and did I do that correctly? | 12:51 |
| *** eharney has joined #openstack-infra | 12:51 | |
| gmann | clarkb: sdague : schema issue fix is merged - https://review.openstack.org/#/c/202837/ | 12:52 |
| clarkb | you enable the plugin as part of your localrc iirc | 12:52 |
| gmann | clarkb: sdague : but host name issue comes sometimes - http://logs.openstack.org/91/202391/5/check/gate-tempest-dsvm-multinode-full/f838b41/logs/testr_results.html.gz | 12:52 |
| clarkb | so ya update that job to enable the plugin, then remove the old code | 12:52 |
| *** julim has joined #openstack-infra | 12:52 | |
| *** julim has quit IRC | 12:53 | |
| pc_m | clarkb: Is the mod I did to gate-hook.sh the right way to enable the plugin in the repo? | 12:53 |
| *** julim has joined #openstack-infra | 12:53 | |
| *** markvoelker has joined #openstack-infra | 12:53 | |
| *** scheuran has quit IRC | 12:54 | |
| pc_m | for tests | 12:54 |
| *** dims_ has quit IRC | 12:54 | |
| sdague | pc_m: you seem to have a very different complex system of hook scripts, so I don't know in that case. If this was in the project-config yaml the answer is yes | 12:54 |
| *** ihrachyshka has joined #openstack-infra | 12:54 | |
| clarkb | pc_m I have no idea depebds on how your gate hook functions | 12:54 |
| pc_m | I set DEVSTACK_LOCAL_CONFIG | 12:55 |
| sdague | so you are on your own here figuring out how to get that into your config | 12:55 |
| clarkb | gmann thanks will have to look through that, the two node names there are the two hosts involved | 12:55 |
| sdague | go look at your generated localrc | 12:55 |
| pc_m | sdague: Following what was done in neutron, and neutron-lbaas pretty much for the hook scripts. | 12:55 |
| *** isviridov_away is now known as isviridov | 12:55 | |
| sdague | pc_m: that's fine, that's different from the way most of the rest of the projects work | 12:55 |
| *** mrmartin has quit IRC | 12:55 | |
| clarkb | gmann the one with the longer name is the compute node and the shorter name is the all in one controller + compute | 12:55 |
| gmann | clarkb: ohhk | 12:56 |
| pc_m | sdague: So look at the logs for the jobs that run with this commit and see if the localrc looks right | 12:56 |
| pc_m | ? | 12:56 |
| clarkb | gmann so almost looks like a migration isnt moving to the other host? | 12:56 |
| *** tlian has joined #openstack-infra | 12:57 | |
| *** markvoelker has quit IRC | 12:58 | |
| gmann | clarkb: and in that case server becomes ACTIVE, like migration is completed successfully | 12:58 |
| pc_m | Second thing is I'm trying to setup grenade testing for this repo. I pushed up https://review.openstack.org/#/c/202773 to make the grenade job experimental for neutron-vpnaas. Can we get that approved, so that I can test out the plugin I'm working? | 12:59 |
| *** dustins has joined #openstack-infra | 12:59 | |
| *** gmann has quit IRC | 13:00 | |
| openstackgerrit | Sébastien Han proposed openstack-infra/project-config: Add Ceph DevStack plugin to Stackforge https://review.openstack.org/188768 | 13:01 |
| *** dalgaaf has joined #openstack-infra | 13:03 | |
| *** isviridov is now known as isviridov_away | 13:03 | |
| *** isviridov_away is now known as isviridov | 13:03 | |
| *** isviridov is now known as isviridov_away | 13:03 | |
| *** alivigni has joined #openstack-infra | 13:04 | |
| *** samuel-dmq has quit IRC | 13:04 | |
| sdague | clarkb: if you could review that devstack-gate patch today for grenade multinode, it would be really appreciated. I'd like to get that into landing state as soon as possible so we can reshuffle all these jobs. | 13:04 |
| *** baoli has quit IRC | 13:04 | |
| *** hdd has quit IRC | 13:04 | |
| *** samueldmq has joined #openstack-infra | 13:04 | |
| clayton | anyone know if there is a way to change the font for gerrit? The monospace font makes it impossible to tell the difference between a hyphen and a tilde | 13:05 |
| sdague | it also exposed the fact that we probably need to do some more robust checking on multinode situations to make sure that the workers are getting used, because if the worker node fails badly enough, it can just not register itself, and all the tests patch | 13:05 |
| *** markvoelker has joined #openstack-infra | 13:05 | |
| sdague | clayton: use something like stylish for your browser | 13:05 |
| pc_m | sdague: clarkb: With the grenade plugin that I'm creating, how do I tell it to enable q-vpn (via the devstack plugin)? | 13:05 |
| mescanef | hi, please add me (mmatuszkowiak@mirantis.com) to the gerrits acl groups regarding https://review.openstack.org/#/c/202233/ | 13:06 |
| mescanef | fungi: ^^ | 13:06 |
| mescanef | or | 13:06 |
| mescanef | SergeyLukjanov: ^^ | 13:06 |
| mescanef | :-) | 13:06 |
| sdague | pc_m: there is an option in grenade to set devstack config file items | 13:06 |
| sdague | and you do that to enable the plugin | 13:06 |
| *** markvoelker_ has joined #openstack-infra | 13:07 | |
| sdague | see https://github.com/openstack-dev/grenade/blob/master/PLUGINS.rst last section | 13:07 |
| pc_m | sdague:Yeah, I saw that but was unsure how to specify the plugin enabling with the devstack_localrc line. | 13:08 |
| pc_m | sdague: I need a "enable_plugin neutron-vpnaas https://git.openstack.org/openstack/neutron-vpnaas" | 13:09 |
| *** markvoelker has quit IRC | 13:09 | |
| *** rlandy has joined #openstack-infra | 13:09 | |
| pc_m | sdague: looked like the settings were to just enable services. Am I missing something? Or misunderstanding something? | 13:10 |
| *** bnemec has joined #openstack-infra | 13:11 | |
| pc_m | sdague: Or can I have multiple devstack_localrc base... lines and one would have all the plugin enable stuff? | 13:11 |
| *** jasondotstar has quit IRC | 13:11 | |
| *** Sherif has joined #openstack-infra | 13:12 | |
| *** baoli has joined #openstack-infra | 13:12 | |
| *** shardy_ has joined #openstack-infra | 13:12 | |
| *** shardy has quit IRC | 13:13 | |
| clayton | sdague: good suggestion, if only I could puzzle out enough CSS to make it work | 13:15 |
| *** baoli has quit IRC | 13:15 | |
| *** Sherif has quit IRC | 13:16 | |
| *** jsavak has joined #openstack-infra | 13:16 | |
| *** isviridov_away is now known as isviridov | 13:16 | |
| *** isviridov is now known as isviridov_away | 13:17 | |
| *** shardy_ has quit IRC | 13:17 | |
| *** shardy has joined #openstack-infra | 13:18 | |
| pc_m | sdague: Looking at the code, it looks like devstack_localrc just adds arbitrary info to localrc and we can call it multiple times, so I can use one line for the devstack plugin. | 13:18 |
| *** doude has quit IRC | 13:18 | |
| *** xyang1 has joined #openstack-infra | 13:18 | |
| *** doude has joined #openstack-infra | 13:19 | |
| sdague | pc_m: you can call it as many times as you like | 13:19 |
| openstackgerrit | Victor Stinner proposed openstack/requirements: Bump version of glance_store to >= 0.7.0 https://review.openstack.org/203021 | 13:19 |
| pc_m | sdague: Had two more questions about grenade plugin... 1) For verify, I was going to check that the vpn agent was running. Is that OK, and should I do it via checking for process or by using 'neutron agent-list' (as there is a delay when service is stopped)? | 13:20 |
| *** Kiall has quit IRC | 13:20 | |
| sdague | whatever you think is appropriate | 13:21 |
| *** Kiall has joined #openstack-infra | 13:21 | |
| *** hdd has joined #openstack-infra | 13:22 | |
| pc_m | sdague: 2) for update.sh, I was going to do neutron-db-migrate --service vpnaas... to update the database for the service. Will I need to also do the neutron-db-migrate for neutron (first)? The neutron update.sh won't get invoked, so I'm thinking that would not happen. | 13:22 |
| pc_m | sdague: I'm a little confused by the neutron update.sh, as it seems to do quite a bit more (not sure if I need to duplicate that or what). | 13:23 |
| * pc_m still trying to wrap my head around all this | 13:23 | |
| *** peristeri has joined #openstack-infra | 13:24 | |
| *** mriedem has joined #openstack-infra | 13:24 | |
| pc_m | sdague: neutron's upgrade.sh (not update.sh, sorry) is under grenade (no neutron plugin). | 13:25 |
| *** baoli has joined #openstack-infra | 13:26 | |
| kashyap | Hi folks, why don't the tox logs open in browser, instead offer to be downloaded? -- http://logs.openstack.org/73/202973/1/check/gate-nova-python27/665d6f4/tox/ | 13:26 |
| clayton | sdague: got it, thanks for the suggestion. tilde's are really easy to tell from hyphens with Monaco | 13:26 |
| fungi | mescanef: i've added you as the initial member of fuel-plugin-ldap-core in gerrit now | 13:28 |
| *** dansmith is now known as superdan | 13:30 | |
| fungi | kashyap: probably just needs some mime magic in the vhost config | 13:30 |
| kashyap | fungi: Thanks, that'd be cool. Meanwhile, I've been using your trick of `BROWSER=lynx gertty`. :-) | 13:31 |
| *** kambiz has quit IRC | 13:31 | |
| mescanef | fungi: thanks :-) | 13:31 |
| fungi | kashyap: yep, i need to submit a patch to gertty to refresh the terminal on exiting the browser, just haven't had time to dig into how to go about it yet. other than that it's working really well for me | 13:32 |
| *** BharatK has joined #openstack-infra | 13:32 | |
| fungi | i've gotten used to hitting ctrl-l in the meantime | 13:32 |
| *** radez_g0n3 is now known as radez | 13:32 | |
| *** Somay has quit IRC | 13:32 | |
| openstackgerrit | Adam Coldrick proposed openstack-infra/infra-specs: StoryBoard: Worklists and Boards https://review.openstack.org/202989 | 13:33 |
| *** markvoelker_ has quit IRC | 13:34 | |
| kashyap | fungi: Ah, I see. /me was looking here -- http://lynx.isc.org/current/breakout/lynx_help/keystrokes/keystroke_help.html | 13:35 |
| *** wyatt88 has quit IRC | 13:35 | |
| *** dguerri` has quit IRC | 13:37 | |
| *** larainema has quit IRC | 13:37 | |
| *** larainema has joined #openstack-infra | 13:37 | |
| *** tcammann has quit IRC | 13:38 | |
| fungi | yeah, the issue (i assume you have it too) is that when you close lynx you end up with a blank screen because gertty doesn't know the terminal has been hijacked by a child process and so doesn't redisplay on its own | 13:38 |
| *** anteaya has quit IRC | 13:38 | |
| pc_m | sdague: Look like I'm stuck a bit. I created 202985 to enable devstack plugin for VPN tests, but because there is no VPN grenade pluging, and it is running gate-grenade-dsvm-neutron, that job is failing. I have 202773 out for review that makes the grenade test run under experimental, but it must already be enabled for neutron-vpnaas somehow? | 13:39 |
| fungi | i think the cheap solution is to just cram in whatever urwid uses for a screen refresh call after the exec for the browser | 13:39 |
| *** bhunter71_2 has joined #openstack-infra | 13:39 | |
| *** SlickNik has quit IRC | 13:39 | |
| *** tcammann has joined #openstack-infra | 13:40 | |
| fungi | since gertty seems to block on the child process in the in-terminal browser situation | 13:40 |
| fungi | i just haven't had time to hack that in | 13:40 |
| *** dguerri` has joined #openstack-infra | 13:40 | |
| *** tvanderwiel has joined #openstack-infra | 13:41 | |
| *** dguerri` is now known as dguerri | 13:41 | |
| *** dguerri has joined #openstack-infra | 13:41 | |
| *** Shrews has quit IRC | 13:41 | |
| *** kambiz has joined #openstack-infra | 13:41 | |
| *** Guest56842 has quit IRC | 13:41 | |
| *** SlickNik has joined #openstack-infra | 13:42 | |
| *** anteaya has joined #openstack-infra | 13:42 | |
| *** spiffxp has joined #openstack-infra | 13:42 | |
| *** spiffxp is now known as Guest67161 | 13:42 | |
| pc_m | sdague: It looks like gate-grenade-dsvm-neutron is part of integrated-gate-neutron. Is that somehow run for neutron-vpnaas repo? | 13:43 |
| *** Shrews has joined #openstack-infra | 13:43 | |
| *** dims_ has joined #openstack-infra | 13:45 | |
| *** Somay has joined #openstack-infra | 13:46 | |
| *** zz_dimtruck has quit IRC | 13:47 | |
| *** amitgandhinz has joined #openstack-infra | 13:47 | |
| *** leseb_ has joined #openstack-infra | 13:48 | |
| *** markvoelker has joined #openstack-infra | 13:49 | |
| *** zz_dimtruck has joined #openstack-infra | 13:49 | |
| *** zz_dimtruck is now known as dimtruck | 13:49 | |
| *** mwagner_lap has joined #openstack-infra | 13:51 | |
| openstackgerrit | Doug Hellmann proposed openstack-infra/project-config: Add validation job for releases repository https://review.openstack.org/202785 | 13:51 |
| *** wenlock has joined #openstack-infra | 13:51 | |
| leseb_ | AJaeger: did I get it right? http://pastebin.com/jtq3Fu5b not sure about the zuul config | 13:52 |
| pc_m | All: So new devstack plugin enabling fails in neutron-vpn, because it runs the grenade job and that does not enable the plugin as there is no grenade plugin yet. Also, I wanted grenade job as experimental, so I can test it, but it looks like it is running in the neutron-vpnaas commits as part of integrated-gate-neutron. Suggestions on how to deal with all this? | 13:53 |
| anteaya | pc_m: morning | 13:53 |
| pc_m | anteaya: hi | 13:54 |
| anteaya | pc_m: this is the repo we are discussing? http://git.openstack.org/cgit/openstack-infra/project-config/tree/zuul/layout.yaml#n2422 | 13:54 |
| *** markvoelker has quit IRC | 13:54 | |
| pc_m | anteaya: Yes. Got into a ball of mud, trying to enable devstack plugin and grenade plugin for neutron-vpnaas | 13:55 |
| anteaya | I see that | 13:55 |
| anteaya | which is the grenade job you wanted to be in the experimental queue? | 13:55 |
| pc_m | anteaya: I've got a pile of commits out right now... | 13:55 |
| pc_m | anteaya: Could I do this approach (thinking out loud)... | 13:56 |
| anteaya | I don't see a job jumping out at me that I recognize as grenade | 13:56 |
| *** doude has quit IRC | 13:56 | |
| anteaya | let's do one thing at a time | 13:56 |
| pc_m | 1) Do grenade plugin that enables the devstack plugin for neutron-vpnaas | 13:56 |
| anteaya | the grenade job you wanted as experiemental | 13:56 |
| anteaya | what is it called? | 13:56 |
| pc_m | 2) Then enable devstack pugin for gate-hook.sh so that it is available for jobs (and grenade already passes) | 13:57 |
| anteaya | the grenade job you want running as experimental | 13:57 |
| anteaya | what is the job called? | 13:57 |
| pc_m | 3) Ignore the attempts to making the grenade job experimental (as it is already in check/gate queue because of the integrated-gate-neutron (template?) | 13:57 |
| pc_m | gate-grenade-dsvm-neutron | 13:58 |
| * anteaya looks at the integrated-gate-neutron | 13:58 | |
| * anteaya looks at gate-grenade-dsvm-neutron | 13:58 | |
| pc_m | anteaya: It appears to already be included in integrated-gate-neutron. | 13:58 |
| *** teran_ has quit IRC | 13:58 | |
| *** teran has joined #openstack-infra | 13:58 | |
| *** otter768 has joined #openstack-infra | 13:58 | |
| *** ayoung has joined #openstack-infra | 13:58 | |
| pc_m | anteaya: I tried to enable the devstack plugin, but the grenade test failed (as there is no grenade plugin yet - I have one, but haven't pushed it yet). | 13:59 |
| *** BharatK has quit IRC | 13:59 | |
| *** markvoelker has joined #openstack-infra | 13:59 | |
| anteaya | http://git.openstack.org/cgit/openstack-infra/project-config/tree/zuul/layout.yaml#n282 | 13:59 |
| anteaya | let's focus on getting the job in the right pipeline first | 13:59 |
| anteaya | then we can talk about the plugin | 14:00 |
| * anteaya looks at how to unwind the jobs | 14:00 | |
| *** gyee has joined #openstack-infra | 14:00 | |
| pc_m | anteaya: Issue is that integrated-gate-neutron is used by other repos. | 14:00 |
| anteaya | I see that | 14:02 |
| *** zeih has quit IRC | 14:02 | |
| anteaya | mind if I offer a patch as a suggestion to how to get the job you want in the pipeline where you want it? | 14:03 |
| openstackgerrit | Sébastien Han proposed openstack-infra/project-config: Add Ceph DevStack plugin to Stackforge https://review.openstack.org/188768 | 14:03 |
| *** otter768 has quit IRC | 14:03 | |
| pc_m | anteaya: sure | 14:03 |
| anteaya | pc_m: thanks, stand by | 14:04 |
| *** shardy_ has joined #openstack-infra | 14:04 | |
| kozhukalov | guys, i can not understand what is wrong here http://logs.openstack.org/41/202541/2/check/gate-project-config-irc-access/29d363e/console.html could you please help me to understand what I shoud fix? | 14:04 |
| *** shardy has quit IRC | 14:04 | |
| openstackgerrit | Jeremy Stanley proposed stackforge/gertty: Redisplay after spawning browser https://review.openstack.org/203041 | 14:04 |
| *** julim_ has joined #openstack-infra | 14:05 | |
| fungi | kashyap: ^ i decided to get around to it | 14:05 |
| kashyap | fungi: Nice, will test in a few. | 14:05 |
| kashyap | Thanks! | 14:05 |
| openstackgerrit | Vladimir Kozhukalov proposed openstack-infra/project-config: Add fuel-upgrade repo https://review.openstack.org/202541 | 14:05 |
| fungi | kashyap: it's working for me here anyway | 14:06 |
| *** sigmavirus24_awa is now known as sigmavirus24 | 14:06 | |
| *** _nadya_ has quit IRC | 14:06 | |
| fungi | no more ctrl-l! (for that anyway) | 14:06 |
| *** eharney has quit IRC | 14:06 | |
| *** _nadya_ has joined #openstack-infra | 14:06 | |
| kashyap | Nice. I'll test too. | 14:06 |
| pc_m | anteaya: Also consider https://review.openstack.org/#/c/201279/, which is disabling services from grenade test, as migration is broken. | 14:07 |
| *** markvoelker has quit IRC | 14:07 | |
| *** dboik has joined #openstack-infra | 14:08 | |
| *** julim has quit IRC | 14:08 | |
| openstackgerrit | Matt Riedemann proposed openstack/requirements: Block mock 1.1.4 https://review.openstack.org/203043 | 14:08 |
| pc_m | anteaya: Another option is to use 201279, which would prevent testing of service in grenade, and then create a new grenade job for neutron-vpnaas? | 14:08 |
| mriedem | jroll: sdague: mdbooth: bauwser: ^ | 14:08 |
| bauwser | mriedem: coolness | 14:08 |
| *** shardy_ has quit IRC | 14:09 | |
| *** cinerama has quit IRC | 14:09 | |
| *** shardy has joined #openstack-infra | 14:10 | |
| *** ajmiller has quit IRC | 14:10 | |
| sdague | fungi: https://review.openstack.org/203043 is needed to unblock nova | 14:10 |
| *** _nadya_ has quit IRC | 14:11 | |
| *** Sherif has joined #openstack-infra | 14:11 | |
| *** ParsectiX has quit IRC | 14:12 | |
| kashyap | fungi: :-) - your commit message there. | 14:12 |
| *** markvoelker has joined #openstack-infra | 14:14 | |
| openstackgerrit | Anita Kuno proposed openstack-infra/project-config: Rearrange pipelines for neutron-vpnaas jobs https://review.openstack.org/203045 | 14:14 |
| anteaya | pc_m: take a look at 203045 and first of all tell me if the commit message is an accurate reflection of your wishes | 14:15 |
| pc_m | anteaya: looking... | 14:15 |
| *** Sherif has quit IRC | 14:16 | |
| pc_m | anteaya: yes, that looks correct. | 14:18 |
| *** markvoelker has quit IRC | 14:18 | |
| anteaya | wonderful | 14:18 |
| *** kushal has joined #openstack-infra | 14:18 | |
| anteaya | pc_m: wonderful please comment and vote | 14:19 |
| pc_m | Will that blend in with https://review.openstack.org/#/c/201279 ? | 14:19 |
| pc_m | anteaya: ^^ | 14:19 |
| anteaya | it will have zero impact on 201279 | 14:20 |
| openstackgerrit | Matthew Booth proposed openstack/requirements: Bump testtools minimum to 1.4.0 https://review.openstack.org/203046 | 14:20 |
| anteaya | the only thing that will change is that the grenade job will not run in the check queue | 14:20 |
| anteaya | it will only run in the experimental queue | 14:20 |
| anteaya | the job is still the same job | 14:20 |
| kozhukalov | AJaeger, pleia2, others could you please review this https://review.openstack.org/#/c/202541/ ? Thanks a lot in advance. | 14:22 |
| pc_m | anteaya: For neutron runs, we don't want the adv services enabled. I take it that the features.yaml change will only apply to neutron jobs for grenade? | 14:22 |
| anteaya | pc_m: let's focus on one thing at a time | 14:22 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/project-config: Remove neutron smoke job from integrated gate templates https://review.openstack.org/203048 | 14:23 |
| anteaya | your grenade job doesn't work on neutron-vpnaas, it fails in teh check queue, you need it moved to the experimental queue | 14:23 |
| fungi | heh. "old patterns, can be removed in July 2014" http://git.openstack.org/cgit/openstack-infra/puppet-openstackci/tree/templates/logs.vhost.erb#n51 | 14:23 |
| anteaya | pc_m: correct? | 14:23 |
| mtreinish | AJaeger, anteaya: ^^^ that'll be good to fix | 14:23 |
| mtreinish | we've been running a job everywhere for no reason | 14:23 |
| *** pal has quit IRC | 14:23 | |
| pc_m | anteaya: yes | 14:23 |
| *** unicell1 has joined #openstack-infra | 14:24 | |
| *** ddieterly has quit IRC | 14:24 | |
| anteaya | mtreinish: agreed, thank you, let's get rid of job cruft | 14:24 |
| pc_m | anteaya: Responded to 203045. Looks good. | 14:25 |
| anteaya | pc_m: great, I will update my patch to be a child of mtreinish's new patch and then we can get that job running in the correct queue | 14:25 |
| *** unicell has quit IRC | 14:25 | |
| pc_m | anteaya: Thanks! | 14:25 |
| anteaya | pc_m: then we can move to -qa and see about the plugin questions | 14:25 |
| *** mestery has quit IRC | 14:25 | |
| *** vlaza has quit IRC | 14:26 | |
| *** pal has joined #openstack-infra | 14:26 | |
| *** erlon has joined #openstack-infra | 14:26 | |
| pc_m | anteaya: Sounds good. I have a devstack plugin enabling commit, and am about to push a grenade plugin commit (as a WIP). | 14:27 |
| openstackgerrit | Matthew Booth proposed openstack/requirements: Increase testtools version to support unittest2 https://review.openstack.org/203050 | 14:27 |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/puppet-openstackci: Clean up logserver vhost templates https://review.openstack.org/203051 | 14:27 |
| *** yamahata has joined #openstack-infra | 14:27 | |
| *** markvoelker has joined #openstack-infra | 14:28 | |
| openstackgerrit | Anita Kuno proposed openstack-infra/project-config: Rearrange pipelines for neutron-vpnaas jobs https://review.openstack.org/203045 | 14:28 |
| samueldmq | good morning | 14:29 |
| *** akshai has joined #openstack-infra | 14:30 | |
| samueldmq | fungi: hi, I've a question concerning the discount code to the summit | 14:30 |
| samueldmq | fungi: I'd like to ask you as you were the one who sent the emails to ATCs :) | 14:30 |
| anteaya | the building is shaking which triggers a panic attack for me I have to move locations | 14:31 |
| anteaya | back in a bit | 14:31 |
| *** jamesmcarthur has joined #openstack-infra | 14:32 | |
| samueldmq | fungi: pm'd you | 14:32 |
| *** eharney has joined #openstack-infra | 14:32 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/puppet-askbot: Migrate to puppet-httpd module https://review.openstack.org/198827 | 14:32 |
| *** Kennan2 has quit IRC | 14:32 | |
| mriedem | is logstash dead? the job queue graph here is flatlined http://status.openstack.org/zuul/ | 14:33 |
| fungi | samueldmq: i don't see a pm, but i can certainly direct you to people who know about such things (i'm just the poor sap generating the list of contributors and firing off the mass mailer) | 14:33 |
| *** markvoelker has quit IRC | 14:33 | |
| *** Kennan has joined #openstack-infra | 14:33 | |
| Sam-I-Am | mornings | 14:37 |
| *** tsekiyama has joined #openstack-infra | 14:37 | |
| *** _ody has joined #openstack-infra | 14:37 | |
| *** ajmiller has joined #openstack-infra | 14:37 | |
| *** apmelton has quit IRC | 14:38 | |
| openstackgerrit | Davanum Srinivas (dims) proposed openstack-infra/system-config: Add fuel-library channel https://review.openstack.org/200310 | 14:38 |
| pabelanger | yolanda: ::Httpd::Vhost is not valid. I tested with ::File like nibalizer suggested | 14:38 |
| yolanda | mm, pabelanger, i tested with ::Httpd::Vhost on my instance and worked... let me check again | 14:39 |
| pabelanger | I tried ::File, but lets me to ::Httpd:Vhost | 14:39 |
| *** harlowja_at_home has joined #openstack-infra | 14:39 | |
| yolanda | i had exactly same doubt as you, so i tried | 14:39 |
| *** apmelton has joined #openstack-infra | 14:40 | |
| yolanda | pabelanger, before => ::Apache::Vhost['abc'], | 14:40 |
| yolanda | works for me | 14:40 |
| yolanda | so ::Httpd::Vhost should | 14:41 |
| pabelanger | I have a stackalytics node I can test on | 14:41 |
| yolanda | gerat | 14:41 |
| yolanda | great | 14:41 |
| kashyap | fungi: What else I need to do apart from `pip wheel -r requirements.txt` to get all of Gertty's deps installed in one go, in a venev? | 14:41 |
| *** _ody has quit IRC | 14:42 | |
| fungi | kashyap: you can just pip install the source tree instead and pbr should take care of pulling in the requirements for you. i do something like `~/pyenvs/gertty/bin/pip install -U ~/git/stackforge/gertty` | 14:43 |
| hashar | hello :) | 14:44 |
| hashar | I found out nodepool refuses to quit on SIGUSR1 . That does stop the scheduler but does not terminate the process :( | 14:44 |
| kashyap | fungi: Yep, that works just fine, from Gerrty's source dir: ./gertty-env/bin/pip install -U . | 14:44 |
| kashyap | Thanks. | 14:44 |
| *** zeih has joined #openstack-infra | 14:45 | |
| *** krtaylor has quit IRC | 14:45 | |
| pabelanger | yolanda: okay, ya it looks to work. But, internal puppet defines don't | 14:45 |
| yolanda | pabelanger, what do you mean, is that failing? | 14:45 |
| fungi | hashar: yeah, it's hit or miss for me. depends on whether there are image updates in progress i think? sigusr1 tries to gracefully stop | 14:45 |
| *** peristeri_ has joined #openstack-infra | 14:46 | |
| pabelanger | yolanda: before => ::File['foobar'] | 14:47 |
| pabelanger | would fail syntax | 14:47 |
| hashar | fungi: no images building for me. The main scheduler did stop though | 14:47 |
| openstackgerrit | Merged openstack/requirements: Bump glance_store to 0.7.0 for py34 compatibility https://review.openstack.org/202791 | 14:47 |
| pabelanger | Error: Invalid tag ::file | 14:47 |
| openstackgerrit | Paul Belanger proposed openstack-infra/puppet-cgit: Migrate to puppet-httpd module https://review.openstack.org/202604 | 14:47 |
| yolanda | mm, so that's a complicated point then, pabelanger. For one side i'd like to go with fully qualified classes, but if that is not following the puppet syntax i don't like it | 14:47 |
| *** hdd has quit IRC | 14:47 | |
| yolanda | maybe we can ignore the :: on before, after, and use that on the other parts | 14:48 |
| pabelanger | yolanda: well, I think we could ignore :: for Httpd::Vhost for example. Since they are type references | 14:49 |
| pabelanger | and I have never seen top-level scoping on that before | 14:49 |
| pabelanger | we should really see what the style guide says | 14:49 |
| *** peristeri has quit IRC | 14:49 | |
| pabelanger | but, I didn't see much | 14:49 |
| pabelanger | crinkle: thoughts? ^ | 14:49 |
| *** sputnik13 has joined #openstack-infra | 14:49 | |
| *** mtanino has joined #openstack-infra | 14:49 | |
| yolanda | pabelanger, yes, i'm not strong on style as well. Let's get feedback from some other | 14:50 |
| *** markvoelker has joined #openstack-infra | 14:50 | |
| yolanda | for requiring a class, i'd say is better to have it | 14:50 |
| *** Somay has quit IRC | 14:51 | |
| *** tonytan4ever has joined #openstack-infra | 14:51 | |
| *** achanda has joined #openstack-infra | 14:53 | |
| *** gyee has quit IRC | 14:53 | |
| *** markvoelker has quit IRC | 14:57 | |
| *** markvoelker_ has joined #openstack-infra | 14:57 | |
| *** markvoelker_ has quit IRC | 14:57 | |
| *** krtaylor has joined #openstack-infra | 14:57 | |
| *** markvoelker has joined #openstack-infra | 14:57 | |
| openstackgerrit | Sébastien Han proposed openstack-infra/project-config: Add Ceph DevStack plugin to Stackforge https://review.openstack.org/188768 | 14:57 |
| hashar | fungi: ah Nodepool.stop() does not stop everything :) | 14:58 |
| *** mestery has joined #openstack-infra | 14:58 | |
| fungi | hashar: sounds like a bug | 14:59 |
| *** hdd has joined #openstack-infra | 14:59 | |
| hashar | giving it a try | 14:59 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/subunit2sql: Update and fix some minor issues with docs https://review.openstack.org/201407 | 15:00 |
| *** jaypipes has joined #openstack-infra | 15:01 | |
| hashar | fungi: fixed one! :) | 15:02 |
| phschwartz | Is there a way with JJB to define a variable and then use it in a section on a job-group? IE, I want to set a version and then use it in all job-groups I have created. Currently I do it via a text setting in each job-group and I would like to make it common | 15:03 |
| *** bknudson has joined #openstack-infra | 15:03 | |
| *** dprince has quit IRC | 15:03 | |
| *** gyee has joined #openstack-infra | 15:03 | |
| *** tonytan4ever has quit IRC | 15:03 | |
| anteaya | morning Sam-I-Am | 15:04 |
| *** Guest77658 has quit IRC | 15:04 | |
| *** pal has quit IRC | 15:04 | |
| *** achanda has quit IRC | 15:05 | |
| *** Hal has joined #openstack-infra | 15:05 | |
| *** jsavak has quit IRC | 15:05 | |
| *** Hal is now known as Guest63561 | 15:05 | |
| *** f1ller is now known as filler | 15:05 | |
| pabelanger | yolanda: asking in #puppet | 15:05 |
| *** adreznec has quit IRC | 15:07 | |
| sdague | fungi: the requirements change just blew up on git cloning | 15:07 |
| sdague | https://jenkins06.openstack.org/job/gate-grenade-dsvm/13166/console | 15:07 |
| *** jsavak has joined #openstack-infra | 15:07 | |
| sdague | from zuul-cloner | 15:07 |
| sdague | which I thought was supposed to fix this | 15:07 |
| fungi | sdague: not cloning, remote update | 15:07 |
| sdague | sure, remote update | 15:07 |
| sdague | anyway, can that be rereenqueued | 15:08 |
| fungi | zuul-cloner doesn't prevent us from needing to talk to the git servers, just reduces the amount of data we need to transfer from them | 15:08 |
| crinkle | pabelanger: yolanda I have never seen that before so I'm guessing it's not necessary | 15:08 |
| sdague | ok, but it apparently doesn't have retry logic? | 15:08 |
| fungi | sdague: and yeah, i just reenqueued it again | 15:08 |
| mordred | why would git remote prune --dry-run origin fail? | 15:08 |
| yolanda | ok | 15:08 |
| crinkle | pabelanger: yolanda there's not a linter for it the way there is for class scope | 15:08 |
| sdague | mordred: no idea, but it did | 15:08 |
| yolanda | crinkle, so it's only needed for definitions, not for inclusions like class, before, anchor, etc... ? | 15:09 |
| sdague | mordred: --dry-run still talks over the network | 15:09 |
| fungi | mordred: i believe exit code 128 is git's generic "something happened on the way to the server" failure | 15:09 |
| *** adreznec has joined #openstack-infra | 15:09 | |
| mordred | sdague: yah - I am seeing that | 15:09 |
| sdague | it doesn't do the local operations | 15:09 |
| sdague | which is deleting refs | 15:09 |
| crinkle | yolanda: the :: is needed for class inclusions | 15:09 |
| mordred | wow - git remote prune is super expensive! | 15:09 |
| yolanda | crinkle, and in requires? | 15:09 |
| crinkle | yolanda: i'm not sure if it's needed for defined type declarations | 15:09 |
| * mordred is running that on a local repo and it's still hanging | 15:09 | |
| crinkle | yolanda: i don't think it's needed for requires | 15:09 |
| yolanda | ok | 15:10 |
| openstackgerrit | Matthew Treinish proposed openstack/os-testr: Improved docs for os-testr commands https://review.openstack.org/200332 | 15:10 |
| mordred | mordred@camelot:~/src/openstack/magnum$ time git remote prune --dry-run origin | 15:10 |
| yolanda | i'll take another look at my reviews then | 15:10 |
| mordred | real0m8.354s | 15:10 |
| mordred | fwiw | 15:10 |
| sdague | mordred: well it's an N by M comparison of all refs | 15:10 |
| mordred | ah | 15:10 |
| sdague | so, yeh, not that surprising | 15:10 |
| crinkle | yolanda: i couldn't actually find any recommendations though, so this is more of a guess | 15:10 |
| sdague | clarkb suggested not doing that any more, don't know where that ended up | 15:10 |
| yolanda | crinkle, will be great if we had some style definitions, even if we have that internally on openstack-infra | 15:10 |
| mordred | why do we care about pruning ... | 15:10 |
| mordred | yeah | 15:10 |
| fungi | mordred: we do prune in our cached clones now, so it's possible we could get by with a zuul-cloner option to skip pruning for performance purposes | 15:10 |
| *** tonytan4ever has joined #openstack-infra | 15:10 | |
| sdague | however, I'm going to go get out for a bike ride while the weather is still good | 15:10 |
| mordred | I mean - these are throwaway nodes | 15:10 |
| fungi | during job runtime | 15:10 |
| mordred | fungi: ++ | 15:11 |
| sdague | before going down another rabbit hole | 15:11 |
| crinkle | yolanda: ++ | 15:11 |
| mordred | sdague: enjoy! | 15:11 |
| sdague | but I will beg for more review of my grenade multinode d-g change - https://review.openstack.org/#/c/199091/ really would love to get that into landing shape next week | 15:11 |
| sdague | later folks | 15:11 |
| yolanda | pabelanger, then i'll remove some of my -1 | 15:11 |
| *** dougshelley66 has quit IRC | 15:12 | |
| *** sdake has quit IRC | 15:13 | |
| *** dougshelley66 has joined #openstack-infra | 15:13 | |
| *** FallenPegasus has quit IRC | 15:13 | |
| *** dtantsur is now known as dtantsur|afk | 15:14 | |
| *** Guest63561 has quit IRC | 15:15 | |
| *** dprince has joined #openstack-infra | 15:16 | |
| *** sergmelikyan has joined #openstack-infra | 15:17 | |
| *** bexelbie has quit IRC | 15:17 | |
| *** annegentle has joined #openstack-infra | 15:17 | |
| kashyap | fungi: Works well here too; thanks. Will add a note in the review. | 15:18 |
| *** annegentle is now known as Guest37424 | 15:18 | |
| openstackgerrit | Matthew Treinish proposed openstack/os-testr: Add TODO entry for moving away from subprocess in ostestr https://review.openstack.org/201636 | 15:18 |
| *** Guest37424 is now known as annegentle_ | 15:18 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/shade: Catch leaky exceptions from create_image() https://review.openstack.org/203076 | 15:19 |
| *** dannywilson has joined #openstack-infra | 15:20 | |
| Shrews | mordred: ^^^^ | 15:20 |
| *** coolsvap is now known as coolsvap|away | 15:23 | |
| *** zeih has quit IRC | 15:25 | |
| *** cinerama has joined #openstack-infra | 15:27 | |
| *** jaypipes is now known as blockedpipes | 15:27 | |
| openstackgerrit | litong01 proposed openstack-infra/project-config: Add new python-monasca repository for OpenStack monitoring service https://review.openstack.org/203081 | 15:28 |
| mordred | Shrews: ++ | 15:28 |
| *** sdake has joined #openstack-infra | 15:28 | |
| *** Sherif has joined #openstack-infra | 15:30 | |
| *** tongli has joined #openstack-infra | 15:31 | |
| *** ajmiller has quit IRC | 15:31 | |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Decouple openstack_project::server from puppetdb class https://review.openstack.org/172871 | 15:32 |
| *** dannywilson has quit IRC | 15:34 | |
| clarkb | fungi mordred sdague ya anything that needs correct remote refs in addition to a zuul ref should prune then z-c and d-g can skip doing it otherwise | 15:34 |
| openstackgerrit | Mark T. Voelker proposed openstack-infra/irc-meetings: Change DefCore Meeting Time https://review.openstack.org/203089 | 15:34 |
| *** samuelBartel has quit IRC | 15:35 | |
| pabelanger | yolanda: ya, have to go back remove some code too | 15:36 |
| *** ajmiller has joined #openstack-infra | 15:36 | |
| yolanda | i'm revisiting the list of changes now | 15:36 |
| fungi | clarkb: i'm pondering the "tox logs are not treated served as text/plain" issue on logs.o.o... there are obviously a number of ways to solve it (force mime type for .log in vhost, use mod_mime-magic, rename to .txt before uploading) but wondering if you have any opinions | 15:37 |
| *** frobware_ has joined #openstack-infra | 15:37 | |
| *** ricky1 has joined #openstack-infra | 15:37 | |
| *** Sherif has quit IRC | 15:38 | |
| jogo | sdague: multinode smoke should go away, now that it showed things work | 15:40 |
| *** earlephilhower has joined #openstack-infra | 15:40 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/puppet-jenkins: Migrate to puppet-httpd module https://review.openstack.org/198825 | 15:40 |
| anteaya | jogo: hey, did we not get rid of multinode smoke yet? | 15:40 |
| *** armax has joined #openstack-infra | 15:40 | |
| jogo | anteaya: https://review.openstack.org/#/c/196274/ | 15:41 |
| *** gyee has quit IRC | 15:41 | |
| jogo | needs a rebase (don't have time to rebase myself today) | 15:41 |
| *** Sherif has joined #openstack-infra | 15:41 | |
| anteaya | jogo: thanks I'll do it | 15:41 |
| jogo | anteaya: thank you! | 15:42 |
| anteaya | jogo: no, thank you! | 15:42 |
| anteaya | :) | 15:42 |
| anteaya | cheers dude | 15:42 |
| jogo | o/ | 15:42 |
| clarkb | fungi we rename to txt as the current convention | 15:45 |
| *** peristeri_ is now known as peristeri | 15:46 | |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/project-config: Rename .log files .log.txt in test-logs publisher https://review.openstack.org/203099 | 15:46 |
| fungi | clarkb: that ^ then | 15:46 |
| yolanda | pabelanger, still seeing some changes with include httpd , instead of include ::httpd | 15:46 |
| *** jogo has quit IRC | 15:46 | |
| fungi | clarkb: that's what i was in the process of leaning toward anyway, obviously | 15:46 |
| fungi | oh, the commit message on that needs work | 15:47 |
| *** jogo has joined #openstack-infra | 15:47 | |
| *** ildikov has joined #openstack-infra | 15:47 | |
| *** shardy has quit IRC | 15:47 | |
| *** Sherif has quit IRC | 15:48 | |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/project-config: Rename .log to .log.txt in test-results publisher https://review.openstack.org/203099 | 15:48 |
| fungi | now with new and improved commit message | 15:48 |
| *** chlong has quit IRC | 15:49 | |
| *** sergmelikyan has quit IRC | 15:49 | |
| *** btully has joined #openstack-infra | 15:50 | |
| *** gyee has joined #openstack-infra | 15:50 | |
| *** earlephilhower has quit IRC | 15:51 | |
| *** Sherif has joined #openstack-infra | 15:53 | |
| pabelanger | yolanda: ya, some slipped throught. working on them now | 15:53 |
| openstackgerrit | Anita Kuno proposed openstack-infra/project-config: Revert "Add temporary non voting multinode smoke jobs" https://review.openstack.org/196274 | 15:54 |
| *** jsavak has quit IRC | 15:54 | |
| anteaya | jogo: ^ | 15:54 |
| *** earlephilhower has joined #openstack-infra | 15:55 | |
| *** jsavak has joined #openstack-infra | 15:55 | |
| anteaya | oops | 15:56 |
| anteaya | incomplete rebase | 15:56 |
| openstackgerrit | Anita Kuno proposed openstack-infra/project-config: Revert "Add temporary non voting multinode smoke jobs" https://review.openstack.org/196274 | 15:57 |
| *** afazekas has quit IRC | 15:58 | |
| anteaya | d'oh | 15:59 |
| *** otter768 has joined #openstack-infra | 15:59 | |
| yolanda | pabelanger, ok, i think i reviewed everything | 16:00 |
| *** mestery has quit IRC | 16:01 | |
| *** jlanoux has quit IRC | 16:01 | |
| *** mestery has joined #openstack-infra | 16:01 | |
| *** akraminsky_ has quit IRC | 16:01 | |
| *** mrmartin has joined #openstack-infra | 16:02 | |
| hashar | fungi: I fixed most issues that prevents graceful stop. Will polish a patch tonight | 16:03 |
| *** hashar is now known as hasharAway | 16:03 | |
| *** devvesa has quit IRC | 16:03 | |
| *** amotoki has joined #openstack-infra | 16:03 | |
| ihrachyshka | I assume everyone is aware zuul misbehaves? | 16:04 |
| *** otter768 has quit IRC | 16:04 | |
| fungi | ihrachyshka: it does? how? | 16:04 |
| greghaynes | Are we going to have to put it in time-out? | 16:04 |
| ihrachyshka | fungi, see the check queue | 16:04 |
| ihrachyshka | all red | 16:04 |
| ihrachyshka | and jobs failing in 2mins | 16:04 |
| fungi | ihrachyshka: looks like it's working as designed | 16:04 |
| *** hdd has quit IRC | 16:04 | |
| *** eglute has quit IRC | 16:04 | |
| ihrachyshka | git I guess: https://jenkins06.openstack.org/job/gate-neutron-pep8/6859/console | 16:05 |
| fungi | ihrachyshka: those are job failures. shouldn't be zuul's fault? | 16:05 |
| *** eglute has joined #openstack-infra | 16:05 | |
| ihrachyshka | meh. ok, it's not zuul. | 16:05 |
| *** annegentle_ has quit IRC | 16:05 | |
| fungi | ihrachyshka: yes, some of those look like we had trouble with the git server farm between 14:30-15:00 while images were updating. i think most of the current failures are because of mock 1.1.4 though (the requirements change at the top of the gate should solve those once the pin propagates to projects which need it) | 16:07 |
| *** svasheka has quit IRC | 16:07 | |
| greghaynes | fungi: oh, we blow up the git farm every day when we update images now? | 16:07 |
| fungi | greghaynes: it may not be us | 16:07 |
| fungi | or at least not _only_ us | 16:08 |
| greghaynes | heh | 16:08 |
| fungi | there are likely plenty of third-party ci systems running nodepool who have copied our config, including the 14:14 utc update time | 16:08 |
| greghaynes | wow, that is sadly extremely plausible | 16:09 |
| openstackgerrit | Oleksii Zamiatin proposed openstack-infra/project-config: Make zeromq functional gate check non-voting https://review.openstack.org/202095 | 16:09 |
| openstackgerrit | Anita Kuno proposed openstack-infra/project-config: Revert "Add temporary non voting multinode smoke jobs" https://review.openstack.org/196274 | 16:09 |
| ihrachyshka | fungi, oh well, another mock hell? | 16:09 |
| fungi | ihrachyshka: https://github.com/testing-cabal/mock/issues/288 | 16:10 |
| openstackgerrit | Merged openstack/requirements: Block mock 1.1.4 https://review.openstack.org/203043 | 16:10 |
| fungi | ihrachyshka: ^ just merged, now to wait for the updates to get proposed in the next few minutes | 16:10 |
| *** mfmcdonagh has quit IRC | 16:10 | |
| *** mcdonagm__ has quit IRC | 16:10 | |
| jeblair | fungi, greghaynes, clarkb: the ip from the node in https://jenkins06.openstack.org/job/gate-neutron-pep8/6859/console does not appear in haproxy log during the time in question | 16:11 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack-dev/hacking: Updated from global requirements https://review.openstack.org/186829 | 16:11 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack-dev/pbr: Updated from global requirements https://review.openstack.org/203110 | 16:11 |
| openstackgerrit | Mehdi Abaakouk (sileht) proposed openstack-infra/project-config: Aodh: add Heat functional tests https://review.openstack.org/203111 | 16:11 |
| anteaya | jogo: that should be it | 16:11 |
| jeblair | fungi, greghaynes, clarkb: this matches the last time we looked too (a few days ago? last week?) | 16:11 |
| clarkb | ya | 16:11 |
| fungi | jeblair: neither ipv4 not ipv6? | 16:11 |
| clarkb | was early this week | 16:11 |
| fungi | s/not/nor/ | 16:11 |
| jeblair | so this isn't like the "overloaded backend" problems from before (and indeed the graphs all look okay) | 16:12 |
| jeblair | fungi: correct | 16:12 |
| *** Kiall has quit IRC | 16:12 | |
| fungi | in this case i assume it would have been trying ipv6 | 16:12 |
| *** Kiall has joined #openstack-infra | 16:13 | |
| fungi | speaking of, i've been having lots of trouble with ipv6 connectivity from home to my instances in dfw. intermittent connection timeouts | 16:13 |
| fungi | i assumed it was my isp | 16:13 |
| clarkb | me too but had lots of local packet loss so blamed that | 16:14 |
| jokke_ | dhellmann: still around? | 16:14 |
| clarkb | I have been using a lot of ssh -4 | 16:14 |
| timrc_ | jeblair, Do you see each Work Item on the Zuul v3 spec a discreet unit of work to be handled by a single developer? | 16:14 |
| *** ivar-lazzaro has joined #openstack-infra | 16:14 | |
| *** rm_work|away is now known as rm_work | 16:14 | |
| jeblair | fungi: interestingly, its ipv4 does show up once in the logs -- during a time in which it was _not_ assigned to a nodepool node. some other openstack-related effort using rax, i guess? | 16:14 |
| fungi | perhaps | 16:14 |
| fungi | nothing _obvious_ on the rackspace status page at the moment (the planned maintenances seem to be skipping today) | 16:15 |
| *** zeih has joined #openstack-infra | 16:16 | |
| *** timrc_ has quit IRC | 16:16 | |
| fungi | (and the listed disruption for cloud servers is boot-related problems in iad) | 16:16 |
| *** mrmartin has quit IRC | 16:16 | |
| jeblair | timrc_: yes, i think that's doable; more subdivision may be possible, but those seem fairly separable | 16:16 |
| * timrc decides he should probably use his irc bouncer HP is paying for :) | 16:16 | |
| timrc | jeblair, Cool. | 16:17 |
| *** ihrachyshka has quit IRC | 16:17 | |
| *** harlowja_at_home has quit IRC | 16:18 | |
| *** rguillebert has quit IRC | 16:19 | |
| *** geoffarnold has joined #openstack-infra | 16:19 | |
| *** zeih has quit IRC | 16:20 | |
| fungi | jeblair: clarkb: my issue looks like an mtu blackhole | 16:20 |
| *** tonytan4ever has quit IRC | 16:21 | |
| fungi | my tunnel interface has an mtu of 1480 but i can't send a ping with -s longer than 1432 | 16:21 |
| *** ildikov_ has joined #openstack-infra | 16:21 | |
| fungi | oh, i guess that is right though... doing the math, 48 bytes does seem to be the correct overhead | 16:23 |
| *** mrmartin has joined #openstack-infra | 16:23 | |
| *** eglute has quit IRC | 16:23 | |
| *** eglute has joined #openstack-infra | 16:24 | |
| clarkb | so there is an update toe clouds.yaml... | 16:24 |
| clarkb | I need to now check we have a new enough shade in use to understand that without a nodepool restart | 16:24 |
| *** frobware_ has quit IRC | 16:24 | |
| *** ildikov has quit IRC | 16:24 | |
| *** shashank_hegde_ has joined #openstack-infra | 16:25 | |
| *** Ala has quit IRC | 16:25 | |
| *** EricGonc_ has joined #openstack-infra | 16:26 | |
| *** asselin__ has joined #openstack-infra | 16:26 | |
| anteaya | can some kind soul take the trouble to re-enable gerrit account id 16660 please? http://paste.openstack.org/show/384099/ | 16:27 |
| jeblair | clarkb, fungi, greghaynes: http://git.openstack.org/cgit/openstack-infra/system-config/tree/modules/openstack_project/manifests/git.pp#n101 | 16:27 |
| jeblair | that's the problem ^ | 16:27 |
| clarkb | *os-client-config is the lib I need to check not shade | 16:27 |
| jeblair | Jul 17 14:44:20 localhost haproxy[60848]: 15.125.123.208:35123 [17/Jul/2015:14:43:43.568] balance_git_daemon balance_git_daemon/git05.openstack.org 3004/1/36679 29258924 -- 74/31/31/3/+3 0/0 | 16:27 |
| jeblair | notice the 31 and +3 there | 16:27 |
| *** stevemar has joined #openstack-infra | 16:27 | |
| *** asselin has quit IRC | 16:27 | |
| stevemar | getting weird behaviour with OSC patches | 16:28 |
| fungi | anteaya: 16660 is set back to active now | 16:28 |
| stevemar | everything is geting a merge conflict | 16:28 |
| stevemar | for instance: https://review.openstack.org/#/c/203143/ | 16:28 |
| anteaya | hey stevemar, when are you folks going to lunch? | 16:28 |
| *** annegentle_ has joined #openstack-infra | 16:28 | |
| * anteaya is sitting in the coffee shop called pavement | 16:28 | |
| stevemar | anteaya: waiting for sargeant adam to give us permission | 16:28 |
| anteaya | fungi: thank you | 16:28 |
| anteaya | stevemar: ha ha ha, is he waving the stick again? | 16:28 |
| stevemar | anteaya: stanek, marek and i will be joining you there, the shaking is getting much worse, we're feeling it | 16:29 |
| *** EricGonczer_ has quit IRC | 16:29 | |
| *** radez is now known as radez_g0n3 | 16:29 | |
| stevemar | anteaya: no stick waving yet | 16:29 |
| anteaya | cool, we can find space | 16:29 |
| fungi | anteaya: stevemar: quake? | 16:29 |
| anteaya | stevemar: is the shaking affecting you? I'm not imagining any of you are in danger of a panic attach | 16:29 |
| stevemar | more merge conflicts when there are no merge conflicts: https://review.openstack.org/#/c/198822/8 | 16:29 |
| anteaya | fungi: piledriver next door | 16:30 |
| stevemar | fungi: construction | 16:30 |
| stevemar | anteaya: its getting worse, even the hardiest of us is feeling it | 16:30 |
| fungi | jeblair: i guess we can up that now that we've increased the size of the lb and the backend pool | 16:30 |
| anteaya | stevemar: :( | 16:30 |
| jeblair | fungi, clarkb: i wonder why we set it at 32 to start? | 16:30 |
| stevemar | fungi: do you know what's going on with these merge conflicts? this is some weird infra mojo going on | 16:30 |
| jeblair | stevemar: looking | 16:30 |
| anteaya | stevemar: well at least you dont' think I'm crazy | 16:30 |
| fungi | jeblair: also i don't recall if that got increased after we switched from two haproxy servers to one | 16:31 |
| anteaya | so that is a reassurance | 16:31 |
| stevemar | anteaya: we're all a little crazy :) | 16:31 |
| anteaya | stevemar: true | 16:31 |
| stevemar | jeblair: thanks dude, this is beyond my infra knowledge | 16:31 |
| *** jistr has quit IRC | 16:31 | |
| fungi | stevemar: i'll need to backtrack from zuul logs. checking now | 16:31 |
| fungi | oh, nm jeblair is looking now | 16:31 |
| fungi | jeblair: did you plan to push a change to increase the maxconn or shall i? | 16:32 |
| stevemar | fungi you mean if two of you look, then it won't be solved twice as fast? :P | 16:32 |
| jeblair | fungi: yours if you want it | 16:32 |
| fungi | doing now | 16:32 |
| *** ashuk has joined #openstack-infra | 16:32 | |
| jeblair | stderr: 'fatal: ambiguous argument 'HEAD': both revision and filename | 16:33 |
| jeblair | stevemar: ^ you seem to have confused zuul's merger | 16:33 |
| *** pal has joined #openstack-infra | 16:33 | |
| *** Swami has joined #openstack-infra | 16:33 | |
| *** eglute has quit IRC | 16:33 | |
| *** tonytan4ever has joined #openstack-infra | 16:34 | |
| jeblair | https://review.openstack.org/#/c/202875/ | 16:34 |
| jeblair | stevemar: that patch killed it | 16:34 |
| *** eglute has joined #openstack-infra | 16:34 | |
| jeblair | this is sort of a zuul dos :( | 16:34 |
| *** asselin__ has quit IRC | 16:34 | |
| stevemar | jeblair: yeah, it happened around that time. | 16:34 |
| stevemar | jeblair: i figured the guy was just silly and added a bunch of junk | 16:35 |
| stevemar | jeblair: is that an actual dos? | 16:35 |
| clarkb | jeblair: it was conservative because git daemon has no connection control iirc | 16:35 |
| jeblair | stevemar: yeah, i'm pretty sure if you propose a patch with a file called HEAD to any project it'll tank it like this. so i know what i'm doing today. :) | 16:35 |
| jeblair | clarkb, fungi: we can also, now that centos6 is not as crazy, change our jobs to reset origin to https and not git | 16:36 |
| ricky1 | mordred: hiya, question about https://review.openstack.org/#/c/141016/ . When you suggested doing keystone, you meant token based auth, right? i.e. a HTTP client requesting a token against keystone, then using that token against Nodepool REST API and the backend verifying the token was valid | 16:36 |
| *** ricky1 is now known as rcarrillocruz | 16:36 | |
| stevemar | the user has been a pain for the few days he's been pushing patches - blah | 16:36 |
| rcarrillocruz | want to be sure we are on the same page | 16:36 |
| fungi | jeblair: clarkb: so breaking down that log entry, we were at actconn:74/feconn:31/beconn:31/srv_conn:3/retries:+3 | 16:36 |
| fungi | for the git tcp backend | 16:37 |
| jeblair | stevemar: i'm going to chose to appreciate the service he has provided by finding this dos for us :) | 16:37 |
| clarkb | ok os-c-c contains profile support in the release after 1.0.0 and w ehave 1.4.0 | 16:37 |
| stevemar | jeblair: thats one way of looking at it | 16:37 |
| *** dims_ has quit IRC | 16:37 | |
| jeblair | clarkb: yeah, and i think nodepool has been emitting deprecation warnings which should back that up | 16:38 |
| *** _ody has joined #openstack-infra | 16:38 | |
| clarkb | I am reenabling puppet no wwhich will pull in config drive for hpcloud dib images | 16:38 |
| clarkb | devstack-trusty is already using it and is happy now | 16:38 |
| clarkb | jeblair: stevemar fwiw we should discourage the use of HEAD as a file name as it will confuse users running commnds like git checkout HEAD | 16:40 |
| clarkb | you would have to instead do git checkout somerev -- HEAD | 16:40 |
| jeblair | clarkb: yeah, it was a messed up change from a confused user | 16:40 |
| jeblair | clarkb: so not an ongoing problem | 16:40 |
| stevemar | clarkb: we don't have a HEAD file in OSC, the user was proposing one | 16:40 |
| clarkb | jeblair: gotcha | 16:40 |
| jeblair | but zuul still needs to not explode on that :) | 16:41 |
| fungi | jeblair: clarkb: looking in the haproxy docs at what those fields represent, we were at 31 frontend/backend connections but only 3 concurrent connections still active on the server (this is what's limited by the maxconn parameter apparently) | 16:41 |
| fungi | jeblair: clarkb: and the +3 retries, according to the logs, indicates unresponsive backends and/or network issues between the proxy and the backend | 16:42 |
| fungi | er, according to the docs | 16:42 |
| stevemar | jeblair: i'm out of here, getting the hell out of this building, the same shaking that got anteaya is getting me | 16:42 |
| stevemar | thanks for looking at the merge issues | 16:42 |
| *** _ody has quit IRC | 16:42 | |
| jeblair | stevemar: okay, i'm just about done fixing it up manually for you, and i will recheck all of the -1 changes in that repo | 16:42 |
| *** stevemar has quit IRC | 16:42 | |
| anteaya | jeblair: he is coming here, I'll share backscroll, and thank you | 16:43 |
| *** Sherif has quit IRC | 16:43 | |
| clarkb | oh wow there are python3.4.3 packages for ubuntu that we can test now | 16:43 |
| fungi | clarkb: jeblair: nevermind, the frontend connections also apparently indicates whether you need to increase maxconn | 16:43 |
| *** Sherif has joined #openstack-infra | 16:43 | |
| clarkb | rough maths is 800 test slaves + a couple hundred image builds ~= 1024 max conn? | 16:44 |
| fungi | clarkb: jeblair: but anyway, it looks more like the backends are having trouble, less that maxconn is simply too low for our load. we can still try upping it and see if it helps | 16:44 |
| clarkb | I have no idea if our backends can actually handle 1024/8 = 128 connections each | 16:44 |
| fungi | i suppose we could try it and see what blows up. can't really be any worse than the current situation i guess | 16:46 |
| *** gyee has quit IRC | 16:46 | |
| clarkb | The other idea I had was to just host the files on the baremetal node | 16:47 |
| clarkb | its basically idle doing the load balancing | 16:47 |
| clarkb | but that is more work and a gerrit restart | 16:47 |
| jeblair | anteaya: i mass-rechecked those changes (with the gertty mass review feature) | 16:47 |
| anteaya | jeblair: good thing gertty has that feature! | 16:47 |
| anteaya | and thank you | 16:47 |
| dhellmann | apparently I missed all the fun with mock this time | 16:47 |
| anteaya | I'll tell stevemar | 16:47 |
| anteaya | dhellmann: I predict there will be more, at some point | 16:48 |
| *** s0nea has quit IRC | 16:48 | |
| fungi | clarkb: jeblair: oh, also docs say that actconn should stay under half of the system's max file handles since it needs two file descriptors per connection | 16:48 |
| jeblair | clarkb: it's not a baremetal node, it's a large vm | 16:48 |
| dhellmann | anteaya: that seems likely, until the backports settle down | 16:48 |
| anteaya | dhellmann: we can hope | 16:48 |
| anteaya | about the settle down part | 16:48 |
| clarkb | jeblair: oh, I thought the plan was to us ea baremetal node? | 16:49 |
| *** cdent has quit IRC | 16:49 | |
| *** esmute has joined #openstack-infra | 16:49 | |
| *** cdent has joined #openstack-infra | 16:50 | |
| *** unicell1 has quit IRC | 16:50 | |
| dhellmann | anteaya: we'll get there | 16:50 |
| anteaya | dhellmann: :) | 16:50 |
| jeblair | fungi: why do you think the backends are having trouble -- it seems like if the frontends are at their maxconn, doesn't that seem to be the most likely problem? | 16:50 |
| clarkb | in that case nevermind, I doubt that a large VM is significantly better than many smaller VMs for this workload | 16:50 |
| anteaya | dhellmann: I picture you with a long flowing robe and a shepherd's crook | 16:50 |
| *** esmute has quit IRC | 16:50 | |
| *** s0nea has joined #openstack-infra | 16:50 | |
| fungi | clarkb: jeblair: looks like soft max is 1024 and hard max is 4096 according to ulimit but we can up that if we think it's warranted | 16:51 |
| anteaya | there is a meadow and some trees in the picture too | 16:51 |
| anteaya | dhellmann: it is kind of nice | 16:51 |
| jeblair | clarkb: er, there was a difficulty in using it... something with the boot volume i think... and this vm had the needed specs anyway | 16:51 |
| *** esmute has joined #openstack-infra | 16:51 | |
| fungi | jeblair: well, you pointed to the +3 retries, which is apparently entirely whether haproxy was able to connect to a backend | 16:51 |
| * dhellmann looks down at the t-shirt and jeans he's wearing | 16:51 | |
| anteaya | dhellmann: are they flowing | 16:52 |
| anteaya | is there a meadow about? | 16:52 |
| *** sigmavirus24 has quit IRC | 16:52 | |
| fungi | jeblair: though i agree the maxconn also looks reached, it seems from the docs like that's independent of the potential cause for the retry count being nonzero | 16:52 |
| *** hrubi has quit IRC | 16:52 | |
| *** Trozz has quit IRC | 16:52 | |
| *** sigmavirus24 has joined #openstack-infra | 16:52 | |
| jeblair | fungi, clarkb: how many connections does one git server support on a backend? | 16:53 |
| fungi | jeblair: clarkb: manpage says it defaults to 32 | 16:55 |
| *** madhu_ak has joined #openstack-infra | 16:55 | |
| fungi | doesn't look like we're setting --max-connections explicitly on the command line to override that | 16:55 |
| clarkb | you mean git daemon? | 16:55 |
| fungi | yeah | 16:55 |
| jeblair | fungi, clarkb: i don't understand why feconn is 31 _and_ beconn is 31 for all of the various backends | 16:55 |
| jeblair | clarkb: yeah | 16:55 |
| clarkb | fungi: I thought it was bound by failure to fork | 16:56 |
| jeblair | shouldn't feconn be 31 and beconn be, on average, 31 divided by 8? | 16:56 |
| fungi | beconn, per the docs, includes queued connections too apparently | 16:56 |
| clarkb | fungi: since we don't run it as a persistent service instead xinetd forks them | 16:56 |
| fungi | "total number of concurrent connections active on servers as well as the number of connections pending in queues" | 16:57 |
| fungi | clarkb: oh! i missed that we had xinetd forking those off | 16:57 |
| openstackgerrit | Paul Belanger proposed openstack-infra/puppet-jenkins: Add Gemfile and puppet 4 checks https://review.openstack.org/203164 | 16:57 |
| clarkb | fungi: pretty sure it does | 16:57 |
| *** sigmavirus24 has quit IRC | 16:57 | |
| clarkb | let me double check now that I have asserted that | 16:58 |
| fungi | yep, it is | 16:58 |
| fungi | er, wait | 16:58 |
| *** sigmavirus24 has joined #openstack-infra | 16:58 | |
| fungi | disable = yes | 16:58 |
| clarkb | there is an /etc/xinetd.d/git | 16:58 |
| freerunner | fungi: Hi! Can u help me a little bit? How I can make a job non-voting for a specific branches? | 16:58 |
| clarkb | but disable = yes | 16:58 |
| jeblair | okay, that explains why beconn is > 4, but why is it also exactly 31? | 16:58 |
| fungi | clarkb: chkconfig says the daemon init is enabled | 16:59 |
| fungi | git-daemon 0:off 1:off 2:on 3:on 4:on 5:on 6:off | 16:59 |
| *** Sherif has quit IRC | 16:59 | |
| fungi | clarkb: so i think we're supporting max 32 connections per backend server for that | 16:59 |
| clarkb | ok so it would be 32 per server yup | 16:59 |
| clarkb | jeblair: is it possible that puppet is configuring it that way for some reason? | 17:00 |
| jeblair | hrm. feconn==beconn always, even when not under load | 17:00 |
| clarkb | maybe it takes the fe value and copies to be? | 17:00 |
| fungi | freerunner: you need different job names to do that. zuul doesn't currently have a mechanism to make one job voting on some branches and nonvoting on others. it simply knows whether a job should be voting and on which branches' changes the job should trigger | 17:00 |
| openstackgerrit | Chris Jones proposed openstack-infra/glean: Support explicitly configured DHCP interfaces https://review.openstack.org/202250 | 17:00 |
| clarkb | oh wait this isn't querying config its querying active conns nevermind | 17:01 |
| *** derekh has quit IRC | 17:01 | |
| openstackgerrit | Nathaniel Dillon proposed openstack-infra/irc-meetings: Adding Security Guide Meeting https://review.openstack.org/200755 | 17:02 |
| *** radez_g0n3 is now known as radez | 17:02 | |
| clarkb | fungi: so maconn bump to 512 to start | 17:02 |
| clarkb | fungi: then we can see if we ned to bump backend git daemon connection counts | 17:03 |
| fungi | btw, using section 8.2.2 at http://www.haproxy.org/download/1.5/doc/configuration.txt to attempt to grok what those values actually mean | 17:03 |
| fungi | clarkb: and yeah, i think we can't go higher than 512 without a ulimit tweak on git.o.o anyway | 17:03 |
| *** patrickeast has joined #openstack-infra | 17:04 | |
| fungi | since the soft file handle max for non-root is 1024 right now | 17:04 |
| *** jamesmcarthur has quit IRC | 17:04 | |
| jeblair | http://permalink.gmane.org/gmane.comp.web.haproxy/2160 | 17:04 |
| jeblair | i'm reading this explanation of the log values right now | 17:04 |
| *** _nadya_ has joined #openstack-infra | 17:07 | |
| freerunner | fungi: Thanks a lot! | 17:07 |
| clarkb | puppet is enabled on nodepool again fwiw | 17:08 |
| jeblair | oh i was confusing beconn and srv_conn | 17:08 |
| jeblair | srv_conn is in fact around 3 | 17:08 |
| fungi | yeah, that's what i was saying before. beconn also includes queued connections | 17:08 |
| *** pvaneck has joined #openstack-infra | 17:08 | |
| *** jerryz has joined #openstack-infra | 17:09 | |
| *** jamesmcarthur has joined #openstack-infra | 17:09 | |
| *** unicell has joined #openstack-infra | 17:10 | |
| jeblair | reading remedial haproxy documentation, a backend is a _set_ of servers, and a frontend is set of ip/port combos that forward to backend(s) | 17:10 |
| jeblair | so i think the way our config is written, we define both frontends and backends together | 17:11 |
| jeblair | in a 1:1 mapping | 17:11 |
| *** yamahata has quit IRC | 17:11 | |
| jeblair | so that explains why feconn==beconn; they are fairly similar; and the only time feconn != beconn would be when we have queued connections | 17:11 |
| *** _nadya_ has quit IRC | 17:12 | |
| asselin_ | fungi, it would be nice to land this change today: https://review.openstack.org/#/c/199780/6 | 17:12 |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/system-config: Increase git haproxy maxconn from 32 to 256 https://review.openstack.org/203165 | 17:12 |
| fungi | clarkb: jeblair: ^ if we want to try it | 17:12 |
| fungi | if we want to go higher than 256, i think we either need to set --max-connections for git-daemon or add more backend servers | 17:13 |
| *** shashank_hegde_ has quit IRC | 17:13 | |
| fungi | and if we want to go over 512 i think we also need to adjust max file handles via ulimit on git.o.o | 17:13 |
| jeblair | we can also set maxconn on the server in haproxy | 17:13 |
| *** Volundr is now known as whoops | 17:13 | |
| jeblair | or do that instead of setting --max-connections on git-daemon | 17:14 |
| clarkb | gmann_: http://logs.openstack.org/91/202391/5/check/gate-tempest-dsvm-multinode-full/f838b41/logs/subnode-2/screen-n-cpu.txt.gz#_2015-07-17_04_10_29_375 is the issue, and seems that tempest doesn't fail at that point instead it sees that the host doesnt change then fails | 17:16 |
| jeblair | fungi: is git-daemon working on git01? | 17:16 |
| clarkb | gmann_: we should have /etc/hosts setup to add those names so I iwll need to debug that | 17:16 |
| jeblair | fungi: i don't see any log entries for it, and that may explain the server connection failures/redispatches | 17:17 |
| fungi | jeblair: well, it's running. i'm testing it now | 17:17 |
| openstackgerrit | Merged openstack-infra/irc-meetings: Change DefCore Meeting Time https://review.openstack.org/203089 | 17:17 |
| jeblair | yeah, there are zero log entries for it | 17:17 |
| jeblair | (for git:// only, http is working) | 17:18 |
| fungi | i can clone from it | 17:18 |
| fungi | git clone git://git01.openstack.org:29418/openstack-infra/devstack-gate.git | 17:18 |
| pabelanger | When we build dibs on nodepool.o.o are we using TMPFS or local HDD? | 17:18 |
| fungi | works for me | 17:18 |
| jeblair | that works from git.o.o too | 17:18 |
| clarkb | pabelanger: local hdd, we don't have enough ram for tmpfs | 17:18 |
| pabelanger | clarkb: 4gb? | 17:19 |
| clarkb | pabelanger: the image is almost 16GB now | 17:19 |
| fungi | pabelanger: the root filesystem | 17:19 |
| *** mrunge has quit IRC | 17:19 | |
| jeblair | git clone git://198.61.223.164:29418/openstack-infra/devstack-gate.git | 17:19 |
| jeblair | that does not work ^ | 17:19 |
| pabelanger | clarkb: fungi any spec configuration to disable it? | 17:19 |
| pabelanger | special* | 17:20 |
| clarkb | pabelanger: iirc nodepool says no tmpfs | 17:20 |
| pabelanger | For my local nodepool, is I use the nodepool image-update command, it will do tmpfs. Even thought --no-tmpfs is passed | 17:20 |
| clarkb | pabelanger: that sounds like a dib bug then | 17:20 |
| *** pal has quit IRC | 17:21 | |
| pabelanger | I had to explicitly set DIB_NO_TMPFS=1 before calling nodepool | 17:21 |
| fungi | jeblair: oh! i can clone from it via ipv6 but not ipv4 | 17:21 |
| pabelanger | clarkb: ya, will check it into it. Wanted to check upstream first | 17:21 |
| clarkb | huh did it stop listening on ipv4 socket? | 17:22 |
| *** zeih has joined #openstack-infra | 17:22 | |
| *** zeih has quit IRC | 17:22 | |
| fungi | jeblair: it claims to be listening on :::29418 but on centos 6 that apparently doesn't also bind to v4 addresses | 17:22 |
| fungi | the other git servers also mention 0.0.0.0:29418 | 17:22 |
| fungi | this is...odd :/ | 17:23 |
| *** _nadya_ has joined #openstack-infra | 17:23 | |
| * fungi despises the IPV6_V6ONLY option in AF_INET6 | 17:24 | |
| *** otter768 has joined #openstack-infra | 17:24 | |
| jeblair | anyone else think we're at the point of 'service git-daemon restart'? | 17:25 |
| fungi | yeah, for now | 17:25 |
| fungi | i'll do it | 17:25 |
| jeblair | kk | 17:25 |
| openstackgerrit | Victor Ryzhenkin proposed openstack-infra/project-config: Make job gate-murano*-requirements use job template https://review.openstack.org/203177 | 17:25 |
| *** harlowja has quit IRC | 17:26 | |
| *** Sherif has joined #openstack-infra | 17:26 | |
| *** harlowja has joined #openstack-infra | 17:26 | |
| *** tonytan4ever has quit IRC | 17:26 | |
| fungi | now 0.0.0.0:29418 is listening too | 17:26 |
| *** ashuk has quit IRC | 17:26 | |
| jeblair | i can clone from the ipv4 addr now | 17:26 |
| *** AJaeger has left #openstack-infra | 17:27 | |
| jeblair | okay, i think this explains all the mysteries in the log file! we've explained the server retries, and we know we're getting the connection resets because of the low fe maxconn | 17:27 |
| clarkb | the logstash job queue is mysteriously flat | 17:27 |
| *** teran has quit IRC | 17:27 | |
| *** vponomaryov has quit IRC | 17:27 | |
| *** AJaeger has joined #openstack-infra | 17:27 | |
| fungi | clarkb: yeah, someone else pointed that out earlier but i haven't looked into it yet | 17:28 |
| clarkb | the client is not running | 17:28 |
| clarkb | which starts the server | 17:28 |
| fungi | net.ipv6.bindv6only = 0 on our git servers, so git-daemon must be to blame for using IPV6_V6ONLY | 17:28 |
| jeblair | fungi, clarkb: apprved haproxy reconfig | 17:29 |
| jeblair | clarkb: i approved a bunch of changes that touched that yesterday | 17:29 |
| jeblair | clarkb: http://git.openstack.org/cgit/openstack-infra/puppet-log_processor/ | 17:31 |
| jeblair | clarkb: apparently that was the day before yesterday :/ | 17:31 |
| openstackgerrit | Merged openstack-infra/project-config: Add validation job for releases repository https://review.openstack.org/202785 | 17:32 |
| *** amotoki has quit IRC | 17:33 | |
| clarkb | jeblair: I think the config change for subnode logs triggered a restart which has failed for some reason | 17:33 |
| clarkb | but the day before last changes did not trigere a restart | 17:33 |
| jeblair | ah | 17:33 |
| openstackgerrit | Merged openstack-infra/project-config: Don't run dsvm tests on unittest only changes https://review.openstack.org/202898 | 17:33 |
| pabelanger | Was curious if +W would be giving to 2 diskimage-builder reviews: https://review.openstack.org/#/c/185841/5 and https://review.openstack.org/#/c/185842/ | 17:33 |
| pabelanger | For fedora 22 stuff I'm helping ianw with | 17:33 |
| *** shashank_hegde_ has joined #openstack-infra | 17:34 | |
| *** alexsyip has joined #openstack-infra | 17:34 | |
| pabelanger | devananda: Clint ^ thoughts | 17:34 |
| *** mestery has quit IRC | 17:35 | |
| clarkb | pabelanger: you may hav ebetter luck in #tripleo | 17:35 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Add Ceph DevStack plugin to Stackforge https://review.openstack.org/188768 | 17:35 |
| pabelanger | clarkb: ya, just join up there | 17:35 |
| *** dougwig has quit IRC | 17:35 | |
| *** yamahata has joined #openstack-infra | 17:35 | |
| *** achanda has joined #openstack-infra | 17:35 | |
| *** eglute has quit IRC | 17:35 | |
| *** eglute has joined #openstack-infra | 17:36 | |
| Clint | pabelanger: SpamapS? | 17:36 |
| *** mestery has joined #openstack-infra | 17:36 | |
| pabelanger | Clint: Ya, my bad. Too many Clints kicking around :) | 17:37 |
| *** radez is now known as radez_g0n3 | 17:37 | |
| *** dougwig has joined #openstack-infra | 17:37 | |
| clarkb | jeblair: if I run it in the foreground it runs fine | 17:38 |
| openstackgerrit | Boris Pavlovic proposed openstack-infra/project-config: Add python jobs for fuel-web https://review.openstack.org/202892 | 17:39 |
| *** radez_g0n3 is now known as radez | 17:40 | |
| SpamapS | Clint: eh? | 17:41 |
| *** dizquierdo has quit IRC | 17:41 | |
| *** Sherif has quit IRC | 17:43 | |
| Clint | SpamapS: i think pabelanger was trying to get your thoughts | 17:43 |
| *** Sukhdev has joined #openstack-infra | 17:43 | |
| *** e0ne has quit IRC | 17:44 | |
| *** boris-42 has joined #openstack-infra | 17:44 | |
| *** esmute has quit IRC | 17:44 | |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/system-config: Stop DIB deleting our dpkg cache https://review.openstack.org/203187 | 17:45 |
| fungi | greghaynes: mordred: clarkb: jeblair: i have no idea how none of us noticed this ^ behavior before now | 17:45 |
| SpamapS | Clint: he got 'em :) | 17:45 |
| fungi | greghaynes: mordred: clarkb: jeblair: unfortunately i'm still trying to unwind the equivalent yum behavior since it seems to lack an override | 17:45 |
| Clint | my job here is done | 17:45 |
| pabelanger | Ya, moved into #tripleo as clarkb suggested | 17:46 |
| *** notnownikki has quit IRC | 17:47 | |
| *** Sherif has joined #openstack-infra | 17:48 | |
| openstackgerrit | Boris Pavlovic proposed openstack-infra/project-config: Add python jobs for fuel-web https://review.openstack.org/202892 | 17:50 |
| *** lykinsbd has joined #openstack-infra | 17:53 | |
| mriedem | clarkb: fungi: jeblair: sdague: seen this? http://graphite.openstack.org/render/?from=-24hours&height=360&until=now&width=668&bgcolor=ffffff&fgcolor=000000&target=color(alias(stats.gauges.logstash.geard.queue.running,%20%27Running%27),%20%27blue%27)&target=color(alias(stats.gauges.logstash.geard.queue.waiting,%20%27Waiting%27),%20%27red%27)&target=color(alias(stats.gauges.logstash.geard.queue.total,%20%27Total%20Jobs%27),%20%27888888%27 | 17:54 |
| openstackgerrit | Mike Perez proposed openstack/requirements: Bumping Cinder client to 1.3.1 https://review.openstack.org/199348 | 17:55 |
| clarkb | mriedem: yes | 17:56 |
| fungi | greghaynes: looking at the yum element, it seems that it temporarily overrides the package cache to be /tmp/yum while installing things, and sets keepcache=0 when its done. any idea for the cleanest way to refactor that so we can make it a toggleable option like with the dpkg element? | 17:56 |
| *** teran has joined #openstack-infra | 17:57 | |
| clarkb | jeblair: the wait fo rname resolution code seems to be breaking but I don't see why yet | 17:59 |
| *** Sherif has quit IRC | 17:59 | |
| openstackgerrit | Madhusudhan Kandadai proposed openstack-infra/project-config: New jobs for neutron-lbaasv2 scenario tests https://review.openstack.org/199754 | 18:00 |
| *** hasharAway has quit IRC | 18:00 | |
| *** teran has quit IRC | 18:01 | |
| *** madhu_ak has quit IRC | 18:01 | |
| *** madhu_ak has joined #openstack-infra | 18:02 | |
| *** puranamr has joined #openstack-infra | 18:03 | |
| *** tonytan4ever has joined #openstack-infra | 18:04 | |
| *** Kennan2 has joined #openstack-infra | 18:04 | |
| fungi | ianw: when you're around (i know it's weekend time there) you might also be interested in noodling on how to make the yum element in dib less package-cache-hostile | 18:05 |
| *** Kennan has quit IRC | 18:05 | |
| *** Sherif has joined #openstack-infra | 18:05 | |
| openstackgerrit | Madhusudhan Kandadai proposed openstack-infra/project-config: New jobs for neutron-lbaasv2 scenario tests https://review.openstack.org/199754 | 18:05 |
| *** SumitNaiksatam has joined #openstack-infra | 18:06 | |
| clarkb | the socket.getaddrinfo() is causing it to just die, I think I have done what I need to to catch and log all exceptions thrown by it and get nothing | 18:06 |
| clarkb | running it interactively works fine | 18:07 |
| clarkb | and running in the foreground works fine | 18:07 |
| *** Sherif has quit IRC | 18:07 | |
| *** tqtran has joined #openstack-infra | 18:08 | |
| madhu_ak | hi, may I know the nick name for Andreas Jaeger | 18:08 |
| clarkb | madhu_ak: AJaeger | 18:08 |
| madhu_ak | AJaeger: thanks for your comments. Just updated them and pushed a new patch. | 18:08 |
| *** JayF has quit IRC | 18:09 | |
| madhu_ak | clarkb: thanks for the input | 18:09 |
| *** bharath has joined #openstack-infra | 18:09 | |
| *** JayF has joined #openstack-infra | 18:09 | |
| *** Sherif has joined #openstack-infra | 18:09 | |
| *** mestery has quit IRC | 18:10 | |
| clarkb | gah I see the bug | 18:10 |
| *** dannywilson has joined #openstack-infra | 18:10 | |
| clarkb | :/ will fix by hand and push a patch shortly | 18:10 |
| *** btully has quit IRC | 18:11 | |
| openstackgerrit | Matt Riedemann proposed openstack/requirements: Bump minimum os-brick>=0.3.0 https://review.openstack.org/203193 | 18:11 |
| madhu_ak | AJaeger: have a question about experimental jobs | 18:13 |
| madhu_ak | or anyone in this channel | 18:14 |
| madhu_ak | if I want to run the experimental jobs, what should I do | 18:14 |
| *** esmute has joined #openstack-infra | 18:14 | |
| madhu_ak | the above job is related to https://review.openstack.org/199754 | 18:14 |
| openstackgerrit | Clark Boylan proposed openstack-infra/puppet-log_processor: Import socket so we can use it to get name info https://review.openstack.org/203196 | 18:15 |
| clarkb | jeblair: ^ | 18:15 |
| *** hdd has joined #openstack-infra | 18:15 | |
| *** aarefiev has quit IRC | 18:16 | |
| *** puranamr has quit IRC | 18:16 | |
| clarkb | thats manually applied right now with puppet disabled | 18:16 |
| clarkb | mriedem: ^ should be working again | 18:17 |
| mriedem | clarkb: nice, i see the flatline is no more, thanks! | 18:18 |
| *** EricGonc_ has quit IRC | 18:21 | |
| clarkb | madhu_ak: to run experimental jobs yo uleave a comment of 'check experimental' | 18:21 |
| clarkb | madhu_ak: so once that chnage merges you can run them that way | 18:21 |
| madhu_ak | clarkb: so I need to wait till that change merges | 18:22 |
| clarkb | yes | 18:22 |
| *** EricGonczer_ has joined #openstack-infra | 18:22 | |
| madhu_ak | hmm, I see. thanks for the info | 18:22 |
| *** puranamr has joined #openstack-infra | 18:25 | |
| clarkb | I have held 3775012 in order to debug the name reoslution errors during the multinode tests on hpcloud | 18:26 |
| *** e0ne has joined #openstack-infra | 18:29 | |
| *** puranamr has quit IRC | 18:29 | |
| *** puranamr has joined #openstack-infra | 18:30 | |
| sdague | fungi: so g-r doesn't block unit tests nodes | 18:30 |
| sdague | how do we prevent mock 1.1.4 from being used in nova unit tests - https://jenkins07.openstack.org/job/gate-nova-python27/12531/console | 18:30 |
| *** puranamr has quit IRC | 18:30 | |
| *** mestery has joined #openstack-infra | 18:31 | |
| fungi | sdague: with https://review.openstack.org/200708 ? | 18:31 |
| sdague | fungi: ah | 18:31 |
| *** jsavak has quit IRC | 18:31 | |
| *** larainema has quit IRC | 18:32 | |
| *** larainema has joined #openstack-infra | 18:32 | |
| *** jsavak has joined #openstack-infra | 18:32 | |
| *** puranamr has joined #openstack-infra | 18:32 | |
| *** markvoelker has quit IRC | 18:32 | |
| fungi | sdague: however, once the tox-using-constraints work gets further, there is a potential to have constraints do that (waves hands in air) | 18:32 |
| sdague | fungi: any chance we could promote that, there is a ton of nova code in the gate queue that is going to tumble fail and gum up everything | 18:33 |
| fungi | yeah, that seems like a waste of good nodes. doing | 18:33 |
| sdague | it looks like cinder is hitting similar things | 18:33 |
| sdague | I don't know if they've got a fix yet | 18:34 |
| *** puranamr has quit IRC | 18:35 | |
| *** annegentle_ has quit IRC | 18:35 | |
| *** annegentle_ has joined #openstack-infra | 18:36 | |
| *** linkedinyou has joined #openstack-infra | 18:37 | |
| *** unicell1 has joined #openstack-infra | 18:38 | |
| *** unicell has quit IRC | 18:38 | |
| sc68cal | sdague: we need to get Sam-I-Am to do diagrams for multinode at the gate | 18:39 |
| sc68cal | sdague: all these bridges | 18:39 |
| *** aviau has quit IRC | 18:40 | |
| clarkb | sc68cal: there are diagrams | 18:40 |
| clarkb | sc68cal: are the existing diagrams not good enough? | 18:40 |
| *** esmute has quit IRC | 18:40 | |
| *** aviau has joined #openstack-infra | 18:41 | |
| *** heyongli has quit IRC | 18:41 | |
| * Sam-I-Am perks up | 18:41 | |
| *** heyongli has joined #openstack-infra | 18:42 | |
| *** puranamr has joined #openstack-infra | 18:43 | |
| *** HeOS has quit IRC | 18:43 | |
| reed | mordred, I'm almost there :) DreamHost cloud doesn't assign a keypair to a new instance by default. How do I specify a keypair with shade? | 18:43 |
| *** puranamr has quit IRC | 18:44 | |
| *** sergmelikyan has joined #openstack-infra | 18:44 | |
| mordred | reed: no cloud does ... paramer to create_server is 'key_name' | 18:45 |
| fungi | reed: btw i like (or rather sympathize!) with your blog post today | 18:45 |
| *** gordc has joined #openstack-infra | 18:47 | |
| clarkb | sc68cal: http://git.openstack.org/cgit/openstack-infra/devstack-gate/tree/multinode_diagram.txt does that help? | 18:47 |
| reed | mordred, thanks... that no cloud does it is kinda silly, isn't it? What use is an instance where I cannot login? | 18:48 |
| clarkb | sc68cal: Sam-I-Am we treat the primary node attached to jenkins as the hub for the usbnode spokes, then they are all bridged together there via gre | 18:48 |
| reed | fungi, thanks, it was a revelation to me | 18:48 |
| mordred | reed: tell. me. about. it | 18:48 |
| clarkb | for neutron we only need one l2 network overlay because neutron sets up the internal VM netowrk overlay itself. For nova net we have to set both internal VM net and "public" net | 18:49 |
| clarkb | reed: well some cloud sdont even give you network by default | 18:49 |
| reed | the best part is realizing I'm not the only one realizing this is way too complicated :) | 18:49 |
| clarkb | so first you hvae to set up networking, then you can worry about ssh | 18:49 |
| *** ildikov_ has quit IRC | 18:49 | |
| reed | for a couple of days, trying to follow the faafo tutorial I felt like a dork | 18:49 |
| reed | I thought it was me :) | 18:50 |
| *** puranamr has joined #openstack-infra | 18:50 | |
| sc68cal | clarkb: ah that's helpful | 18:50 |
| mriedem | sdague: fungi: it still fails | 18:50 |
| mriedem | mock 1.1.4 that is | 18:51 |
| mriedem | https://jenkins02.openstack.org/job/gate-nova-python27/1136/console | 18:51 |
| mriedem | something else must be pulling it in which isn't synced yet | 18:51 |
| mriedem | oslo.versoinedobjects mayhap? | 18:51 |
| sc68cal | clarkb: sdague: I guess it's pedantic but http://git.openstack.org/cgit/openstack-infra/devstack-gate/tree/multinode_diagram.txt#n35 - should't it be named br-tun ? | 18:51 |
| *** mrmartin has quit IRC | 18:52 | |
| sdague | mriedem: so, you could do the bazooka approach that we did with sphinx | 18:52 |
| *** Somay has joined #openstack-infra | 18:52 | |
| sdague | I don't actually get why the pins no longer work | 18:52 |
| mriedem | superdan: nice https://review.openstack.org/#/c/203131/ - we'll need a release of o.vo with that probably | 18:52 |
| mriedem | dimtruck: ^ | 18:52 |
| clarkb | sc68cal: nope, we don't touch br-tun at all | 18:52 |
| clarkb | sc68cal: neutron does that all by itself | 18:52 |
| mriedem | sdague: yeah.... | 18:52 |
| *** esmute has joined #openstack-infra | 18:52 | |
| sdague | it's actually incredibly frustrating that we're going to lose a whole day of development because of a mock release | 18:52 |
| sdague | so, I kind of feel like if the whole pinning mechanism is broken, we should stop bothering, and just override in tox.ini directly | 18:54 |
| sdague | anyway, it's effectively end of day for me, so I'm dropping off shortly | 18:54 |
| *** sergmelikyan has quit IRC | 18:54 | |
| *** otter768 has quit IRC | 18:54 | |
| mriedem | sdague: yeah, just let me do that | 18:54 |
| mriedem | this is the g-r sync that bumps min pbr which allows us to remove the sphinx bazooka too | 18:55 |
| clarkb | sdague: yes that is why I hav ealways argued against pins | 18:55 |
| clarkb | they don't work | 18:55 |
| *** sergmelikyan has joined #openstack-infra | 18:55 | |
| sdague | clarkb: != used to work | 18:56 |
| *** sergmelikyan has quit IRC | 18:56 | |
| clarkb | they never worked | 18:56 |
| fungi | depends on whether it's early enough in the requirements list though, right? | 18:56 |
| clarkb | the way pip does/did dep resolution made them not work properly | 18:56 |
| clarkb | fungi: and if its top level or not | 18:56 |
| fungi | right, that too | 18:56 |
| clarkb | and if you do two passes with pip or one | 18:56 |
| *** _nadya_ has quit IRC | 18:56 | |
| fungi | oh, for sure a second pass with pip is trouble | 18:57 |
| *** heyongli has quit IRC | 18:57 | |
| *** _nadya_ has joined #openstack-infra | 18:57 | |
| *** Sherif has quit IRC | 18:57 | |
| clarkb | and if easy_install is triggered | 18:58 |
| reed | bugs for shade go to ... storyboard? | 18:58 |
| *** heyongli has joined #openstack-infra | 18:58 | |
| reed | although I think I found a bug at dreamhost, actually | 18:58 |
| fungi | reed: for the moment, yes | 18:58 |
| reed | cool, thanks fungi | 18:58 |
| fungi | also entirely likely ;) | 18:58 |
| reed | ehehe :) | 18:59 |
| reed | when I specify auto_ip = True in shade, am I requesting a floating IP? | 18:59 |
| reed | I'm trying to understand why I get "OpenStackCloudException: unable to bind a floating ip to server 118933fa-1af0-46f1-8a85-02d4f1233e47: Bad floatingip request: Port 2c2615f3-7bba-4068-8ed0-a2b3dcc20f7f has multiple fixed IPs. Must provide a specific IP when assigning a floating IP" | 18:59 |
| *** bknudson has quit IRC | 19:00 | |
| reed | dreamhost assigns a public IPv6 by default, and a private IP, too | 19:00 |
| Shrews | reed: there's a fix for that. upgrade shade | 19:00 |
| *** gyee has joined #openstack-infra | 19:00 | |
| reed | Shrews, I installed it yesterday... new version? | 19:00 |
| Shrews | reed: you need 0.11.0 | 19:01 |
| fungi | fast-moving clouds? | 19:01 |
| Shrews | fungi: hurricane force | 19:01 |
| fungi | reed: yes, tagged 20 hours ago | 19:01 |
| *** _nadya_ has quit IRC | 19:01 | |
| *** mrmartin has joined #openstack-infra | 19:01 | |
| fungi | reed: so maybe right after you downloaded? | 19:01 |
| reed | oh... I had 0.10.0 | 19:01 |
| reed | wow | 19:02 |
| reed | :) | 19:02 |
| *** adreznec has quit IRC | 19:02 | |
| fungi | reed: it's a bit more disappointing to keep in mind that every line of code in shade basically represents another interoperability bug we're papering over so we can use these providers | 19:03 |
| *** shashank_hegde_ has quit IRC | 19:03 | |
| fungi | which means its rate of development has been disappointingly fast | 19:03 |
| *** stevemar has joined #openstack-infra | 19:04 | |
| *** heyongli has quit IRC | 19:04 | |
| * reed adds cron.hourly job :) | 19:04 | |
| openstackgerrit | Merged openstack-infra/project-config: Remove neutron smoke job from integrated gate templates https://review.openstack.org/203048 | 19:04 |
| anteaya | AJaeger: thank you | 19:05 |
| *** heyongli has joined #openstack-infra | 19:05 | |
| mordred | reed: I'm out of town next week - Shrews is your go-to guy for shade questions | 19:05 |
| reed | mordred, me too, I'll be at OSCON | 19:06 |
| reed | Shrews, I get the same error as before with the IP stuff, even after I upgrade shade | 19:06 |
| reed | Shrews, the server is created alright though | 19:06 |
| Shrews | reed: well that's disappointing. dguerri ^^^^^ | 19:07 |
| openstackgerrit | Swaminathan Vasudevan proposed openstack-infra/project-config: Make neutron-dvr job voting at the gate https://review.openstack.org/180230 | 19:07 |
| *** cdent has quit IRC | 19:07 | |
| * mrmartin very excited about app dev experience of reed | 19:07 | |
| *** adreznec has joined #openstack-infra | 19:07 | |
| fungi | sdague: mriedem: in py27-1.log mock==1.1.3 is downloaded, respecting the pin. then in py27-2.log you can see oslo.versionedobjects upgrades gets 1.1.4 in a second pass | 19:08 |
| mrmartin | in da cloud | 19:08 |
| reed | Shrews, it may be caused by dreamhost setup... do you have an account on DH to test shade? I can give you one | 19:08 |
| *** shashank_hegde has quit IRC | 19:08 | |
| Shrews | reed: i do not have one | 19:08 |
| mriedem | fungi: yup, o.vo is what i expected b/c that pulls it in via runtime requirements, which is what broke us a week ago | 19:08 |
| reed | Shrews, at least that's what my colleagues first said when they saw that error | 19:08 |
| mrmartin | reed, what is not working? | 19:09 |
| *** esmute has quit IRC | 19:09 | |
| anteaya | who feels like helping a jogo patch across the finish line? it removes jobs: https://review.openstack.org/#/c/196274/ | 19:10 |
| reed | mrmartin, all is getting better with shade :) | 19:10 |
| mrmartin | yeap, mordred had a nice presentation about shade in Vancouver | 19:11 |
| *** stevemar has quit IRC | 19:12 | |
| *** btully has joined #openstack-infra | 19:13 | |
| reed | mrmartin, he has proposed one for Tokyo too, let's vote that one :) | 19:14 |
| clarkb | fungi so teo pip runs ya? | 19:14 |
| clarkb | this is why we shouldnt let pbr install requirements | 19:14 |
| clarkb | its not redundant | 19:14 |
| *** dangers_away is now known as dangers | 19:15 | |
| dguerri | hey reed, how are you doing? :) | 19:15 |
| mordred | reed: can you comp my dh account? I'm currently just paying for one to test with | 19:15 |
| reed | dguerri, good! glad to see you back on OpenStack :) | 19:15 |
| fungi | clarkb: well, part of this may be because nova's tox.ini has install_command = pip install -U --force-reinstall {opts} {packages} | 19:15 |
| fungi | i'm still playing around with it to see | 19:16 |
| dguerri | reed: I hope the IP stuff didn't cause you too much trouble :P .. reading the scrollback | 19:16 |
| reed | dguerri, no, things are a lot better, I can't really complain :) | 19:17 |
| *** btully has quit IRC | 19:17 | |
| *** dims_ has joined #openstack-infra | 19:18 | |
| * reed heading to lunch, ttyl | 19:18 | |
| *** afazekas has joined #openstack-infra | 19:19 | |
| *** otter768 has joined #openstack-infra | 19:20 | |
| *** unicell1 has quit IRC | 19:20 | |
| fungi | clarkb: yeah, some local testing is making me more and more suspicious of their tox.ini contributing to the issue | 19:21 |
| fungi | clarkb: zeroing in on which option(s) is/are at fault now | 19:21 |
| *** unicell has joined #openstack-infra | 19:22 | |
| *** dustins has quit IRC | 19:22 | |
| *** tonytan4ever has quit IRC | 19:23 | |
| *** tonytan4ever has joined #openstack-infra | 19:25 | |
| fungi | clarkb: mriedem: it looks like the -U is what's causing this (well, specifically it's that tox is telling pip to ignore on-disk dependencies and drag in latest of whatever) | 19:25 |
| fungi | not sure what else might break in nova if you drop the -U though | 19:25 |
| *** hashar has joined #openstack-infra | 19:25 | |
| *** tonytan4ever has quit IRC | 19:26 | |
| dguerri | reed: about the "has multiple fixed IP" error, handling ports with multiple fixed IPs has been fixed in attach_ip_to_server() but using auto IP in create server a fixed IP is not passed to that function. | 19:26 |
| mriedem | fungi: hmm, good point | 19:27 |
| mriedem | fungi: added here https://github.com/openstack/nova/commit/cab21d229bd361aeca1c1cb71e7bafb791f2a28f | 19:27 |
| mriedem | at least --force-reinstall was | 19:27 |
| dguerri | reed: that seems to be due to the additional fixed (IPv6) address assigned by dreamhost (and by the fact that auto ip is intended for basic networking configurations) | 19:28 |
| dguerri | reed: but I think we can (and should) support your use case | 19:28 |
| dguerri | mordred: your take? ^ | 19:28 |
| dguerri | Shrews: ^ your too :) | 19:28 |
| clarkb | fungi we cant fix that with a single install pass of requirements.txt and test requirements? | 19:29 |
| openstackgerrit | Merged openstack-infra/system-config: use updated jenkins_master.pp manifest https://review.openstack.org/199780 | 19:29 |
| mordred | dguerri: yes - I think we should support the use case | 19:29 |
| mordred | dguerri: and since the additional address is ipv6 - we should be able to detect that there is one ipv4 address and attach to that | 19:29 |
| dguerri | the only problem I see is about pinking the "right" fixed ip | 19:29 |
| dguerri | yep, that's true for the IPv6 case which is reed's use case | 19:30 |
| *** erlon has quit IRC | 19:30 | |
| dguerri | are you aware of some cloud assigning multiple IPv4 addresses? | 19:30 |
| mordred | I thnk if there are actually 2 ipv4 - the user is going to have to be involved | 19:30 |
| *** dims__ has joined #openstack-infra | 19:30 | |
| mordred | I am not | 19:31 |
| mordred | not by default at least | 19:31 |
| fungi | clarkb: not in this case i don't think... mock and oslo.versionedobjects are both in test-requirements.txt | 19:31 |
| dguerri | +1 | 19:31 |
| mordred | and if there are - they should be ashamed :) | 19:31 |
| *** amitgandhinz has quit IRC | 19:31 | |
| *** stevemar has joined #openstack-infra | 19:31 | |
| dguerri | eh | 19:31 |
| fungi | clarkb: no, i stand corrected. oslo.versionedobjects is in requirements.txt | 19:31 |
| *** pc_m is now known as pc_m_AWAY | 19:31 | |
| fungi | so pbr is presumably responsible for pass #2 in this case | 19:32 |
| *** sdake has quit IRC | 19:32 | |
| Shrews | dguerri: yeah, seems like we should support this use case | 19:32 |
| *** sergmelikyan has joined #openstack-infra | 19:33 | |
| mordred | fungi: this should be fixed by putting tests in extras and doing .[test] | 19:33 |
| fungi | mordred: but would it? | 19:33 |
| mordred | fungi: however, lifeless was saying he didn't think the 2 passes were an actual functional problem - | 19:33 |
| mordred | fungi: tox does two installs | 19:34 |
| *** dims_ has quit IRC | 19:34 | |
| lifeless | right | 19:34 |
| fungi | mordred: well, it looks like nova relies on pip install -U | 19:34 |
| mordred | yay lifeless! | 19:34 |
| clarkb | you nltge issue is they are evaluated separately | 19:35 |
| fungi | without pip install -U in tox.ini it's not happening, but wondering how pip install -U .[test] is going to do any differently than pip install -U -r test-requirements.txt | 19:35 |
| clarkb | so mock isnt a top level dep second pass | 19:35 |
| clarkb | thats why old way is a thibg | 19:35 |
| *** pc_m_AWAY has quit IRC | 19:36 | |
| *** Sherif has joined #openstack-infra | 19:36 | |
| lifeless | pi pinstall -U -r test-requirements.txt will get pip from the list in test-requirements.txt | 19:36 |
| lifeless | erm | 19:36 |
| lifeless | mock | 19:36 |
| lifeless | or whatever you're discussing | 19:36 |
| fungi | right, in this case pass #1 is installing things in requirements.txt and test-requirements.txt (and so the pin in test-requirements.txt is being respected). then pass #2 installs nova and pbr injects the setup_requires from requirements.txt, but the pin from test-requirements.txt is not present in that pass | 19:36 |
| lifeless | same with .[test] | 19:36 |
| lifeless | the final install of just mock | 19:37 |
| lifeless | sorry | 19:37 |
| dguerri | reed: can you provide a DH account :D | 19:37 |
| lifeless | just the sdist | 19:37 |
| lifeless | or just develop | 19:37 |
| lifeless | will upgrade everything if -U is present | 19:37 |
| lifeless | I personally wouldn't put -U there | 19:37 |
| fungi | yep | 19:37 |
| mordred | tox does "pip install -U -r requirements.txt -r test-requirements.txt" then "pip instal -U ." which pulls in . and requirements | 19:37 |
| lifeless | as it will upgrade anything transitively reached from requirements | 19:37 |
| mordred | but mock is a dep of oslo.versionedrequirements | 19:37 |
| *** esmute has joined #openstack-infra | 19:37 | |
| lifeless | yup | 19:38 |
| lifeless | the -U is the cause | 19:38 |
| mordred | so the pin in test-requirements.txt is ignored | 19:38 |
| mordred | yah | 19:38 |
| *** jsavak has quit IRC | 19:38 | |
| fungi | i believe -U was a workaround for system sitepackages making earlier versions of things present we couldn't uninstall (like jinja2) | 19:38 |
| openstackgerrit | litong01 proposed openstack-infra/project-config: Add new python-monasca repository for OpenStack monitoring service https://review.openstack.org/203081 | 19:38 |
| mordred | fungi: yes. that is true | 19:38 |
| clarkb | and devs not upgrading | 19:38 |
| fungi | well, that too | 19:38 |
| *** jsavak has joined #openstack-infra | 19:38 | |
| mordred | it's there because of sitepackages | 19:39 |
| clarkb | no sitepackages are dead | 19:39 |
| fungi | they are now | 19:39 |
| clarkb | its there now because its easy mode upgrades | 19:39 |
| fungi | but they weren't when -U was added | 19:39 |
| fungi | is my point | 19:39 |
| clarkb | I thought we removed -U after sitepackages died | 19:39 |
| clarkb | theb added it back | 19:39 |
| fungi | in theory correct lower bounds would solve both of those issues, but for some reason the sitepackages one was particularly sticky (pip bug maybe?) | 19:39 |
| clarkb | because we added it everywhere not just nova | 19:40 |
| fungi | at least in nova's tox.ini, the -U has been there continually in various forms all the way back to https://review.openstack.org/36514 that introduced it two years ago | 19:42 |
| fungi | took a few minutes to follow those breadcrumbs through several tox.ini refactors | 19:43 |
| *** Sukhdev has quit IRC | 19:43 | |
| *** jsavak has quit IRC | 19:44 | |
| *** Swami has quit IRC | 19:44 | |
| fungi | so it doesn't look like it got removed, broke nova tests and then got added back again | 19:44 |
| clarkb | I wonder why we have it in all the tox.ini then | 19:44 |
| *** otter768 has quit IRC | 19:44 | |
| clarkb | I would check git log but busy afk | 19:44 |
| *** otter768 has joined #openstack-infra | 19:45 | |
| *** jsavak has joined #openstack-infra | 19:45 | |
| fungi | clarkb: http://git.openstack.org/cgit/openstack-dev/cookiecutter/tree/%7b%7bcookiecutter.repo_name%7d%7d/tox.ini#n8 seems to have been in mordred's initial import | 19:46 |
| *** _nadya_ has joined #openstack-infra | 19:46 | |
| *** rm_work has quit IRC | 19:47 | |
| *** freyes has quit IRC | 19:47 | |
| *** ig0r_ has quit IRC | 19:47 | |
| *** rm_work has joined #openstack-infra | 19:48 | |
| *** freyes has joined #openstack-infra | 19:49 | |
| fungi | so ripping out -U everywhere seems like a good idea, but should probably wait until we've got lower constraints testing | 19:49 |
| lifeless | no | 19:49 |
| lifeless | we should rip out -U as soon as we have constraints at all | 19:49 |
| *** Sherif has quit IRC | 19:50 | |
| *** sicarie has joined #openstack-infra | 19:50 | |
| lifeless | (two reasons - its meaningless when constraints are on, and effectively untested wild west when they aren't in use by a given dev) | 19:50 |
| fungi | i have a feeling that will cause a bunch of new issues to bubble up out of persistent developer environments | 19:50 |
| lifeless | which tox -repy27 will fix every time :) | 19:51 |
| fungi | unless by constraints at all you mean all devs running tox enviroments consuming the constraints list | 19:51 |
| fungi | environments | 19:51 |
| lifeless | that is the goal | 19:51 |
| lifeless | to make that easily consumable | 19:51 |
| lifeless | but TBH having the gate reliable is the first and key goal | 19:51 |
| clarkb | fungi that'll do it | 19:52 |
| lifeless | I'm more worried about confusion | 19:52 |
| lifeless | pip install -c constraints -U thing | 19:52 |
| lifeless | is meaningless | 19:52 |
| lifeless | and I think folk will be asking 'why didn't new X' get installed ... | 19:52 |
| fungi | hopefully pip dtrt there and ignores -U? | 19:52 |
| jogo | anteaya: thanks for championing my patch | 19:52 |
| openstackgerrit | Chris Jones proposed openstack-infra/glean: Support explicitly configured DHCP interfaces https://review.openstack.org/202250 | 19:53 |
| lifeless | fungi: it won't ignore it, because you might be only partially constraining things | 19:53 |
| fungi | fair enough, but it won't upgrade anything you're constraining | 19:53 |
| lifeless | fungi: but for *our* constraints files... it will be effectively constrained | 19:53 |
| lifeless | except for setuptools | 19:54 |
| lifeless | pip | 19:54 |
| fungi | i.e. -U won't ignore a constraint and install something newer | 19:54 |
| lifeless | and anything else stripped from pip freeze | 19:54 |
| lifeless | right | 19:54 |
| lifeless | so I can just imagine 'why didn't new-release-X' get used questions | 19:54 |
| fungi | clarkb: did you see my dib package caching revelations from earlier? | 19:55 |
| *** sergmelikyan has quit IRC | 19:55 | |
| clarkb | fungi yes but havent looked yet | 19:57 |
| fungi | basically all the effort we currently go to in pre-caching packages is completely wasted, because dib goes out of its way to avoid caching packages (for yum) and by default cleans up any that were cached (for apt-get) | 19:57 |
| clarkb | yay | 19:57 |
| *** gyee has quit IRC | 19:57 | |
| clarkb | our images get even bigger now :) | 19:57 |
| fungi | heh | 19:57 |
| lifeless | so yo ufolk are aware | 19:57 |
| lifeless | I've released mock 1.2.0 | 19:58 |
| lifeless | with the mock_open fix reverted | 19:58 |
| lifeless | so please stop running around like head-missing-chickens :) | 19:58 |
| fungi | lifeless: okay, so nova can get rid of the override they put in their tox.ini now i guess | 19:58 |
| fungi | mriedem: sdague: ^ | 19:58 |
| jeblair | clarkb: aprvd log socket change | 19:58 |
| jeblair | fungi: :( comment on https://review.openstack.org/203187 | 19:59 |
| fungi | jeblair: well, at least i called it out as a potential issue! | 19:59 |
| jeblair | fungi: thank you very much for doing so! i probably wouldn't have thought of checking otherwise :) | 20:00 |
| *** annegentle_ has quit IRC | 20:00 | |
| jeblair | clarkb, fungi: i'm going over to a different desktop to work on the zuul dos; biab. | 20:00 |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/system-config: Stop DIB deleting our dpkg cache https://review.openstack.org/203187 | 20:01 |
| fungi | jeblair: ^ now with quoting! | 20:01 |
| fungi | k | 20:01 |
| fungi | i'll see if i can whip up a nodepool config parsing patch for env-vars values now | 20:02 |
| mriedem | lifeless: fungi: yup doing that locally now | 20:02 |
| clarkb | fungi you disable the apt cleanup on yum machines | 20:03 |
| clarkb | is that intentional? | 20:03 |
| fungi | clarkb: hah! shotgun. i'll fix--thanks! ;) | 20:03 |
| *** jsavak has quit IRC | 20:04 | |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/system-config: Stop DIB deleting our dpkg cache https://review.openstack.org/203187 | 20:04 |
| fungi | clarkb: now slightly shorter ^ | 20:04 |
| Clint | doh | 20:05 |
| openstackgerrit | Scott Moser proposed openstack-infra/git-review: support urls as change-id input https://review.openstack.org/199225 | 20:05 |
| *** mjturek1 has quit IRC | 20:06 | |
| *** geoffarnold has quit IRC | 20:09 | |
| *** teran has joined #openstack-infra | 20:10 | |
| openstackgerrit | lifeless proposed openstack/requirements: mock 1.2 has the 1.1.4 regression fixed and is 2.6 compatible https://review.openstack.org/203231 | 20:10 |
| openstackgerrit | lifeless proposed openstack/requirements: mock 1.2 has the 1.1.4 regression fixed and is 2.6 compatible https://review.openstack.org/203231 | 20:11 |
| lifeless | mriedem: already ahead of you ^ | 20:11 |
| mriedem | lifeless: :) | 20:12 |
| *** radez is now known as radez_g0n3 | 20:13 | |
| openstackgerrit | Antoine Musso proposed openstack-infra/nodepool: Stop all threads on SIGUSR1 https://review.openstack.org/203232 | 20:13 |
| hashar | fungi: https://review.openstack.org/#/c/203232/ might teach nodepool to properly stop / exit :) | 20:14 |
| hashar | fungi: a bunch of threads were not exited from the Nodepool class | 20:14 |
| *** jasondotstar has joined #openstack-infra | 20:16 | |
| *** stevemar has quit IRC | 20:16 | |
| *** jsavak has joined #openstack-infra | 20:17 | |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/nodepool: Recast env-vars values to str https://review.openstack.org/203233 | 20:21 |
| fungi | jeblair: ^ in hopes we might eventually be able to drop the quotes ;) | 20:21 |
| lifeless | mordred: sdague: https://review.openstack.org/#/c/199782/ | 20:22 |
| lifeless | if one of you could +A that, that would be nice | 20:22 |
| *** ddieterly has joined #openstack-infra | 20:23 | |
| fungi | lifeless: on a related note, did you see my take two on the exit codes behavior? | 20:23 |
| lifeless | fungi: I saw it go by, I did not look yet | 20:23 |
| lifeless | let me quickly do so now | 20:23 |
| *** HeOS has joined #openstack-infra | 20:24 | |
| fungi | it's broken into three patches for easier reviewing/reasoning | 20:24 |
| lifeless | jeblair: FWIW I'm finding I have to ctrl-R every patch in gertty to get them to update | 20:24 |
| lifeless | jeblair: particularly old ones | 20:24 |
| lifeless | again, sorry, I haven't dug into the plumbing | 20:25 |
| asselin_ | fungi, thanks, looks like it was a noop, as expected | 20:25 |
| lifeless | fungi: actually,20m of family time, *then* I'll look | 20:26 |
| *** dboik has quit IRC | 20:26 | |
| fungi | lifeless: it's not in any way urgent. don't feel like you need to look at it on your weekend | 20:26 |
| hashar | lifeless: thanks for mock. It is quite useful :) | 20:26 |
| openstackgerrit | Davide Guerri proposed openstack-infra/shade: Always use a fixed address when attaching a floating IP to a server https://review.openstack.org/203234 | 20:26 |
| dguerri | reed: mordred Shrews ^ it needs to be tested, tho | 20:27 |
| *** adreznec has quit IRC | 20:27 | |
| fungi | clarkb: glancing at puppetboard, it seems there's a puppet failure for elasticsearch02 | 20:27 |
| Shrews | dguerri: i just got connected to DH. I'll give it a spin, assuming I can reproduce reed's error | 20:27 |
| dguerri | Shrews: thanks! | 20:27 |
| fungi | clarkb: nevermind. it looks like there's something weird about my authorized key removal for that one | 20:28 |
| fungi | clarkb: oh! i see why... root has filled up | 20:29 |
| fungi | can't create any tempfiles | 20:30 |
| clarkb | gah we have a cron to fix that | 20:31 |
| fungi | some very large (3gb) elasticsearch logs up until a week ago when the server ran out of space | 20:31 |
| clarkb | ya you can delete most of those | 20:31 |
| *** amitgandhinz has joined #openstack-infra | 20:31 | |
| clarkb | and I need to debug the thibg setup to clean them out | 20:32 |
| clarkb | root cron should have a find to do it | 20:32 |
| clarkb | cant logrotate because it rotates for you but doesnt delete | 20:32 |
| clarkb | unless maybe that change hasnt merged | 20:32 |
| fungi | i just deleted elasticsearch.log.2015-07-0[234] but left elasticsearch.log.2015-07-05 in case you need to see anything it was logging prior to the disk filling up all the way | 20:33 |
| *** larainema has quit IRC | 20:33 | |
| *** larainema has joined #openstack-infra | 20:34 | |
| *** tongli has quit IRC | 20:34 | |
| *** dims__ has quit IRC | 20:34 | |
| clarkb | 02 does the logstash queries so it logs a bunch | 20:34 |
| *** adreznec has joined #openstack-infra | 20:34 | |
| fungi | similarly full root filesystem on pypi.dfw which i'm looking into now | 20:34 |
| clarkb | were there only those files? | 20:34 |
| clarkb | maybe the cleanup works but isnt aggressive enough | 20:34 |
| fungi | apparently we generate some very large apache access logs on pypi.dfw | 20:35 |
| *** Swami has joined #openstack-infra | 20:35 | |
| *** sergmelikyan has joined #openstack-infra | 20:35 | |
| fungi | yesterday's pypi.dfw.openstack.org_access.log was 4.4gb | 20:36 |
| Shrews | reed, mordred: lol, i love the dreamhost flavor names: hyperspeed, ridiculous, ludicrous, plaid | 20:36 |
| fungi | Shrews: makes you feel like your brains are going to your feet? | 20:37 |
| * fungi hasn't seen that film in far too long now | 20:37 | |
| *** _ody has joined #openstack-infra | 20:39 | |
| *** ihrachyshka has joined #openstack-infra | 20:39 | |
| *** shashank_hegde has joined #openstack-infra | 20:41 | |
| *** julim_ has quit IRC | 20:43 | |
| *** e0ne has quit IRC | 20:43 | |
| *** _ody has quit IRC | 20:44 | |
| *** FallenPegasus has joined #openstack-infra | 20:44 | |
| *** FallenPegasus has quit IRC | 20:44 | |
| *** _nadya_ has quit IRC | 20:45 | |
| *** FallenPegasus has joined #openstack-infra | 20:45 | |
| *** _nadya_ has joined #openstack-infra | 20:45 | |
| *** jamesmcarthur has quit IRC | 20:46 | |
| *** FallenPegasus has quit IRC | 20:46 | |
| fungi | okay, i see the problem with pypi.dfw | 20:46 |
| *** mrmartin has quit IRC | 20:46 | |
| fungi | apparently we have third-party ci systems defaulting to that as their pypi?!? | 20:46 |
| *** aysyd has quit IRC | 20:47 | |
| *** gordc has quit IRC | 20:47 | |
| Shrews | reed: mordred: the fix by dguerri (https://review.openstack.org/203234) seems to fix the dreamhost issue | 20:47 |
| *** FallenPegasus has joined #openstack-infra | 20:47 | |
| asselin_ | fungi, ours wasn't supposed to, but I saw some log messages indicating it was using that mirror in the past few days. I had to force it to use our own mirros as part of our ci jobs. | 20:49 |
| *** _nadya_ has quit IRC | 20:49 | |
| fungi | asselin_: yeah, all i can figure is that ci systems are copying our modules/openstack_project/files/pydistutils.cfg from the system-config repo | 20:52 |
| *** dprince has quit IRC | 20:53 | |
| asselin_ | fungi, we do point to much of upstream files for better or worse. | 20:53 |
| Shrews | fungi: i'm trying really hard to not make a joke about my schwartz, but i will refrain since this is a public forum | 20:54 |
| *** bnemec has quit IRC | 20:54 | |
| fungi | indeed | 20:54 |
| lifeless | hashar: well, I've just started looking after backports of it | 20:56 |
| lifeless | hashar: don't blame me for mock per se :) | 20:56 |
| *** eharney has quit IRC | 20:57 | |
| hashar | lifeless: given my python skill level, it fully matches my expectations :D | 20:57 |
| *** jogo is now known as flashgordon | 20:57 | |
| lifeless | russellb: can you make the networking-ovn thing into a pbr bug ? | 20:59 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Reset repos with files named 'HEAD' https://review.openstack.org/203243 | 21:00 |
| *** dims_ has joined #openstack-infra | 21:00 | |
| russellb | lifeless: ack, or maybe otherwiseguy will do it for me ;-) | 21:00 |
| jeblair | clarkb, fungi: https://review.openstack.org/203243 | 21:00 |
| dhellmann | I seem to be experiencing an unusual delay between submitting a patch and having a node available to run the jobs. https://review.openstack.org/203212 has been waiting just over 2 hours. Is our capacity down, or is demand unusually high? | 21:00 |
| *** geoffarnold has joined #openstack-infra | 21:00 | |
| jeblair | clarkb, fungi: fix with previously failing test, along with future-improving pull-request to gitpython | 21:00 |
| *** btully has joined #openstack-infra | 21:01 | |
| jeblair | dhellmann: the graphs show an alarming lack of data that i would normally use to answer that question :/ | 21:01 |
| dhellmann | jeblair: yeah, I wasn't able to interpret them either, so I'm glad it's not just me | 21:02 |
| jeblair | oh, huh, data just showed up. | 21:02 |
| jeblair | i've never seen that happen before. | 21:02 |
| dhellmann | oh, hmm, maybe I'm still just not sussing the information out from the graphs | 21:03 |
| jeblair | dhellmann: red line on 'zuul job queue' is the backlog (so roughly 1500 right now) | 21:03 |
| dhellmann | I see a rather long job queue, but those numbers seem an order of magnitude higher than the number of jobs reported as being in the queues above | 21:03 |
| dhellmann | ah, ok, so there really are that many jobs | 21:04 |
| jeblair | dhellmann: yeah, the numbers in the pipelines above are 'changes in queue' as opposed to 'jobs in queue'. so 'order of magnitude' is about right (woud mean average of 10 jobs per change) | 21:04 |
| dhellmann | oh, sure, that makes sense | 21:04 |
| fungi | jeblair: yep, that's a pretty unfortunate bug. i want to say i've run into that in other places too where we git reset to some branch and there's a file in the working tree with the same name as the branch | 21:04 |
| *** bcourt has quit IRC | 21:05 | |
| fungi | or refname in general, not just a branch | 21:05 |
| jeblair | dhellmann: also, roughly speaking looking at our capacity, that's probably a bit more than a 2-hour backlog | 21:05 |
| *** btully has quit IRC | 21:05 | |
| *** jasondotstar has quit IRC | 21:05 | |
| *** jsavak has quit IRC | 21:05 | |
| clarkb | might need to see f hpcloud has lealed fips again | 21:06 |
| dhellmann | jeblair: ok, cool. it seemed unusual, esp. for a friday afternoon, so I thought I'd ask in case something was going on. It sounds like I have my excuse to call it a day, though. :-) | 21:06 |
| *** jsavak has joined #openstack-infra | 21:06 | |
| jeblair | dhellmann: so then finding your change in the check queue and then walking up the queue, i see changes about 15 minutes ahead of it receiving nodes | 21:06 |
| *** btully has joined #openstack-infra | 21:07 | |
| jeblair | dhellmann: yeah, we seem to be especially busy bees. :) | 21:07 |
| *** alivigni has quit IRC | 21:07 | |
| dhellmann | jeblair: yep, thanks for looking at it, and for the explanation | 21:08 |
| jeblair | fungi: yeah, my fix is general too, so should take care of similar errors | 21:08 |
| *** alivigni has joined #openstack-infra | 21:08 | |
| jeblair | hashar: if you're around, you may want to look at https://review.openstack.org/203243 | 21:08 |
| *** alivigni has quit IRC | 21:08 | |
| hashar | jeblair: yeah still there | 21:09 |
| hashar | ahah nice one | 21:09 |
| jeblair | hashar: it's a potential DoS we found this morning | 21:09 |
| *** alivigni has joined #openstack-infra | 21:09 | |
| *** alivigni has quit IRC | 21:09 | |
| clarkb | jeblair: reviewing the zuul fix now | 21:10 |
| hashar | jeblair: seems to be an issue in upstream GitPYthon isn't it ? | 21:10 |
| *** akshai has quit IRC | 21:10 | |
| jeblair | hashar: yep; i proposed a fix upstream and a workaround in zuul | 21:10 |
| hashar | \O/ | 21:10 |
| fungi | clarkb: nibalizer: i'm trying to figure out how to go about scaling back the log retention for apache on some of our servers. is there a clean way to override the retention in /etc/logrotate.d/apache2 with our puppet-logrotate even though it's provided separately by the distro package for the webserver? | 21:11 |
| clarkb | fungi: anteaya was working on that for the git servers | 21:12 |
| jeblair | fungi: just use it to write that file? | 21:12 |
| *** jsavak has quit IRC | 21:12 | |
| *** dguerri is now known as dguerri` | 21:12 | |
| jeblair | like "logrotate::file { 'apache2' ...}" | 21:13 |
| fungi | yeah, that's what i was considering (and just replicating the values the distro uses other than what we want to change) but didn't know if that was problematic in some way | 21:13 |
| jeblair | fungi: i don't _think_ so -- it just has a regular "file{}" invocation under the hood | 21:14 |
| Clint | there's an augeas lens too | 21:14 |
| clarkb | jeblair: its repo.head.reset does that not imply we are reseting to HEAD? wondering why you need the HEAD argument to reset and not just -- | 21:14 |
| clarkb | jeblair: is it because of the --hard? | 21:14 |
| openstackgerrit | Paul Belanger proposed openstack/diskimage-builder: [WIP] Test both fedora 21 and fedora 22 https://review.openstack.org/203247 | 21:14 |
| fungi | assuming our logrotate module supports things like postrotate and prerotate scripts | 21:14 |
| jeblair | clarkb: where is it 'repo.head.reset'? | 21:15 |
| fungi | yeah, seems to | 21:15 |
| *** peristeri has quit IRC | 21:15 | |
| jeblair | clarkb: it is in the _removed_ code... | 21:15 |
| *** ldnunes has quit IRC | 21:15 | |
| clarkb | jeblair: sorry, repo.head.repo.git.reset | 21:15 |
| clarkb | line 30 of merger.py | 21:15 |
| *** ayoung has quit IRC | 21:16 | |
| clarkb | ok the reason name resolution doesn't work in hpcloud for multinode job is its hostname.novalocal in /etc/hosts | 21:16 |
| clarkb | :| | 21:17 |
| clarkb | cloud init bites again | 21:17 |
| fungi | grar | 21:17 |
| jeblair | clarkb: ah, yeah, i think that could be shortened to 'repo.git.reset' | 21:17 |
| lifeless | russellb: thanks; just to be clear, I'm not going to try to look at it from a gerrit review, the UI is just all sorts of wrong for me | 21:17 |
| jeblair | i think those two repo objects are the same. i was just blindly following the call path it normally takes. | 21:17 |
| jeblair | clarkb: (repo.head.reset called self.repo.git.reset in the old path, so that translates to repo.head.repo.git.reset; but we can shorten it) | 21:18 |
| clarkb | ok | 21:19 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Reset repos with files named 'HEAD' https://review.openstack.org/203243 | 21:19 |
| jeblair | clarkb: done ^ | 21:19 |
| hashar | jeblair: looks legit to me :D | 21:19 |
| jeblair | w00t | 21:20 |
| *** alivigni has joined #openstack-infra | 21:20 | |
| *** alivigni has quit IRC | 21:20 | |
| hashar | meanwhile I am deploying nodepool with a .deb package :D | 21:20 |
| hashar | gotta poke Thomas Goirand to figure out where to host the Debianization work and probably bring it up on par with other openstack packages | 21:21 |
| clarkb | jeblair: +2 | 21:22 |
| *** FallenPegasus has quit IRC | 21:22 | |
| clarkb | fungi: we have been asked ot test the new python3.4 package in trusty | 21:22 |
| clarkb | I will be holding a bare-trusty node to do that shortly | 21:23 |
| fungi | clarkb: awesome--thanks! | 21:23 |
| greghaynes | clarkb: whoa, they made a new package? | 21:23 |
| fungi | i'm about to disappear to do house things for a while | 21:23 |
| *** cbader has quit IRC | 21:23 | |
| clarkb | greghaynes: ya | 21:23 |
| Clint | hashar: where is it now? | 21:23 |
| clarkb | so novalocal | 21:23 |
| pabelanger | hashar: nice, working on RPMs for local soon | 21:24 |
| clarkb | mordred: novalocal is a thing set by metadata thta cloud init tells the host is its domain name? | 21:24 |
| davideagnello | hello, I have a small change to Cue's Rally CI job that was pushed out on Monday, can someone please take a quick look/merge: https://review.openstack.org/#/c/201285/2 | 21:24 |
| hashar | Clint: pabelanger: on Wikimedia gerrit git clone https://gerrit.wikimedia.org/r/operations/debs/nodepool branches are debian and patch-queue/debian | 21:24 |
| hashar | Clint: pabelanger: got something at https://people.wikimedia.org/~hashar/debs/nodepool/ :D | 21:24 |
| hashar | Clint: pabelanger: I am even looking at wrapping it with systemd :) | 21:25 |
| *** geoffarnold has quit IRC | 21:26 | |
| *** EricGonczer_ has quit IRC | 21:27 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/puppet-jenkins: Add Gemfile and puppet 4 checks https://review.openstack.org/203164 | 21:27 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/subunit2sql: Add db migration to align column size to 255 and drop FKs https://review.openstack.org/203251 | 21:27 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/subunit2sql: Compact existing schema and start 1.x.x release branch https://review.openstack.org/203252 | 21:27 |
| *** annegentle_ has joined #openstack-infra | 21:28 | |
| mtreinish | jeblair: ^^^ that should help with the start of dropping oslo.db (although it's not really ready yet) | 21:28 |
| pabelanger | hashar: nice. I need to spend more time getting the packaging spec going for infra | 21:28 |
| pabelanger | it fell off my plate :( | 21:28 |
| jeblair | yeah, i'd love to be able to host both here | 21:28 |
| *** sergmelikyan has quit IRC | 21:28 | |
| hashar | pabelanger: I am sure Thomas Goirand can help somehow | 21:28 |
| hashar | pabelanger: I nicely adjusted the python-gear package already | 21:28 |
| *** teran has quit IRC | 21:28 | |
| pabelanger | hashar: well, we need the spec approaved first. I actually have code up for build environments in gerrit, but gatting on spec | 21:29 |
| *** tvanderwiel has quit IRC | 21:29 | |
| pabelanger | and example gate job | 21:29 |
| hashar | pabelanger: have you looked at http://jenkins-debian-glue.org/ ? | 21:29 |
| *** blockedpipes has quit IRC | 21:29 | |
| pabelanger | hashar: ya. Most of the stuff I did was just straight git-buildpackage stuff. Was considering using fedpkg for fedora stuff, since they support git-buildpackage / mock | 21:30 |
| Clint | hashar: looks like a decent start | 21:30 |
| hashar | Clint: thx :) | 21:30 |
| hashar | once we get something polished enough, I will rebuild it with latest tag and toward Debian sid | 21:31 |
| Clint | does the nodepool user really need to own those directories getting chowned in the postinst? | 21:31 |
| *** sergmelikyan has joined #openstack-infra | 21:31 | |
| *** jasondotstar has joined #openstack-infra | 21:31 | |
| hashar | Clint: can't remember :( | 21:35 |
| openstackgerrit | James E. Blair proposed openstack-infra/puppet-logrotate: Require logrotate::file names not be paths https://review.openstack.org/202316 | 21:35 |
| hashar | Clint: I think the idea was to get diskimages / config to be editable by members of the 'nodepool' group | 21:35 |
| Clint | hashar: i imagine that's doable another way | 21:36 |
| Clint | and the chmodding should be completely unnecessary | 21:36 |
| jeblair | fungi: speaking of logrotate, i made suggestions in https://review.openstack.org/192187 based on the latest version of the puppet module change which i just approved, and then made my logrotate api change at https://review.openstack.org/202316 depend on it (because we need to get the slashes out of those filenames for my change to work) | 21:37 |
| fungi | jeblair: thanks! | 21:38 |
| hashar | Clint: that is to make the file writable to the nodepool user. IIRC dirs created from debian/dirs ends up readonly :( | 21:38 |
| openstackgerrit | Merged openstack-infra/system-config: Increase git haproxy maxconn from 32 to 256 https://review.openstack.org/203165 | 21:38 |
| fungi | now i'm headed out to grab dinner and do house stuff, but may be back on (much) later tonight | 21:39 |
| jeblair | fungi: have a nice house! and enjoy dinner! | 21:39 |
| Clint | hashar: fixable in debian/rules instead of the postinst | 21:39 |
| Clint | that way you don't break dpkg-statoverride | 21:40 |
| nibalizer | fungi: probably we just overwrite the package-provided logrotate config file with our own config file | 21:40 |
| nibalizer | thats cleaner I think than other options | 21:41 |
| Clint | (i mean, the chowns are going to break dpkg-statoverride too) | 21:41 |
| hashar | Clint: that is above my .deb skill level :D | 21:41 |
| *** FallenPegasus has joined #openstack-infra | 21:41 | |
| * Clint grins. | 21:41 | |
| openstackgerrit | Merged openstack-infra/puppet-logrotate: Add subclass to remove logrotate configurations https://review.openstack.org/192242 | 21:42 |
| pabelanger | Hmm | 21:43 |
| pabelanger | does anybody know why the following would have been cancelled? https://jenkins04.openstack.org/job/gate-diskimage-builder-docs/85/ | 21:43 |
| pabelanger | of how I could check | 21:43 |
| hashar | Clint: ahhh at the end of https://www.debian.org/doc/debian-policy/ch-files.html#s10.9.1 . I have seen that before but gave up | 21:44 |
| pabelanger | I cannot see any new patchsets that were uploaded | 21:45 |
| openstackgerrit | Clark Boylan proposed openstack-infra/devstack-gate: Setup name resolution for non long name https://review.openstack.org/203254 | 21:45 |
| jeblair | pabelanger: it's in the gate, so a change ahead probably bombed out and zuul needed to discard restart after reconfiguring the queue | 21:45 |
| *** lykinsbd has quit IRC | 21:45 | |
| *** linkedinyou|2 has joined #openstack-infra | 21:45 | |
| jeblair | pabelanger: if you look at the status page, you can see it's still in the gate with only success and queued jobs, so it has been succeeded by a later build | 21:45 |
| jeblair | er s/discard restart/discard the current build state and restart jobs/ | 21:46 |
| asselin_ | nibalizer, did you evr get a chance to look at this? https://review.openstack.org/#/c/200643/ | 21:46 |
| nibalizer | looking now | 21:47 |
| jeblair | pabelanger: https://jenkins05.openstack.org/job/gate-diskimage-builder-docs/225/ is the latest, though still potentially not final, build | 21:47 |
| *** annegentle_ has quit IRC | 21:47 | |
| Clint | hashar: i'm sure we can converge on something sensible | 21:47 |
| clarkb | I have held 3781190 to test the new python package on trusty | 21:48 |
| clarkb | will get to thta once the test running there completes | 21:48 |
| *** linkedinyou has quit IRC | 21:48 | |
| pabelanger | jeblair: so, question that I have is, how is review 185841 linked to review 201327 ahead of it? I'm surprised to see the black line connecting them. Unless I am overlooking something obvious | 21:50 |
| pabelanger | on status.o.o/zuul | 21:50 |
| jeblair | pabelanger: all changes for projects that share a change queue in zuul are put in a fifo queue in the order they are approved; each change in that queue represents an incremental change to the entire suite of projects in the queue. | 21:52 |
| jeblair | pabelanger: it's sort of zuul's raison detre; explained with words and pix at http://docs.openstack.org/infra/zuul/gating.html | 21:53 |
| lifeless | good news - python 3.4 in trusty will be updated soon | 21:54 |
| lifeless | just building on ARM I believe | 21:54 |
| jeblair | lifeless: clarkb is preparing to test the update | 21:54 |
| hashar | Clint: I will soon have systemd setup. Will then finish up the package and make sure it runs against sid then announce it on the infra list | 21:54 |
| lifeless | cool | 21:54 |
| *** mriedem has quit IRC | 21:54 | |
| clarkb | ya I have a test slave held currently confirming the fail case is still present in oslo.messaging so Ican compare | 21:55 |
| clarkb | if not I iwll have to go checkout a version with it | 21:55 |
| clarkb | I have never been so happy to see a segmentation fault | 21:57 |
| pabelanger | jeblair: okay. Never noticed that before. Must be lack of coffee atm | 21:57 |
| Clint | hashar: great | 21:58 |
| pabelanger | thanks for the info | 21:58 |
| *** amitgandhinz has quit IRC | 21:58 | |
| *** hdd has quit IRC | 21:59 | |
| pabelanger | Of course the gate fails now :( | 22:00 |
| *** FallenPegasus has quit IRC | 22:00 | |
| jeblair | pabelanger: yeah, now you're seeing 841 failed, and since 842 is git-dependent on it, zuul has pulled it out line as well | 22:01 |
| jeblair | out of line | 22:01 |
| clarkb | lifeless: where do I look to see if hte package is built for amd64? | 22:01 |
| pabelanger | jeblair: right | 22:01 |
| pabelanger | jeblair: how is the integrated queue decided? | 22:02 |
| pabelanger | I couldn't see anything configuring that | 22:02 |
| jeblair | pabelanger: projects that share jobs with the same name | 22:02 |
| pabelanger | ah | 22:02 |
| jeblair | pabelanger: so i think dib shares a job with ironic which shares jobs with the rest | 22:02 |
| pabelanger | okay, so that explains it then | 22:02 |
| pabelanger | right | 22:02 |
| pabelanger | that make sense now | 22:02 |
| pabelanger | first time working in diskimage-builder | 22:02 |
| openstackgerrit | Ian Cordasco proposed openstack/requirements: Add keystoneauth to project.txt https://review.openstack.org/203256 | 22:02 |
| lifeless | fungi: reviewed | 22:03 |
| jeblair | pabelanger: all the infra puppet modules share a different queue because they run the apply (and now also beaker) jobs | 22:03 |
| lifeless | clarkb: has it been released from NEW yet ? | 22:03 |
| clarkb | https://launchpad.net/ubuntu/+source/python3.4/3.4.3-1ubuntu1~14.04/+build/7664830 | 22:03 |
| clarkb | lifeless: ^ I have no idea | 22:03 |
| pabelanger | jeblair: Ya, I was going crazying trying to figure out how openstack/keystone and dib were dependent on each other. Now that you explained that, it makes total sense now | 22:04 |
| lifeless | clarkb: while I look at that. https://review.openstack.org/#/c/199782/ <- +2A por favor | 22:04 |
| clarkb | I can always brute force it and grab the debs too if I don't want to wait for them to be in proposed | 22:04 |
| jeblair | the change at the head looks suspiciously like a bug; it has a merge conflict but still has 1 job queued | 22:04 |
| pabelanger | ya, seen that | 22:05 |
| lifeless | clarkb: so all the builds are finished - https://launchpad.net/ubuntu/+source/python3.4/3.4.3-1ubuntu1~14.04 | 22:05 |
| lifeless | clarkb: let me ask | 22:05 |
| *** akraminsky_ has joined #openstack-infra | 22:08 | |
| clarkb | heh -y --force-yes | 22:09 |
| openstackgerrit | Paul Belanger proposed openstack/diskimage-builder: [WIP] Test both fedora 21 and fedora 22 https://review.openstack.org/203247 | 22:10 |
| openstackgerrit | litong01 proposed openstack-infra/project-config: Add new cloud-monitor repository for OpenStack monitoring service https://review.openstack.org/203259 | 22:10 |
| clarkb | lifeless: the -b and -f other-requirements.txt make sense given the bindep help output, but what is 'python' doing there? | 22:11 |
| lifeless | clarkb: thats the profile selector | 22:12 |
| *** rlandy has quit IRC | 22:12 | |
| lifeless | clarkb: to tell it to install the numpy and lxml as debs | 22:12 |
| *** sergmelikyan has quit IRC | 22:12 | |
| lifeless | clarkb: I wanted to be able to separate the minimal set from the optimised set | 22:13 |
| lifeless | clarkb: see other-requirements.txt | 22:13 |
| clarkb | ok, the argparse help output doesn't mention taking args just options | 22:13 |
| *** rlandy has joined #openstack-infra | 22:13 | |
| clarkb | lifeless: approved and its self testing and came back jenkins +1 | 22:14 |
| lifeless | clarkb: we should fix the help. Its in the docs. | 22:14 |
| openstackgerrit | K Jonathan Harker proposed openstack-infra/system-config: HPCloud has updated their Precise images https://review.openstack.org/203261 | 22:14 |
| *** esmute has quit IRC | 22:14 | |
| clarkb | lifeless: ok I can stab at that real quick | 22:14 |
| lifeless | clarkb: cool, thanks! | 22:15 |
| lifeless | no word yet from infinity | 22:15 |
| jesusaurus | clarkb: jeblair: looks like hpcloud deprecated their precise image last week | 22:15 |
| *** wenlock has quit IRC | 22:15 | |
| jeblair | jesusaurus: i love it when they do that | 22:15 |
| openstackgerrit | litong01 proposed openstack-infra/project-config: Add new cloud-monitor repository for OpenStack monitoring service https://review.openstack.org/203259 | 22:15 |
| *** isviridov_away is now known as isviridov | 22:15 | |
| *** pc_m has joined #openstack-infra | 22:16 | |
| clarkb | oh wow optparse again and I just forgot how to use it in favor of argparse | 22:16 |
| *** esmute has joined #openstack-infra | 22:16 | |
| openstackgerrit | sebastian marcet proposed openstack-infra/openstackid: OIDC - OpenId Connect Implementation https://review.openstack.org/198458 | 22:17 |
| hashar | merry week-end to everyone | 22:17 |
| *** dalgaaf has quit IRC | 22:18 | |
| openstackgerrit | litong01 proposed openstack-infra/project-config: Add new cloud-monitor repository for OpenStack monitoring service https://review.openstack.org/203259 | 22:19 |
| *** Yichen has joined #openstack-infra | 22:20 | |
| *** stevemar has joined #openstack-infra | 22:20 | |
| Yichen | Hi, everyone. | 22:21 |
| *** davideagnello has quit IRC | 22:21 | |
| Yichen | Basically I need some helps on my review. https://review.openstack.org/#/c/200331/ | 22:21 |
| Yichen | It has been there for a week, and I am wondering can someone sign up and give a +2 on that? | 22:22 |
| jeblair | okay, yeah, apparently if you add a job to a change sitting in merge-conflict in queue, it thinks it should be run. so i think it's stuck. | 22:22 |
| *** crc32 has joined #openstack-infra | 22:22 | |
| *** stevemar has quit IRC | 22:24 | |
| pabelanger | jeblair: if we stood up a http proxy for apt / yum, would you think we'd use AFS for the proxies? Or just use local storage with some sort of expire option stop from running out of HDD space? | 22:25 |
| lifeless | clarkb: looks like its in proposed to me | 22:25 |
| clarkb | lifeless: thanks | 22:25 |
| jeblair | pabelanger: i imagine we'd use local space if we did a proxy | 22:26 |
| clarkb | (and I have figured out how to edit the usage text for bindep sp patch soon. Currently trying to convince the state that we don't need to do even more doctor visits) | 22:26 |
| jeblair | pabelanger: i think afs makes sense if we want to run a mirror (because we get to run it once and distribute everywhere) | 22:26 |
| lifeless | pabelanger: jeblair: btw HTTP proxies for apt are not as good as mirrors, due to the atomicity problem. | 22:26 |
| pabelanger | jeblair: Right, that makes sense to me. | 22:26 |
| jeblair | lifeless: yeah, i think afs will be extra nice for that aspect | 22:27 |
| pabelanger | lifeless: Ya, thats what would need to be decided. proxy HTTP or mirror for apt / yum | 22:27 |
| jeblair | i confess though that i'm reconsidering the proxy idea though because of the number of different _things_ that we seem to want to start having the mirrors handle | 22:27 |
| jeblair | i think it's worth a good thinking-about in that spec. | 22:27 |
| lifeless | I'm not entirely sure what a proxy will buy you, since at least Ubuntu have mirrors in most cloud DC's | 22:28 |
| *** Sherif has joined #openstack-infra | 22:28 | |
| jeblair | lifeless: it seems debian may not? | 22:28 |
| openstackgerrit | Merged openstack-dev/pbr: Updated from global requirements https://review.openstack.org/203110 | 22:28 |
| jeblair | (i noticed the dib changes failed due to debian a moment ago) | 22:29 |
| mordred | yah. also - the cloud mirrors have usually been not better in terms of uptime | 22:29 |
| mordred | like, they're not an improvement | 22:29 |
| lifeless | jeblair: thats more likely; AIUI Ubuntu got its mirrors into many dc's as part of various contractual things, and Debian has just its standard mirror network, vs dc-local aliases | 22:30 |
| mordred | they _Also_ have problems - and one more than one occasion we've had to do work to disable them | 22:30 |
| lifeless | mordred: right, which speaks to mirror vs proxy | 22:30 |
| jeblair | on the afs front, i approved one of mordred's afs-related changes, but not the one that actually installs the client on the git farm; that's up next, then we can try it with git. | 22:30 |
| lifeless | mordred: a proxy will go bad when the upstream is bad | 22:30 |
| mordred | jeblair: woot! | 22:30 |
| lifeless | mordred: I think we're agreeing | 22:30 |
| mordred | lifeless: yes, well, both | 22:30 |
| *** asselin has joined #openstack-infra | 22:30 | |
| mordred | I don't think the per-DC mirrors gain us much | 22:31 |
| mordred | but I am also not sure what a proxy will buy | 22:31 |
| jeblair | but considering we have people wanting npm mirrors, and mirrors of random disk images, etc, that has made me rethink proxies. | 22:31 |
| mordred | jeblair: nod | 22:31 |
| *** linkedinyou has joined #openstack-infra | 22:32 | |
| jeblair | i wonder if maybe we do proxies to catch things that aren't worth the effort of running a mirror for... or something... just brainstorming. | 22:32 |
| *** linkedinyou|2 has quit IRC | 22:32 | |
| persia | In another context, I heard there was a swift feature that could populate a mirror by means of being a caching proxy. Is this something that could be used to provide this need? | 22:33 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Reset repos with files named 'HEAD' https://review.openstack.org/203243 | 22:33 |
| jeblair | thanks flake8! ^ | 22:33 |
| openstackgerrit | Yichen Wang proposed openstack-infra/project-config: Register new project KloudBuster into stackforge https://review.openstack.org/200331 | 22:33 |
| jeblair | persia: i suspect it's subject to the same atomicity issue? i think any proxy would unless it kept files itself that were deleted on its upstream. | 22:34 |
| openstackgerrit | Joshua Harlow proposed openstack/requirements: Bump lower/upper constraint of futurist to newly released 0.1.2 https://review.openstack.org/201709 | 22:35 |
| *** asselin has quit IRC | 22:35 | |
| persia | Ah, yes, as I understand it, deletion notifications would remove the files from the on-demand mirrors. | 22:35 |
| *** Yichen has quit IRC | 22:36 | |
| openstackgerrit | Paul Belanger proposed openstack/diskimage-builder: [WIP] Test both fedora 21 and fedora 22 https://review.openstack.org/203247 | 22:37 |
| jeblair | i'm going to climb back into my zuul-hole and fix the top-of-gate bug... | 22:37 |
| *** Somay has quit IRC | 22:38 | |
| *** isviridov is now known as isviridov_away | 22:39 | |
| *** dimtruck is now known as zz_dimtruck | 22:39 | |
| *** _ody has joined #openstack-infra | 22:40 | |
| *** annegentle_ has joined #openstack-infra | 22:40 | |
| *** annegentle_ has quit IRC | 22:41 | |
| *** jasondotstar has quit IRC | 22:44 | |
| *** hashar has quit IRC | 22:44 | |
| *** _ody has quit IRC | 22:44 | |
| openstackgerrit | Merged openstack-infra/puppet-log_processor: Import socket so we can use it to get name info https://review.openstack.org/203196 | 22:46 |
| madhu_ak | anyone running into a problem that zuul is not sending an email after the job is finished? | 22:48 |
| *** sicarie has quit IRC | 22:48 | |
| *** ivar-lazzaro has quit IRC | 22:49 | |
| asselin_ | madhu_ak, I don't thinkg upstream zuul does that. or is this your own ci? | 22:52 |
| asselin_ | own zuul? | 22:52 |
| clarkb | you may need to configure an MTA? | 22:53 |
| *** ddieterl_ has joined #openstack-infra | 22:53 | |
| clarkb | zuul basically expects to talk to a local MTA iirc and hvae it do the heavy lifting | 22:53 |
| clarkb | WOOT oslo.messaging py34 tests pass with new python from ubuntu | 22:53 |
| * clarkb updates the bug | 22:53 | |
| asselin_ | madhu_ak, if you're using puppet-openstackci, zuul won't send e-mails until this lands (hopefully soon) https://review.openstack.org/#/c/200643/ | 22:54 |
| madhu_ak | actually I am seeing my patch: https://review.openstack.org/#/c/199754/ status here: http://status.openstack.org/zuul. It is succeeded and waiting for a confirmation email from jenkins | 22:56 |
| *** ddieterly has quit IRC | 22:56 | |
| *** mriedem has joined #openstack-infra | 22:57 | |
| *** flashgordon has quit IRC | 22:58 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/bindep: Add profile to the usage text https://review.openstack.org/203272 | 23:01 |
| clarkb | lifeless: fungi ^ thats a simple first stab | 23:01 |
| *** rlandy has quit IRC | 23:02 | |
| openstackgerrit | Armando Migliaccio proposed openstack/requirements: Add networking-l2gw to the list https://review.openstack.org/203275 | 23:03 |
| *** dangers is now known as dangers_away | 23:03 | |
| *** achanda has quit IRC | 23:04 | |
| pabelanger | Perfect, my tests for fedora21 and fedora22 in diskimage-builder passed. | 23:06 |
| pabelanger | now I can go get some food | 23:06 |
| pabelanger | later all | 23:06 |
| asselin_ | madhu_ak, it would be gerrit that sends the e-mail, which would be when you see the verified vote show up on your patch | 23:06 |
| madhu_ak | asselin_ : right. does it take longer to get an email? | 23:07 |
| *** zz_dimtruck is now known as dimtruck | 23:07 | |
| asselin_ | madhu_ak, well the vote didn't show up yet, so the job is not yet 'done' | 23:08 |
| asselin_ | still queued gate-openstackci-beaker-trusty-dsvm: queued (non-voting) | 23:09 |
| madhu_ak | asseslin_ oh yeah, sorry didnt check that. no rush, I can wait till monday ;) | 23:09 |
| madhu_ak | asselin_ | 23:10 |
| *** btully has quit IRC | 23:11 | |
| *** Sherif has quit IRC | 23:11 | |
| *** pc_m has quit IRC | 23:12 | |
| *** flashgordon has joined #openstack-infra | 23:14 | |
| *** esmute has quit IRC | 23:14 | |
| *** esmute has joined #openstack-infra | 23:15 | |
| *** davideagnello has joined #openstack-infra | 23:17 | |
| *** ivar-lazzaro has joined #openstack-infra | 23:18 | |
| *** davideagnello has quit IRC | 23:20 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Handle adding a job to a failing change during reconfig https://review.openstack.org/203281 | 23:24 |
| *** dimtruck is now known as zz_dimtruck | 23:26 | |
| jeblair | clarkb, mordred, fungi, jhesketh: if any of you are around and able to review that change, i'd like to get it it during the zuul restart for the other bug as well. | 23:27 |
| *** ivar-lazzaro has quit IRC | 23:27 | |
| mriedem | jbernard: https://bugs.launchpad.net/glance/+bug/1475811 | 23:27 |
| openstack | Launchpad bug 1475811 in Glance "gate-tempest-dsvm-full-ceph fails to upload image with "TypeError: an integer is required"" [Critical,Confirmed] | 23:27 |
| jeblair | clarkb, mordred, fungi, jhesketh: it's the bug that's causing 192056 to stay at the top of the queue (though fortunately, it's a severed head, so it's not actually holding up the queue). | 23:28 |
| clarkb | looking | 23:30 |
| clarkb | jeblair it fails tests | 23:30 |
| jeblair | :/ | 23:31 |
| *** Swami has quit IRC | 23:32 | |
| jeblair | clarkb: it passed for me; which failed? | 23:32 |
| *** puranamr has quit IRC | 23:32 | |
| jeblair | oh, i guess our backlog is caught up, it ran in zuul | 23:33 |
| *** puranamr has joined #openstack-infra | 23:33 | |
| jeblair | ah forgot to add file | 23:33 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Handle adding a job to a failing change during reconfig https://review.openstack.org/203281 | 23:33 |
| jeblair | clarkb: there ^ | 23:33 |
| mriedem | jbernard: nvm, i'm working a patch | 23:34 |
| clarkb | jeblair if build.result wont that have it ignore the new jobs (builds)? | 23:37 |
| clarkb | line 1183 of scheduler.py | 23:37 |
| lifeless | fungi: 1475811 - we probably want the ceph job to gate openstack/requirements | 23:37 |
| lifeless | fungi: if its generally reliable enough | 23:37 |
| lifeless | mriedem: ^ | 23:37 |
| jeblair | wow, according to the graph, our public clouds had a moment of high efficiency and we ran at almost 87% of quota | 23:37 |
| jeblair | clarkb: looking | 23:37 |
| openstackgerrit | Merged openstack-infra/zuul: Reset repos with files named 'HEAD' https://review.openstack.org/203243 | 23:38 |
| jeblair | clarkb: yes it will -- basically that translates to "for any build that has a result, re-assign that result to itself for the side effect of updating child builds in that job's build tree" | 23:39 |
| jeblair | clarkb: let me point you at the corresponding code | 23:39 |
| jeblair | clarkb: http://git.openstack.org/cgit/openstack-infra/zuul/tree/zuul/model.py#n219 | 23:39 |
| jeblair | clarkb: so the new jobs will not have a corresponding build yet, and actually won't even show up in the list at all (nevermind that if they did their result would be None). _however_, if the new jobs are part of a job tree that has failed, then setting their job-parent to FAILED again will cause them to be set to SKIPPED | 23:40 |
| *** puranamr has quit IRC | 23:42 | |
| *** alexsyip has quit IRC | 23:42 | |
| clarkb | so in the case we have now the grenade job would be skipped because the other jobs for that item were skipped? | 23:43 |
| *** puranamr has joined #openstack-infra | 23:43 | |
| jeblair | clarkb: the case now is actually the first one -- the merge conflict. it's actually fixed by line 1187 -- setUnableToMerge | 23:43 |
| jeblair | clarkb: the rest was just me covering the other ways something can be set to skipped | 23:44 |
| openstackgerrit | Merged openstack/diskimage-builder: Add YUM variable to for Fedora >= 22 https://review.openstack.org/185841 | 23:44 |
| jeblair | clarkb: (and line 231 of model.py) | 23:44 |
| openstackgerrit | Merged openstack/diskimage-builder: Work around yum/dnf differences https://review.openstack.org/185842 | 23:44 |
| jeblair | pabelanger: your changes merged after all ^ :) | 23:45 |
| openstackgerrit | Matt Riedemann proposed openstack/requirements: Exclude glance_store 0.7.0 due to gate blocking regression https://review.openstack.org/203295 | 23:45 |
| mriedem | lifeless: dhellmann: fungi: flaper87: nikhil_k: ^ | 23:45 |
| mriedem | only my 2nd g-r blocker of the day | 23:45 |
| mriedem | :( | 23:45 |
| clarkb | gotcha that explains my confusion, tha ks | 23:45 |
| jeblair | mriedem: should we run the ceph job on requirements? | 23:46 |
| mriedem | jeblair: i'm not sure how that would have caught this | 23:46 |
| mriedem | but i'm not up to date on the new world order with upper-constraints | 23:46 |
| mriedem | upper-constraints for glance_store were made 0.7.0 here https://review.openstack.org/#/c/202791/ | 23:47 |
| mriedem | so if the ceph job were running on that change would it have caught it? idk | 23:47 |
| jeblair | mriedem: i hope so? | 23:47 |
| jeblair | fungi, lifeless: ^ expert analysis? :) | 23:47 |
| mriedem | but glance/cinder/nova are'nt using upper-constraints are they? | 23:47 |
| mriedem | the previous g-r was just >=0.3.0 | 23:47 |
| mriedem | which would have used 0.7.0 in the ceph job and that would have blown up everything | 23:47 |
| mriedem | oh i guess before upper-constraints had this: glance-store===0.6.0 | 23:48 |
| mriedem | so if that would actually prevent devstack from using 0.7.0, then yeah that was probabl yit | 23:48 |
| jeblair | should probably run the ceph job on glance_store? | 23:48 |
| mriedem | b/c 0.7.0 was released yesterday but the gate didn't explode until g-r was updated today | 23:48 |
| mriedem | jeblair: yes that for sure | 23:48 |
| mriedem | that would have caught this | 23:49 |
| * jeblair writes that change | 23:49 | |
| mriedem | so i'd say no to ceph on requirements | 23:49 |
| mriedem | but yes to ceph on glance_store | 23:49 |
| jeblair | yeah that sounds sensible | 23:49 |
| mriedem | \o/ | 23:49 |
| *** larainema has quit IRC | 23:52 | |
| *** larainema has joined #openstack-infra | 23:53 | |
| *** ihrachyshka has quit IRC | 23:53 | |
| openstackgerrit | Matt Riedemann proposed openstack-infra/elastic-recheck: Add query for ceph glance_store bug 1475811 https://review.openstack.org/203296 | 23:53 |
| openstack | bug 1475811 in Glance "gate-tempest-dsvm-full-ceph fails to upload image with "TypeError: an integer is required"" [Critical,In progress] https://launchpad.net/bugs/1475811 - Assigned to Matt Riedemann (mriedem) | 23:53 |
| clarkb | jeblair thinking about that a bit more, will that reset the other jobs to skipped even though they have a fail result? | 23:55 |
| clarkb | jeblair say two jobs failed the we reconfigure, the first fail will cause us to skip the second ? | 23:55 |
| clarkb | the test history length seems to confirm that if I have counted correctly | 23:56 |
| jeblair | mriedem, clarkb: er, does the "install libraries from git" thing screw this up? | 23:58 |
| jeblair | rather, install from pypi? | 23:58 |
| *** tqtran has quit IRC | 23:59 | |
| jhesketh | o/ | 23:59 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!