| jeblair | ianw: thx | 00:00 |
|---|---|---|
| *** bpokorny_ has joined #openstack-infra | 00:02 | |
| *** yamamoto_ has joined #openstack-infra | 00:02 | |
| *** tdurakov_ has quit IRC | 00:02 | |
| jeblair | okay something has gone very wrong with the nodepool host | 00:03 |
| jeblair | possibly because i stopped the builder while it was running dib | 00:03 |
| jeblair | /dev/urandom is gone | 00:04 |
| *** keedya has quit IRC | 00:04 | |
| jeblair | /dev is gone | 00:04 |
| jeblair | /dev/pts on /opt/dib_tmp/image.X12SLNEn/mnt/dev/pts type none (rw,bind) | 00:05 |
| jeblair | /dev on /opt/dib_tmp/image.X12SLNEn/mnt/dev type none (rw,bind) | 00:05 |
| jeblair | /dev/pts on /opt/dib_tmp/image.X12SLNEn/mnt/dev/pts type none (rw,bind) | 00:05 |
| *** tdurakov_ has joined #openstack-infra | 00:05 | |
| jeblair | nodepool@nodepool:~$ umount /dev/pts | 00:05 |
| jeblair | umount: it seems /dev/pts is mounted multiple times | 00:05 |
| *** bpokorny has quit IRC | 00:05 | |
| *** hdd has quit IRC | 00:05 | |
| greghaynes | jeblair: hrm. Is there a symlink somewhere in the dib build dir path? | 00:05 |
| greghaynes | jeblair: that was the bug we found yesterday which causes /dev to be deleted | 00:06 |
| jeblair | greghaynes: it's not deleted, it's mounted over top | 00:06 |
| greghaynes | ah | 00:06 |
| greghaynes | that... shouldnt be possible on the build host | 00:06 |
| ianw | oh dear :( | 00:07 |
| jeblair | oh, i may be wrong about that | 00:08 |
| jeblair | it might be deleted | 00:08 |
| *** KimJ has quit IRC | 00:08 | |
| jeblair | greghaynes: i am not aware of any symlinks. where would you like me to look? | 00:09 |
| *** masahito has quit IRC | 00:09 | |
| *** agentle has joined #openstack-infra | 00:09 | |
| jeblair | greghaynes: /opt/dib_cache /opt/dib_tmp and /opt/nodepool_dib are all plain directories | 00:10 |
| greghaynes | jeblair: /opt/dib_tmp | 00:10 |
| greghaynes | hrm | 00:10 |
| greghaynes | so there must be another way we can hit this case | 00:10 |
| jeblair | do we expect a reboot to allow udev to repopulate /dev? | 00:10 |
| greghaynes | yes | 00:10 |
| jeblair | off it goes | 00:10 |
| ianw | jeblair: and /opt is just a regular mount point, right? | 00:10 |
| jeblair | ianw: yes | 00:10 |
| ianw | double oh-dear :( | 00:11 |
| *** rbrndt_ has quit IRC | 00:11 | |
| greghaynes | ok, the fix for the symlink bug also added some debug logging I believe, right ianw ? | 00:12 |
| *** rcarrillocruz has left #openstack-infra | 00:12 | |
| ianw | some echos on what was being unmounted | 00:12 |
| greghaynes | awesome, so I should release dib | 00:12 |
| *** angdraug has quit IRC | 00:13 | |
| jeblair | nodepool is running again | 00:13 |
| ianw | greghaynes: yep :) i'm just looking now at what's required to get puppet-diskimage_builder an option to install from git | 00:13 |
| *** aorourke is now known as aorourke-afk | 00:14 | |
| *** hdd has joined #openstack-infra | 00:15 | |
| *** Sukhdev has quit IRC | 00:18 | |
| jeblair | i have deleted the rows from the dib_image mysql table that are in state=1 | 00:20 |
| jeblair | greghaynes: hopefully once deletedibimage gets smarter as we discussed, it can handle that... though also i believe we discussed the idea that nodepool should mark those for deletion when it starts up? | 00:21 |
| greghaynes | jeblair: yep, I was hoping the periodic cleanup would just run on start | 00:21 |
| openstackgerrit | Travis Tripp proposed openstack-infra/project-config: Add searchlight-specs to specs index https://review.openstack.org/273830 | 00:21 |
| jeblair | huh | 00:22 |
| jeblair | i think rax got both dib fedora23 and snapshot fedora23 images | 00:22 |
| openstackgerrit | Merged openstack-infra/project-config: Fix permission error in ready script https://review.openstack.org/273823 | 00:24 |
| jeblair | okay the ready script has been updated on nodepool.o.o | 00:26 |
| *** yamamoto_ has quit IRC | 00:27 | |
| ianw | jeblair: there are no snapshot f23 images? | 00:27 |
| jeblair | ianw: then maybe something very weird happened 14 days ago | 00:28 |
| jeblair | | 354595 | rax-dfw | fedora-23 | fedora-23 | 1453506949 | d80fa260-f48c-40cf-927b-b882332ac2cc | None | ready | 05:23:53:55 | | 00:28 |
| jeblair | | 354377 | rax-dfw | fedora-23 | template-fedora-23-1452784784 | 1452784784 | 9e7db059-919c-4b9d-bd41-8374b36b038e | None | ready | 14:04:36:38 | | 00:28 |
| jeblair | it's true it doesn't have a server id, but the hostname looks like it was from a template. | 00:29 |
| jeblair | shrug. | 00:29 |
| *** denisra has quit IRC | 00:29 | |
| jeblair | i believe i have deleted all of the bad images | 00:29 |
| openstackgerrit | Matthew Thode proposed openstack/diskimage-builder: package-installs doesn't work in python3/gentoo https://review.openstack.org/273782 | 00:29 |
| openstackgerrit | Ian Wienand proposed openstack-infra/puppet-diskimage_builder: Add a use_git flag https://review.openstack.org/273832 | 00:30 |
| *** denisra has joined #openstack-infra | 00:30 | |
| jeblair | nodepool is uploading a centos7 image to gra1, it's possible it might be bad, but i don't think it will have a huge impact | 00:30 |
| jeblair | nodepool will not build any further images until the daily rebuild cycle | 00:31 |
| jeblair | and we're close enough to the end of my day that i can't sit through another one | 00:31 |
| jeblair | if there is another infra-root around that would like to babysit them, feel free | 00:31 |
| *** acruz has quit IRC | 00:31 | |
| jeblair | otherwise, we should either (a) start using the afs mirrors some time after 1400 tomorrow, or (b) stop being able to build new nodes some time after 1400 tomorrow | 00:32 |
| openstackgerrit | Matthew Thode proposed openstack/diskimage-builder: add gentoo support to growroot https://review.openstack.org/273769 | 00:32 |
| *** tdurakov_ has quit IRC | 00:32 | |
| *** tdurakov_ has joined #openstack-infra | 00:33 | |
| jeblair | greghaynes: would you mind reviewing https://review.openstack.org/272849 ? it's running in production and i think we need it for our sanity. | 00:33 |
| jhesketh | jeblair: I can baby-sit new images, but I haven't been following what's happening.. I assume you mean rebuild centos7 if the one that is currently uploading turns out to be bad? | 00:34 |
| jeblair | ianw: ^ lost your +2 | 00:34 |
| greghaynes | jeblair: ok | 00:34 |
| *** ZZelle_ has quit IRC | 00:34 | |
| *** tdurakov_ has quit IRC | 00:35 | |
| openstackgerrit | Tim Birkett proposed openstack-infra/jenkins-job-builder: Add trigger for RabbitMQ trigger module https://review.openstack.org/273833 | 00:35 |
| jeblair | jhesketh: i _think_ the centos7 upload in progress is okay because i think it predates the bad ready script. even if it is bad, i'm not worried about it; we don't use that many centos7 nodes and we have other sources. | 00:37 |
| *** Sukhdev has joined #openstack-infra | 00:37 | |
| jeblair | jhesketh: i meant if anyone wants to babysit a full set of image rebuilds to start using the afs mirrors | 00:37 |
| jhesketh | ah right | 00:37 |
| *** Daisy has joined #openstack-infra | 00:37 | |
| jhesketh | hmm, a full set of rebuilds sound scary given that I haven't done any work on the afs mirrors | 00:38 |
| jhesketh | unless you think it'll be smooth or really want it tonight, it might be best I leave that for somebody else | 00:38 |
| *** thorst has joined #openstack-infra | 00:39 | |
| *** abregman|afk has quit IRC | 00:41 | |
| *** thorst has quit IRC | 00:43 | |
| *** tdurakov_ has joined #openstack-infra | 00:43 | |
| jhesketh | jeblair: the multi-thread lgtm | 00:46 |
| greghaynes | jeblair: have you tried calling stop() on the builder with that patch? | 00:47 |
| greghaynes | jeblair: I think we dont have a way to do that when running as its own process... | 00:47 |
| greghaynes | but I syspect it will hang | 00:47 |
| greghaynes | suspect, even | 00:47 |
| jeblair | greghaynes: there is a stop/start test | 00:48 |
| jeblair | possibly even a start/stop test | 00:48 |
| ianw | pabelanger: remind me again how to do something like run puppet-lint locally? bundle / bundler something? | 00:48 |
| *** ayoung has joined #openstack-infra | 00:49 | |
| greghaynes | jeblair: hrm, I dont see where self._running will ever get set to false, and we have a while self._running: sleep in stop() | 00:49 |
| *** chlong has joined #openstack-infra | 00:50 | |
| jhesketh | greghaynes: line 238 | 00:50 |
| jhesketh | greghaynes: also line 90 | 00:50 |
| *** agentle has quit IRC | 00:51 | |
| greghaynes | left comments on patch, I dont see that on line 238 on the new side... | 00:51 |
| openstackgerrit | Matthew Thode proposed openstack/diskimage-builder: add support for openrc to dib-init-system https://review.openstack.org/273772 | 00:51 |
| *** agentle has joined #openstack-infra | 00:51 | |
| openstackgerrit | Merged openstack-infra/nodepool: Make builder multi-threaded https://review.openstack.org/272849 | 00:52 |
| openstackgerrit | Matthew Thode proposed openstack/diskimage-builder: add support for openrc to dib-init-system https://review.openstack.org/273772 | 00:52 |
| openstackgerrit | Matthew Thode proposed openstack/diskimage-builder: fix gentoo hardened support https://review.openstack.org/273795 | 00:52 |
| *** angdraug has joined #openstack-infra | 00:52 | |
| greghaynes | oh derp | 00:52 |
| jhesketh | greghaynes: sorry, my bad, I had hit +W before your comments | 00:52 |
| greghaynes | jhesketh: heh, its fine, my only valid comment was a nit which we can fix later | 00:53 |
| jhesketh | greghaynes: yeah I was about to say it looks like something we can do as a follow-up | 00:53 |
| *** diana_clarke has quit IRC | 00:54 | |
| jhesketh | greghaynes: I don't understand what you mean above " I dont see that on line 238 on the new side..." | 00:54 |
| jhesketh | did you mean to say you didn't see that? | 00:54 |
| greghaynes | jhesketh: ya, I just glanced over where we do the join on all the worker threads in runForever | 00:54 |
| prometheanfire | ianw: around? | 00:54 |
| ianw | prometheanfire: yep | 00:55 |
| openstackgerrit | Merged openstack-infra/nodepool: Builder: Also unregister image-delete https://review.openstack.org/273203 | 00:55 |
| jhesketh | greghaynes: because the workers are stopped, the threads will all join and then self._running will be false... it looks correct to me | 00:55 |
| prometheanfire | about https://review.openstack.org/#/c/273790/ | 00:55 |
| jhesketh | greghaynes: do you want to propose what you meant? | 00:56 |
| greghaynes | jhesketh: yep, I just overlooked that | 00:56 |
| jhesketh | greghaynes: oh so your comments in the change are incorrect? | 00:56 |
| *** agentle has quit IRC | 00:56 | |
| greghaynes | jhesketh: Yep, I left another comment about that ;) The nit one is correct but not a big deal | 00:56 |
| jhesketh | ah cool, I see | 00:56 |
| jhesketh | yes, the _stop_running appears unusued | 00:57 |
| *** diana_clarke has joined #openstack-infra | 00:57 | |
| *** mtanino has quit IRC | 00:57 | |
| *** marcusvrn_ has quit IRC | 00:57 | |
| *** SumitNaiksatam has quit IRC | 00:57 | |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/nodepool: Remove unused _stop_running variable https://review.openstack.org/273836 | 00:58 |
| jhesketh | greghaynes: ^ | 00:58 |
| greghaynes | jhesketh: cool | 00:58 |
| prometheanfire | say you set your install type to source, so you have https://github.com/openstack/diskimage-builder/blob/master/elements/simple-init/source-repository-simple-init installed | 00:58 |
| mgagne | When running nodepool dib-image-delete, command gets stuck and I can't CTRL+C :-/ | 00:59 |
| prometheanfire | that means https://github.com/openstack/diskimage-builder/blob/master/elements/simple-init/install.d/simple-init-source-install/40-glean gets run | 00:59 |
| prometheanfire | but https://github.com/openstack/diskimage-builder/blob/master/elements/simple-init/install.d/simple-init-source-install/40-glean is not a source install, it's a package install | 01:00 |
| prometheanfire | if you set your install type to repo then the git repo won't be laid down and https://github.com/openstack/diskimage-builder/blob/master/elements/simple-init/install.d/simple-init-repo-install/40-glean will fail | 01:00 |
| prometheanfire | ianw: ^ | 01:00 |
| prometheanfire | ianw: repo should probably get renamed package as well, to be clear | 01:01 |
| ianw | prometheanfire: yeah, but "source" means pip? and repo means git -> e.g. https://git.openstack.org/cgit/openstack-infra/project-config/tree/nodepool/nodepool.yaml#n606 | 01:01 |
| ianw | i'm not going to argue the names aren't confusing :) | 01:01 |
| prometheanfire | the git repo get laid down and installed? | 01:02 |
| *** ccrouch has quit IRC | 01:02 | |
| prometheanfire | it's one or the other | 01:02 |
| prometheanfire | not both | 01:02 |
| greghaynes | prometheanfire: package == system package, source == pip, repo == git | 01:02 |
| ianw | well, i mean if you say "DIB_INSTALLTYPE_simple_init: repo" then you want it to get from git | 01:02 |
| prometheanfire | https://github.com/openstack/diskimage-builder/blob/master/elements/source-repositories/extra-data.d/98-source-repositories#L96-L99 | 01:03 |
| prometheanfire | that doesn't cache the repo unless it's set | 01:03 |
| *** ccrouch has joined #openstack-infra | 01:04 | |
| prometheanfire | I set INSTALL_TYPE=source, the git repo get's laid down and the pip package installed? | 01:04 |
| prometheanfire | that seems wrong | 01:04 |
| prometheanfire | I set INSTALL_TYPE to repo, git repo doesn't get cloned, and the pip install fails because it's refrencing a non-existant location | 01:04 |
| *** rajinir has quit IRC | 01:05 | |
| prometheanfire | it seems like INSTALL_TYPE was meant to be user settable globally | 01:06 |
| *** tdurakov_ has quit IRC | 01:06 | |
| greghaynes | prometheanfire: install type does not effect source-repositories | 01:06 |
| ianw | prometheanfire: so, if you want simple-init from source, you want to set DIB_INSTALLTYPE_simple_init=repo | 01:07 |
| prometheanfire | yes it does, it effects fetching them | 01:07 |
| ianw | that will pick up the *-repo* intsall file, and install from /tmp, which is the git checkout | 01:07 |
| greghaynes | prometheanfire: oh, huh, thats new to me | 01:07 |
| * greghaynes looks for when that got added | 01:07 | |
| *** tdurakov_ has joined #openstack-infra | 01:08 | |
| prometheanfire | ya, https://github.com/openstack/diskimage-builder/blob/master/elements/source-repositories/extra-data.d/98-source-repositories#L94-L99 | 01:08 |
| prometheanfire | it's probably just that code that needs to be cleaned up | 01:08 |
| prometheanfire | ianw: I do ack that what you say works, I just encountered a bug around that code and 'fixed it wrong' | 01:08 |
| greghaynes | TBH that whole thing needs to go away and get replaced with zuul-cloner | 01:08 |
| prometheanfire | heh | 01:09 |
| greghaynes | but it hasn't been super high on my priority list | 01:09 |
| prometheanfire | it does seem overly complicated | 01:09 |
| prometheanfire | I'll undo that portion of the patch then | 01:09 |
| *** yamahata has joined #openstack-infra | 01:10 | |
| greghaynes | Yea, so that is a hangover from tripleo where source =='d run git clone | 01:11 |
| openstackgerrit | Matthew Thode proposed openstack/diskimage-builder: add support for gentoo to simple-init https://review.openstack.org/273790 | 01:12 |
| *** agentle has joined #openstack-infra | 01:12 | |
| *** amotoki has joined #openstack-infra | 01:12 | |
| ianw | [iwienand@fedora19 puppet-diskimage_builder (git-install)]$ bundle exec rake lint | 01:13 |
| ianw | rake aborted! | 01:13 |
| ianw | NameError: uninitialized constant Syck | 01:13 |
| ianw | does anyone understand that ^ | 01:13 |
| *** diana_clarke has quit IRC | 01:13 | |
| *** jpr has quit IRC | 01:14 | |
| *** amotoki has quit IRC | 01:15 | |
| mgagne | ianw all I found is something related to bad yaml | 01:16 |
| *** diana_clarke has joined #openstack-infra | 01:17 | |
| *** agentle has quit IRC | 01:17 | |
| prometheanfire | ianw: ok, I think I addressed all your comments from the reviews | 01:17 |
| *** piet_ has quit IRC | 01:17 | |
| ianw | mgagne: yeah, i'm guessing my ruby install system is "syck" | 01:18 |
| prometheanfire | well, it's ruby | 01:18 |
| mgagne | yea, I don't miss it much today =) | 01:18 |
| prometheanfire | that said, we do use puppet in gentoo's infra | 01:19 |
| prometheanfire | openstack-infra running puppet-server? | 01:19 |
| mgagne | anyone with broken nodepool where you can't issue dib-image-delete without cli hanging forever? | 01:19 |
| mgagne | maybe I'm missing a new config introduced with latest nodepool... | 01:20 |
| *** tdurakov_ has quit IRC | 01:21 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/puppet-diskimage_builder: Add a use_git flag https://review.openstack.org/273832 | 01:21 |
| *** angdraug has quit IRC | 01:22 | |
| *** yamamoto_ has joined #openstack-infra | 01:23 | |
| *** Qiming has quit IRC | 01:24 | |
| *** pvaneck has quit IRC | 01:24 | |
| *** piet has joined #openstack-infra | 01:25 | |
| *** sdake_ has quit IRC | 01:27 | |
| *** amotoki has joined #openstack-infra | 01:28 | |
| *** rlandy has quit IRC | 01:29 | |
| *** zul has joined #openstack-infra | 01:30 | |
| *** Qiming has joined #openstack-infra | 01:31 | |
| *** hockeynut has quit IRC | 01:31 | |
| *** hughsaunders has quit IRC | 01:32 | |
| *** mgagne has quit IRC | 01:32 | |
| *** mgagne has joined #openstack-infra | 01:32 | |
| *** mgagne has quit IRC | 01:32 | |
| *** mgagne has joined #openstack-infra | 01:32 | |
| *** hughsaunders has joined #openstack-infra | 01:32 | |
| *** hockeynut has joined #openstack-infra | 01:34 | |
| *** pradk has quit IRC | 01:35 | |
| *** rvasilets___ has quit IRC | 01:36 | |
| *** yamamoto_ has quit IRC | 01:37 | |
| *** Qiming has quit IRC | 01:38 | |
| *** dalgaaf has quit IRC | 01:38 | |
| *** fawadkhaliq has joined #openstack-infra | 01:38 | |
| *** fawadkhaliq has quit IRC | 01:38 | |
| mgagne | ok, I had to manually delete the entry in the nodepool database. the image was building but associated DIB folder didn't exist, probably due to catastrophic reboot of the server during build. | 01:39 |
| *** kzaitsev_mb has quit IRC | 01:40 | |
| *** tdurakov_ has joined #openstack-infra | 01:41 | |
| *** jbryce has quit IRC | 01:43 | |
| *** jbryce has joined #openstack-infra | 01:44 | |
| *** amotoki has quit IRC | 01:48 | |
| *** Jeffrey4l has joined #openstack-infra | 01:48 | |
| *** hdd has quit IRC | 01:49 | |
| *** aeng has quit IRC | 01:51 | |
| *** pradk has joined #openstack-infra | 01:51 | |
| *** jmccrory has joined #openstack-infra | 01:53 | |
| *** flwang1 has joined #openstack-infra | 01:53 | |
| *** Sukhdev has quit IRC | 01:56 | |
| *** flwang1 has quit IRC | 01:58 | |
| *** Daisy has quit IRC | 01:58 | |
| *** tiswanso has joined #openstack-infra | 01:59 | |
| *** tiswanso has quit IRC | 01:59 | |
| *** tiswanso has joined #openstack-infra | 02:00 | |
| *** Daisy has joined #openstack-infra | 02:01 | |
| openstackgerrit | Merged openstack-infra/nodepool: Remove unused _stop_running variable https://review.openstack.org/273836 | 02:01 |
| *** agentle has joined #openstack-infra | 02:02 | |
| *** moravec has joined #openstack-infra | 02:04 | |
| *** gildub has quit IRC | 02:07 | |
| *** yamahata has quit IRC | 02:08 | |
| *** amotoki has joined #openstack-infra | 02:08 | |
| *** changbl has quit IRC | 02:09 | |
| *** amotoki_ has joined #openstack-infra | 02:11 | |
| *** amotoki has quit IRC | 02:13 | |
| *** zul has quit IRC | 02:13 | |
| *** Qiming has joined #openstack-infra | 02:15 | |
| *** amotoki_ has quit IRC | 02:16 | |
| *** bpokorny_ has quit IRC | 02:16 | |
| *** SumitNaiksatam has joined #openstack-infra | 02:16 | |
| *** woodster_ has quit IRC | 02:16 | |
| *** flwang1 has joined #openstack-infra | 02:17 | |
| *** amotoki has joined #openstack-infra | 02:19 | |
| *** agentle has quit IRC | 02:22 | |
| *** yamamoto_ has joined #openstack-infra | 02:23 | |
| *** moravec has quit IRC | 02:26 | |
| *** changzhi has joined #openstack-infra | 02:28 | |
| *** beisner- has joined #openstack-infra | 02:29 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/puppet-diskimage_builder: Add a use_git flag https://review.openstack.org/273832 | 02:29 |
| *** amotoki has quit IRC | 02:29 | |
| *** beisner has quit IRC | 02:29 | |
| *** beisner- is now known as beisner | 02:29 | |
| *** unicell has quit IRC | 02:30 | |
| *** pkoniszewski has quit IRC | 02:31 | |
| *** pkoniszewski has joined #openstack-infra | 02:31 | |
| *** apoorvad has quit IRC | 02:32 | |
| *** vivekd has joined #openstack-infra | 02:32 | |
| *** Daisy has quit IRC | 02:33 | |
| *** coolsvap|away is now known as coolsvap | 02:36 | |
| *** tdurakov_ has quit IRC | 02:40 | |
| *** gildub has joined #openstack-infra | 02:41 | |
| *** jpr has joined #openstack-infra | 02:42 | |
| *** Shrews has quit IRC | 02:50 | |
| *** mgagne has quit IRC | 02:51 | |
| *** mgagne has joined #openstack-infra | 02:51 | |
| *** baoli has joined #openstack-infra | 03:00 | |
| *** e0ne has joined #openstack-infra | 03:00 | |
| *** agentle has joined #openstack-infra | 03:02 | |
| *** dingyichen has quit IRC | 03:05 | |
| *** tlian has quit IRC | 03:07 | |
| *** agentle has quit IRC | 03:08 | |
| *** baoli has quit IRC | 03:11 | |
| *** amotoki has joined #openstack-infra | 03:11 | |
| mgagne | dmsimard what's up with all those people instantly liking whatever I write on twitter o_O | 03:11 |
| *** dims has quit IRC | 03:11 | |
| *** wei-wang has joined #openstack-infra | 03:12 | |
| *** baoli has joined #openstack-infra | 03:12 | |
| *** achanda has joined #openstack-infra | 03:13 | |
| *** dingyichen has joined #openstack-infra | 03:14 | |
| *** hdd has joined #openstack-infra | 03:15 | |
| *** pradk has quit IRC | 03:16 | |
| *** vivekd has quit IRC | 03:17 | |
| openstackgerrit | Masayuki Igawa proposed openstack/requirements: Add openstack-health requirements libraries https://review.openstack.org/273856 | 03:17 |
| *** e0ne has quit IRC | 03:19 | |
| *** signed8__ has quit IRC | 03:20 | |
| *** keedya has joined #openstack-infra | 03:21 | |
| *** wei-wang has quit IRC | 03:23 | |
| *** hdd has quit IRC | 03:23 | |
| *** wei-wang has joined #openstack-infra | 03:24 | |
| *** pradk has joined #openstack-infra | 03:28 | |
| *** coolsvap is now known as coolsvap|away | 03:30 | |
| *** sridhar_ram has quit IRC | 03:32 | |
| *** yuanying_ has joined #openstack-infra | 03:38 | |
| *** yuanying has quit IRC | 03:40 | |
| *** baoli has quit IRC | 03:43 | |
| *** moravec has joined #openstack-infra | 03:43 | |
| *** agentle has joined #openstack-infra | 03:44 | |
| *** sdake has joined #openstack-infra | 03:45 | |
| *** Daisy has joined #openstack-infra | 03:47 | |
| *** dingyichen has quit IRC | 03:48 | |
| *** keedya has quit IRC | 03:49 | |
| *** agentle has quit IRC | 03:49 | |
| *** diana_clarke has quit IRC | 03:49 | |
| *** agentle has joined #openstack-infra | 03:49 | |
| *** coolsvap|away is now known as coolsvap | 03:49 | |
| *** Daisy has quit IRC | 03:51 | |
| *** agentle has quit IRC | 03:54 | |
| *** baoli has joined #openstack-infra | 03:54 | |
| *** baoli has quit IRC | 03:55 | |
| *** baoli has joined #openstack-infra | 03:55 | |
| *** dingyichen has joined #openstack-infra | 03:56 | |
| *** dingyichen has quit IRC | 03:58 | |
| *** vivekd has joined #openstack-infra | 04:01 | |
| openstackgerrit | greghaynes proposed openstack-infra/nodepool: Test for gear disconnect with queued builds https://review.openstack.org/273703 | 04:02 |
| *** achanda has quit IRC | 04:03 | |
| *** yuanying has joined #openstack-infra | 04:06 | |
| *** yuanying_ has quit IRC | 04:06 | |
| *** salv-orlando has joined #openstack-infra | 04:07 | |
| *** salv-orlando has quit IRC | 04:07 | |
| *** piet has quit IRC | 04:07 | |
| *** salv-orl_ has quit IRC | 04:09 | |
| *** yuanying has quit IRC | 04:11 | |
| *** dingyichen has joined #openstack-infra | 04:16 | |
| openstackgerrit | Matthew Thode proposed openstack/diskimage-builder: Add support for OpenRC to dib-init-system https://review.openstack.org/273772 | 04:17 |
| prometheanfire | ianw: thanks for the reviews, updated https://review.openstack.org/273772 | 04:17 |
| *** yuanying has joined #openstack-infra | 04:17 | |
| *** dahoo has quit IRC | 04:22 | |
| *** baoli has quit IRC | 04:24 | |
| *** hdd has joined #openstack-infra | 04:25 | |
| *** piet has joined #openstack-infra | 04:26 | |
| *** wei-wang has quit IRC | 04:27 | |
| openstackgerrit | Merged openstack/diskimage-builder: Fix debian-minimal image building https://review.openstack.org/273544 | 04:32 |
| openstackgerrit | Merged openstack/diskimage-builder: Don't use wc -l for the umount check https://review.openstack.org/273386 | 04:35 |
| *** tiswanso has quit IRC | 04:36 | |
| *** markvoelker has quit IRC | 04:38 | |
| openstackgerrit | Dong Ma proposed openstack-infra/jenkins-job-builder: Add doxygen builder support https://review.openstack.org/131330 | 04:42 |
| *** achanda has joined #openstack-infra | 04:42 | |
| prometheanfire | woot, finally passed both... | 04:43 |
| prometheanfire | https://review.openstack.org/#/c/273790/ | 04:43 |
| *** vivekd has quit IRC | 04:46 | |
| *** camunoz_ has quit IRC | 04:51 | |
| openstackgerrit | greghaynes proposed openstack-infra/nodepool: Cleanup bulder jobs and add disconnect support https://review.openstack.org/272384 | 04:53 |
| openstackgerrit | greghaynes proposed openstack-infra/nodepool: Test for gear disconnect with queued builds https://review.openstack.org/273703 | 04:53 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder: Only match #!/bin/bash in scripts https://review.openstack.org/273879 | 04:53 |
| ianw | prometheanfire: ^ i wondered why dib-lint wasn't pick them up | 04:56 |
| ianw | dib-lint could really do with a bit of a cleanup | 04:56 |
| greghaynes | heh, are you sensing a theme? | 04:57 |
| prometheanfire | ianw: openrc init scripts don't start with /bin/bash | 04:57 |
| prometheanfire | and can't be | 04:57 |
| prometheanfire | ianw: https://review.openstack.org/#/c/273769/3/elements/growroot/init-scripts/openrc/growroot | 04:58 |
| *** thorst has joined #openstack-infra | 04:59 | |
| ianw | prometheanfire: that's ok, because it's not bash | 04:59 |
| *** thorst has quit IRC | 04:59 | |
| *** thorst has joined #openstack-infra | 05:00 | |
| prometheanfire | ah, I see it now | 05:00 |
| ianw | gerrit 503 sad :( | 05:01 |
| *** sflanigan has joined #openstack-infra | 05:01 | |
| *** sflanigan has quit IRC | 05:01 | |
| *** sflanigan has joined #openstack-infra | 05:01 | |
| prometheanfire | you doubleposted :P | 05:03 |
| prometheanfire | I changed to +1 though | 05:03 |
| prometheanfire | your patch will probably conflict with mine though :( | 05:03 |
| *** thorst has quit IRC | 05:04 | |
| *** vgridnev has joined #openstack-infra | 05:04 | |
| prometheanfire | actually, that's fine | 05:05 |
| prometheanfire | no conflict, nice | 05:06 |
| *** salv-orlando has joined #openstack-infra | 05:08 | |
| openstackgerrit | Matthew Thode proposed openstack/diskimage-builder: fix gentoo hardened support https://review.openstack.org/273795 | 05:10 |
| prometheanfire | ianw: and fixed that review | 05:11 |
| *** amotoki_ has joined #openstack-infra | 05:11 | |
| *** Sukhdev has joined #openstack-infra | 05:11 | |
| *** chlong has quit IRC | 05:12 | |
| prometheanfire | removed the /usr/bin/env I was adding | 05:12 |
| *** salv-orlando has quit IRC | 05:13 | |
| *** coolsvap is now known as coolsvap|away | 05:15 | |
| *** dahoo has joined #openstack-infra | 05:17 | |
| *** jaosorior has joined #openstack-infra | 05:20 | |
| *** piet has quit IRC | 05:21 | |
| *** baoli has joined #openstack-infra | 05:24 | |
| *** krtaylor has quit IRC | 05:25 | |
| *** exploreshaifali has joined #openstack-infra | 05:29 | |
| *** baoli has quit IRC | 05:30 | |
| *** thorst has joined #openstack-infra | 05:30 | |
| *** chlong has joined #openstack-infra | 05:33 | |
| *** maishsk has quit IRC | 05:34 | |
| *** krtaylor has joined #openstack-infra | 05:37 | |
| *** Daisy has joined #openstack-infra | 05:39 | |
| *** markvoelker has joined #openstack-infra | 05:39 | |
| *** thorst has quit IRC | 05:41 | |
| *** markvoelker has quit IRC | 05:43 | |
| *** rguillebert has quit IRC | 05:47 | |
| *** amotoki_ has quit IRC | 05:47 | |
| *** vgridnev has quit IRC | 05:49 | |
| *** Daisy has quit IRC | 05:52 | |
| *** Daisy has joined #openstack-infra | 05:56 | |
| *** asselin_ has quit IRC | 05:57 | |
| *** Daisy has quit IRC | 06:00 | |
| *** vivekd has joined #openstack-infra | 06:02 | |
| openstackgerrit | Masayuki Igawa proposed openstack/requirements: Add openstack-health requirements libraries https://review.openstack.org/273856 | 06:04 |
| *** thorst has joined #openstack-infra | 06:04 | |
| *** pahuang has quit IRC | 06:07 | |
| *** pahuang has joined #openstack-infra | 06:11 | |
| *** thorst has quit IRC | 06:18 | |
| *** sdake has quit IRC | 06:19 | |
| *** vgridnev has joined #openstack-infra | 06:22 | |
| *** tangchen has quit IRC | 06:27 | |
| *** chlong has quit IRC | 06:28 | |
| *** rcernin has quit IRC | 06:29 | |
| *** coolsvap|away is now known as coolsvap | 06:31 | |
| *** yamahata has joined #openstack-infra | 06:33 | |
| *** salv-orlando has joined #openstack-infra | 06:34 | |
| *** amotoki has quit IRC | 06:39 | |
| *** chlong has joined #openstack-infra | 06:39 | |
| *** amotoki has joined #openstack-infra | 06:39 | |
| *** jaosorior has quit IRC | 06:40 | |
| *** maishsk has joined #openstack-infra | 06:41 | |
| *** amotoki has quit IRC | 06:43 | |
| *** amotoki has joined #openstack-infra | 06:43 | |
| *** Jeffrey4l has quit IRC | 06:44 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/requirements: Updated from generate-constraints https://review.openstack.org/273907 | 06:44 |
| AJaeger | ianw: do you have time to review https://review.openstack.org/273522 and https://review.openstack.org/270697, please? | 06:44 |
| *** unicell has joined #openstack-infra | 06:47 | |
| *** jaosorior has joined #openstack-infra | 06:47 | |
| *** maishsk has quit IRC | 06:50 | |
| *** vgridnev has quit IRC | 06:51 | |
| *** flwang1 has quit IRC | 06:51 | |
| *** maishsk has joined #openstack-infra | 06:53 | |
| *** Jeffrey4l has joined #openstack-infra | 06:57 | |
| *** vgridnev has joined #openstack-infra | 06:58 | |
| *** flepied has quit IRC | 07:00 | |
| *** xek_ has quit IRC | 07:00 | |
| *** _nadya_ has joined #openstack-infra | 07:00 | |
| *** xek_ has joined #openstack-infra | 07:01 | |
| *** _nadya_ has quit IRC | 07:01 | |
| *** flepied has joined #openstack-infra | 07:01 | |
| *** flepied has left #openstack-infra | 07:02 | |
| openstackgerrit | Doug Wiegley proposed openstack/requirements: Add neutron-lib to projects.txt https://review.openstack.org/273913 | 07:02 |
| *** flepied has joined #openstack-infra | 07:03 | |
| *** Thelo has quit IRC | 07:03 | |
| *** Thelo has joined #openstack-infra | 07:06 | |
| *** vgridnev has quit IRC | 07:06 | |
| *** maishsk has quit IRC | 07:07 | |
| *** flepied has quit IRC | 07:09 | |
| *** vgridnev has joined #openstack-infra | 07:10 | |
| *** _nadya_ has joined #openstack-infra | 07:10 | |
| *** scheuran has joined #openstack-infra | 07:11 | |
| *** Daisy has joined #openstack-infra | 07:12 | |
| *** Jeffrey4l has quit IRC | 07:14 | |
| *** dahoo has quit IRC | 07:15 | |
| *** cgross has joined #openstack-infra | 07:16 | |
| *** maishsk has joined #openstack-infra | 07:18 | |
| openstackgerrit | xiexs proposed openstack-infra/nodepool: Add ipv6-preferred into config-validation https://review.openstack.org/232981 | 07:18 |
| *** hdd has quit IRC | 07:20 | |
| *** k4n0 has joined #openstack-infra | 07:22 | |
| *** maishsk has quit IRC | 07:22 | |
| *** amotoki has quit IRC | 07:22 | |
| *** amotoki has joined #openstack-infra | 07:24 | |
| *** k4n0 has quit IRC | 07:25 | |
| *** rcernin has joined #openstack-infra | 07:25 | |
| *** salv-orlando has quit IRC | 07:26 | |
| openstackgerrit | xiexs proposed openstack-infra/nodepool: Add ipv6-preferred into config-validation https://review.openstack.org/232981 | 07:27 |
| *** chlong has quit IRC | 07:32 | |
| *** Jeffrey4l has joined #openstack-infra | 07:37 | |
| *** Liuqing has joined #openstack-infra | 07:38 | |
| openstackgerrit | greghaynes proposed openstack/diskimage-builder: WIP: Move hook generation in to python https://review.openstack.org/271139 | 07:39 |
| *** markvoelker has joined #openstack-infra | 07:40 | |
| *** TechNick99 has joined #openstack-infra | 07:41 | |
| *** markvoelker has quit IRC | 07:44 | |
| *** amotoki has quit IRC | 07:49 | |
| *** Sukhdev has quit IRC | 07:50 | |
| *** agentle has joined #openstack-infra | 07:51 | |
| *** amotoki has joined #openstack-infra | 07:51 | |
| openstackgerrit | greghaynes proposed openstack/diskimage-builder: Move hook generation in to python https://review.openstack.org/271139 | 07:54 |
| openstackgerrit | greghaynes proposed openstack/diskimage-builder: Move hook generation in to python https://review.openstack.org/271139 | 07:55 |
| *** agentle has quit IRC | 07:55 | |
| *** boris-42 has joined #openstack-infra | 07:58 | |
| *** kushal has quit IRC | 08:01 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/infra-manual: WIP: Add information about enabling translations https://review.openstack.org/273759 | 08:07 |
| *** abregman|afk has joined #openstack-infra | 08:09 | |
| *** Daisy has quit IRC | 08:11 | |
| *** tlian has joined #openstack-infra | 08:11 | |
| *** arxcruz has joined #openstack-infra | 08:11 | |
| *** tzn has joined #openstack-infra | 08:16 | |
| *** _nadya_ has quit IRC | 08:16 | |
| *** gmann has quit IRC | 08:16 | |
| *** exploreshaifali has quit IRC | 08:16 | |
| *** Daisy has joined #openstack-infra | 08:17 | |
| *** oomichi has quit IRC | 08:17 | |
| *** gmann has joined #openstack-infra | 08:17 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/infra-manual: WIP: Add information about enabling translations https://review.openstack.org/273759 | 08:19 |
| *** achanda has quit IRC | 08:19 | |
| *** achanda has joined #openstack-infra | 08:20 | |
| *** ihrachys has joined #openstack-infra | 08:21 | |
| *** skraynev has quit IRC | 08:21 | |
| *** yaume has joined #openstack-infra | 08:23 | |
| *** kushal has joined #openstack-infra | 08:23 | |
| *** skraynev has joined #openstack-infra | 08:23 | |
| *** tzn has quit IRC | 08:24 | |
| *** flepied has joined #openstack-infra | 08:26 | |
| *** pblaho has joined #openstack-infra | 08:26 | |
| *** abregman|afk has quit IRC | 08:26 | |
| *** tzn has joined #openstack-infra | 08:27 | |
| *** tzn has quit IRC | 08:27 | |
| *** ihrachys has quit IRC | 08:31 | |
| *** samuelBartel has joined #openstack-infra | 08:32 | |
| *** achanda has quit IRC | 08:33 | |
| *** mikelk has joined #openstack-infra | 08:34 | |
| openstackgerrit | David Caro proposed openstack-infra/jenkins-job-builder: Added github-pull-request-merge publisher https://review.openstack.org/273167 | 08:35 |
| *** vgridnev has quit IRC | 08:36 | |
| *** jcoufal has joined #openstack-infra | 08:37 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack-infra/project-config: Remove gate-openstack-ansible-lint https://review.openstack.org/272731 | 08:37 |
| *** jlanoux has joined #openstack-infra | 08:38 | |
| yolanda | morning | 08:41 |
| *** Guest44742 has quit IRC | 08:41 | |
| nibalizer | yolanda: hello | 08:43 |
| nibalizer | yolanda: are you coming to fosdem this year? | 08:43 |
| *** vgridnev has joined #openstack-infra | 08:43 | |
| yolanda | hi nibalizer | 08:43 |
| yolanda | no, not going | 08:43 |
| nibalizer | ah sad | 08:43 |
| yolanda | i had the idea to go some time ago, but things in Belgium were not so good at that time, so i prefered to don't travel, just in case | 08:43 |
| *** Hal has joined #openstack-infra | 08:44 | |
| yolanda | nibalizer, when is it? early february? | 08:44 |
| *** Hal is now known as Guest73338 | 08:45 | |
| *** zeih has joined #openstack-infra | 08:46 | |
| *** amotoki has quit IRC | 08:46 | |
| *** matrohon has joined #openstack-infra | 08:46 | |
| openstackgerrit | Merged openstack-infra/project-config: Blacklist some more repos for translation sync https://review.openstack.org/273522 | 08:47 |
| nibalizer | yolanda: this weekend | 08:47 |
| nibalizer | tomorrow | 08:47 |
| yolanda | are you there now? | 08:48 |
| *** mikelk has quit IRC | 08:48 | |
| nibalizer | in ams | 08:48 |
| *** mikelk has joined #openstack-infra | 08:48 | |
| yolanda | ah nice | 08:51 |
| yolanda | i have it so near, and never gone there | 08:51 |
| yolanda | well, actually to the airport lots of times | 08:51 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack-infra/project-config: Remove gate-openstack-ansible-lint https://review.openstack.org/272731 | 08:52 |
| *** shardy has joined #openstack-infra | 08:55 | |
| *** derekh has joined #openstack-infra | 08:56 | |
| openstackgerrit | David Caro proposed openstack-infra/jenkins-job-builder: Adding the build-discarder property https://review.openstack.org/273952 | 08:59 |
| openstackgerrit | David Caro proposed openstack-infra/jenkins-job-builder: Adding the build-discarder property https://review.openstack.org/273952 | 09:00 |
| *** amotoki has joined #openstack-infra | 09:00 | |
| *** zul has joined #openstack-infra | 09:00 | |
| *** shardy has quit IRC | 09:01 | |
| *** shardy has joined #openstack-infra | 09:02 | |
| openstackgerrit | David Caro proposed openstack-infra/jenkins-job-builder: Adding the build-discarder property https://review.openstack.org/273952 | 09:06 |
| *** tdurakov_ has joined #openstack-infra | 09:06 | |
| *** Daisy has quit IRC | 09:07 | |
| *** jcoufal has quit IRC | 09:07 | |
| *** _nadya_ has joined #openstack-infra | 09:07 | |
| *** jcoufal has joined #openstack-infra | 09:08 | |
| *** amotoki has quit IRC | 09:08 | |
| *** vgridnev has quit IRC | 09:08 | |
| *** esikachev has joined #openstack-infra | 09:09 | |
| *** flepied has quit IRC | 09:12 | |
| yolanda | nibalizer, care reviewing https://review.openstack.org/#/c/259338/ and https://review.openstack.org/#/c/265778/ ? | 09:13 |
| *** coolsvap is now known as coolsvap|away | 09:15 | |
| *** amotoki has joined #openstack-infra | 09:17 | |
| *** amotoki has quit IRC | 09:20 | |
| *** notnownikki has joined #openstack-infra | 09:20 | |
| *** amotoki has joined #openstack-infra | 09:20 | |
| *** Daisy has joined #openstack-infra | 09:22 | |
| *** jaosorior has quit IRC | 09:22 | |
| *** jaosorior has joined #openstack-infra | 09:22 | |
| *** amotoki has quit IRC | 09:24 | |
| *** vgridnev has joined #openstack-infra | 09:28 | |
| *** tdurakov_ has quit IRC | 09:29 | |
| *** jistr has joined #openstack-infra | 09:30 | |
| *** Liuqing has quit IRC | 09:34 | |
| *** links has joined #openstack-infra | 09:38 | |
| *** links has quit IRC | 09:38 | |
| *** vgridnev has quit IRC | 09:40 | |
| *** markvoelker has joined #openstack-infra | 09:41 | |
| tsufiev | can somebody help me with understanding what gate-requirements-tox-validate job wants here: https://review.openstack.org/#/c/273750/ ? | 09:44 |
| tsufiev | I read Review Guidelines for proposing patches into openstack/requirements, yet we need to cap Selenium versions until it's fixed | 09:45 |
| *** markvoelker has quit IRC | 09:45 | |
| *** flwang1 has joined #openstack-infra | 09:46 | |
| openstackgerrit | Mateusz Matuszkowiak proposed openstack-infra/project-config: Added two new repos for fuel-plugins https://review.openstack.org/269007 | 09:47 |
| mescanef | hello roots, please review https://review.openstack.org/269007 | 09:48 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack-infra/project-config: Remove gate-openstack-ansible-lint https://review.openstack.org/272731 | 09:48 |
| *** jcoufal_ has joined #openstack-infra | 09:53 | |
| *** jcoufal has quit IRC | 09:54 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack-infra/project-config: Remove gate-openstack-ansible-lint https://review.openstack.org/272731 | 09:56 |
| *** jcoufal has joined #openstack-infra | 09:57 | |
| *** jcoufal_ has quit IRC | 10:00 | |
| *** Guest73338 has quit IRC | 10:03 | |
| *** baoli has joined #openstack-infra | 10:08 | |
| *** jaosorior has quit IRC | 10:08 | |
| *** Qiming has quit IRC | 10:08 | |
| *** jaosorior has joined #openstack-infra | 10:08 | |
| *** jordanP has joined #openstack-infra | 10:11 | |
| *** agentle has joined #openstack-infra | 10:12 | |
| *** tdurakov_ has joined #openstack-infra | 10:12 | |
| *** baoli has quit IRC | 10:13 | |
| Zara | hi! we've noticed that storyboard.openstack.org hasn't updated in about a week (you can see the difference between the webclient version here: https://storyboard.openstack.org/#!/page/about and the commit history here: https://github.com/openstack-infra/storyboard-webclient/commits ) | 10:13 |
| Zara | anyone know what's up? | 10:13 |
| *** dalgaaf has joined #openstack-infra | 10:14 | |
| *** ildikov has quit IRC | 10:14 | |
| *** yamamoto_ has quit IRC | 10:16 | |
| *** agentle has quit IRC | 10:16 | |
| nibalizer | i do sorta | 10:17 |
| nibalizer | its out of ram i thibk | 10:17 |
| nibalizer | so puppet actually crashes | 10:17 |
| AJaeger | tsufiev: if you update global-requirements, you often need to update upper-constraints as well. Are those both ok? | 10:17 |
| nibalizer | preventing updates | 10:17 |
| AJaeger | nibalizer: enjoy FOSDEM! | 10:17 |
| nibalizer | AJaeger: youu're not comming? | 10:17 |
| nibalizer | i had hoped to see you | 10:18 |
| tsufiev | AJaeger, hmm. I thought upper-constraints are updated automatically. Okay, I'll update it as well | 10:18 |
| *** maishsk has joined #openstack-infra | 10:18 | |
| AJaeger | tsufiev: yes - but the versoin in the repo always have to work together. | 10:18 |
| AJaeger | The repo has selenium===2.49.2 and you want it in global-requirements < 2.49, so that cannot work... | 10:18 |
| *** jcoufal has quit IRC | 10:18 | |
| AJaeger | both constraints are applied together | 10:19 |
| AJaeger | nibalizer: no, I won't be there | 10:19 |
| nibalizer | AJaeger: :( | 10:19 |
| *** Hal has joined #openstack-infra | 10:20 | |
| *** Hal is now known as Guest96421 | 10:21 | |
| *** Jeffrey4l has quit IRC | 10:21 | |
| SotK | nibalizer: ah, is there anything we can do easily to fix that? | 10:21 |
| nibalizer | i dont think so | 10:23 |
| nibalizer | falls on the rooters to poke and dind out qhats out of ram | 10:23 |
| nibalizer | you could help by siggibg up the cacti graph for that host and confirming that it is out of ram | 10:24 |
| *** rguillebert has joined #openstack-infra | 10:24 | |
| nibalizer | i would do it but i am on a phone and about to board a train | 10:24 |
| openstackgerrit | Timur Sufiev proposed openstack/requirements: Cap Selenium package to not exceed version 2.49 https://review.openstack.org/273750 | 10:24 |
| *** jcoufal has joined #openstack-infra | 10:25 | |
| openstackgerrit | Timur Sufiev proposed openstack/requirements: Cap Selenium package to not exceed version 2.49 https://review.openstack.org/273750 | 10:25 |
| *** tzn has joined #openstack-infra | 10:26 | |
| *** vgridnev has joined #openstack-infra | 10:27 | |
| *** rossella_s has quit IRC | 10:31 | |
| *** rossella_s has joined #openstack-infra | 10:32 | |
| *** changzhi has quit IRC | 10:33 | |
| * SotK fails to find a cacti graph for storyboard.openstack.org | 10:34 | |
| *** BobBall_AWOL is now known as BobBall | 10:34 | |
| openstackgerrit | Steven Hardy proposed openstack-infra/tripleo-ci: Override HeatWorkers parameter for deployed overcloud https://review.openstack.org/273431 | 10:36 |
| openstackgerrit | Steven Hardy proposed openstack-infra/tripleo-ci: Minimise memory usage for deployed overcloud https://review.openstack.org/273431 | 10:38 |
| *** maishsk has quit IRC | 10:38 | |
| *** tdurakov_ has quit IRC | 10:40 | |
| *** tdurakov_ has joined #openstack-infra | 10:41 | |
| yolanda | AJaeger, sorry that i cannot help so much today, but i'm on another urgent projects today | 10:41 |
| *** thorst has joined #openstack-infra | 10:42 | |
| *** MCoLo has joined #openstack-infra | 10:42 | |
| Zara | yeah, I can't find the cacti graph either | 10:42 |
| *** tdurakov_ has quit IRC | 10:43 | |
| AJaeger | thanks for the notice, yolanda ! You helped already a lot this week ;) | 10:44 |
| *** tdurakov_ has joined #openstack-infra | 10:44 | |
| yolanda | next week should be better | 10:45 |
| yolanda | or i hope | 10:45 |
| AJaeger | ;) | 10:45 |
| AJaeger | yolanda: i have the most critical stuff in and we can move some steps forward, so that's good | 10:46 |
| *** Liuqing has joined #openstack-infra | 10:48 | |
| *** dingyichen has quit IRC | 10:49 | |
| *** kzaitsev_mb has joined #openstack-infra | 10:50 | |
| *** jaypipes has joined #openstack-infra | 10:50 | |
| *** markvoelker has joined #openstack-infra | 10:56 | |
| *** Daisy has quit IRC | 10:59 | |
| *** ldnunes has joined #openstack-infra | 10:59 | |
| nibalizer | Mem: 4003 3859 143 0 6 74 | 11:00 |
| nibalizer | certainly looks out of memory | 11:00 |
| *** markvoelker has quit IRC | 11:02 | |
| *** ociuhandu has joined #openstack-infra | 11:02 | |
| *** ildikov has joined #openstack-infra | 11:03 | |
| *** salv-orlando has joined #openstack-infra | 11:03 | |
| *** salv-orlando has quit IRC | 11:04 | |
| *** salv-orl_ has joined #openstack-infra | 11:05 | |
| *** jcoufal has quit IRC | 11:05 | |
| *** Jeffrey4l has joined #openstack-infra | 11:06 | |
| *** Jeffrey4l has quit IRC | 11:06 | |
| *** sbalukoff has quit IRC | 11:06 | |
| *** Jeffrey4l has joined #openstack-infra | 11:06 | |
| *** _nadya_ has quit IRC | 11:06 | |
| *** dims has joined #openstack-infra | 11:08 | |
| *** Jeffrey4l has quit IRC | 11:08 | |
| *** Jeffrey4l has joined #openstack-infra | 11:08 | |
| *** baoli has joined #openstack-infra | 11:08 | |
| *** aysyd has joined #openstack-infra | 11:09 | |
| *** jaypipes is now known as leakypipes | 11:10 | |
| *** kzaitsev_mb has quit IRC | 11:12 | |
| *** baoli has quit IRC | 11:13 | |
| *** Qiming has joined #openstack-infra | 11:13 | |
| Zara | nibalizer: ah, just saw that. thank you! | 11:14 |
| *** tosky has joined #openstack-infra | 11:14 | |
| *** Liuqing has quit IRC | 11:15 | |
| *** zul has quit IRC | 11:15 | |
| *** maishsk has joined #openstack-infra | 11:17 | |
| *** pahuang has quit IRC | 11:17 | |
| *** mriedem has joined #openstack-infra | 11:18 | |
| openstackgerrit | Bertrand Roussel proposed openstack-infra/jenkins-job-builder: Fix default result for reverse trigger https://review.openstack.org/241802 | 11:19 |
| nibalizer | my train wifi is poo though | 11:19 |
| nibalizer | so not fixing it rright bow | 11:19 |
| nibalizer | er now | 11:20 |
| *** jcoufal has joined #openstack-infra | 11:20 | |
| openstackgerrit | Adam Coldrick proposed openstack-infra/storyboard-webclient: Fix broken link on task cards in boards https://review.openstack.org/274018 | 11:22 |
| *** sbalukoff has joined #openstack-infra | 11:22 | |
| *** thorst has quit IRC | 11:24 | |
| *** shardy has quit IRC | 11:27 | |
| Zara | heh, that's okay, we appreciate the help! :) | 11:28 |
| *** shardy has joined #openstack-infra | 11:28 | |
| *** mrmartin has joined #openstack-infra | 11:29 | |
| *** thorst has joined #openstack-infra | 11:29 | |
| *** salv-orl_ has quit IRC | 11:30 | |
| openstackgerrit | Timur Sufiev proposed openstack/requirements: Cap Selenium package to not exceed version 2.49 https://review.openstack.org/273750 | 11:30 |
| *** flwang1 has quit IRC | 11:32 | |
| *** kzaitsev_mb has joined #openstack-infra | 11:37 | |
| *** erlon has joined #openstack-infra | 11:41 | |
| *** gildub has quit IRC | 11:42 | |
| *** Daisy has joined #openstack-infra | 11:45 | |
| ansiwen | AJaeger, mordred, lifeless: this has been merged: https://github.com/python-babel/babel/pull/335 | 11:45 |
| *** jcoufal has quit IRC | 11:46 | |
| mordred | ansiwen: oh awesome! | 11:47 |
| *** thorst has quit IRC | 11:47 | |
| *** derekh has quit IRC | 11:47 | |
| ansiwen | I wonder how long it takes for this to end up in a babel release... | 11:48 |
| *** kzaitsev_ws has quit IRC | 11:49 | |
| *** Daisy has quit IRC | 11:49 | |
| *** vivekd has quit IRC | 11:49 | |
| *** kzaitsev_ws has joined #openstack-infra | 11:49 | |
| *** doug-fish has quit IRC | 11:51 | |
| *** maishsk_ has joined #openstack-infra | 11:53 | |
| *** maishsk has quit IRC | 11:53 | |
| *** maishsk_ is now known as maishsk | 11:53 | |
| openstackgerrit | Marton Kiss proposed openstack-infra/system-config: Update ask.o.o to latest 0.7.x branch https://review.openstack.org/274032 | 11:55 |
| AJaeger | ansiwen: yeah! | 11:57 |
| AJaeger | ansiwen: thanks a lot for getting this in! | 11:57 |
| openstackgerrit | Merged openstack-infra/storyboard-webclient: Fix broken link on task cards in boards https://review.openstack.org/274018 | 11:57 |
| ansiwen | AJaeger, mordred: it's not autodetection, as in my pbr patch, but I guess listing the domains in the setup.cfg is good enough | 11:59 |
| AJaeger | ansiwen: babel 2.0 was July, babel 2.1 was in September, Babel 2.2 2nd Janauary | 11:59 |
| *** jlanoux has quit IRC | 11:59 | |
| AJaeger | ansiwen: there's hope for a release in the next few months | 11:59 |
| ansiwen | AJaeger: ok... so until then I will fix our packaging with call setup.py for each domain I guess... | 12:00 |
| AJaeger | ansiwen: that meansonce 2.3 is out, you can send 50+ patches for all repos updating setup.cfg ;) | 12:00 |
| AJaeger | ansiwen: yeah - or try to backport babel ;) | 12:01 |
| ansiwen | and requirements.txt I guess | 12:01 |
| AJaeger | requirements comes from global-requirements file, so one change only | 12:01 |
| *** salv-orlando has joined #openstack-infra | 12:01 | |
| *** Daisy has joined #openstack-infra | 12:01 | |
| *** jlanoux has joined #openstack-infra | 12:02 | |
| Zara | can anyone help with the puppet-out-of-ram issue? the storyboard repos are updating but storyboard.openstack.org can't get any of the updates, and SotK and myself don't have powers to fix it. | 12:02 |
| ansiwen | but how many repos actually have several text domains? | 12:02 |
| AJaeger | mordred: do you have time for one translation setup review for project-config? https://review.openstack.org/#/c/270697 | 12:02 |
| AJaeger | ansiwen: nearly all that have translatoins - due to the log level files | 12:03 |
| *** thorst has joined #openstack-infra | 12:03 | |
| *** jcoufal has joined #openstack-infra | 12:03 | |
| Zara | (we merged a fair amount of patches for the API and the webclient before we noticed, so now we're a little worried that weird things will happen if the webclient should update without the API updating, etc.) | 12:04 |
| AJaeger | ansiwen: we have 60 repos that are setup for translation - but not all of them actually have translated content. | 12:04 |
| AJaeger | The core projects do have translated content - including the log levels | 12:05 |
| *** maishsk has quit IRC | 12:06 | |
| *** salv-orlando has quit IRC | 12:06 | |
| *** thorst has quit IRC | 12:08 | |
| *** dstepanenko_ has quit IRC | 12:09 | |
| *** mriedem has quit IRC | 12:11 | |
| *** _nadya_ has joined #openstack-infra | 12:13 | |
| *** daemontool has joined #openstack-infra | 12:15 | |
| *** _amrith_ is now known as amrith | 12:16 | |
| *** ujuc has joined #openstack-infra | 12:17 | |
| *** rcernin has quit IRC | 12:18 | |
| *** placko has joined #openstack-infra | 12:18 | |
| *** rcernin has joined #openstack-infra | 12:18 | |
| *** rfolco has joined #openstack-infra | 12:19 | |
| *** thiagop has quit IRC | 12:20 | |
| *** thiagop has joined #openstack-infra | 12:20 | |
| *** EricGonczer_ has joined #openstack-infra | 12:21 | |
| openstackgerrit | Sam Betts proposed openstack-infra/project-config: Add a tinyIPA non-voting test to Ironic https://review.openstack.org/269684 | 12:23 |
| *** maishsk has joined #openstack-infra | 12:24 | |
| *** EricGonc_ has joined #openstack-infra | 12:30 | |
| *** EricGonczer_ has quit IRC | 12:33 | |
| *** zul has joined #openstack-infra | 12:36 | |
| mordred | ansiwen: I've been mostly out this week - I take it lifeless suggested just fixing babel upstream instead of adding the feature to pbr? | 12:36 |
| mordred | I wonder if there's a good way to detect if babel can handle the space-separated list- if so, we could make a pbr workaround for old babel, then update everyone's setup.cfg now, and then when new babel comes out the pbr patch becomes a no-op | 12:38 |
| * mordred is thinking outloud | 12:38 | |
| ansiwen | mordred: yes. but the auto-detection with .pot files I didn't want to bring upstream, so it's just the extension of the "domain" config field. it can contain a space separated list of tet domains now | 12:38 |
| *** weshay_xchat has joined #openstack-infra | 12:38 | |
| mordred | ah - nod. so is pbr still going to have a patch to do the auto-detection? | 12:39 |
| *** yamamoto has joined #openstack-infra | 12:39 | |
| ansiwen | mordred: the babel people just indicated, that they might do another release quite soon for that. also they offered a "development release", which you can pull over pypi as well, it you put the versin into the requiremens | 12:41 |
| *** weshay_xchat is now known as weshay | 12:42 | |
| *** tzn has quit IRC | 12:43 | |
| *** yamamoto has quit IRC | 12:43 | |
| *** vivekd has joined #openstack-infra | 12:44 | |
| mordred | ansiwen: neat! | 12:45 |
| *** julim has joined #openstack-infra | 12:47 | |
| mordred | ansiwen: fwiw, real release would be better - since babel is a runtime dep for openstack things, adding a depend on a pre-release is something we usually don't do | 12:47 |
| mordred | ansiwen: although if they do make a dev release, we can certainly make some WIP patches that explicitly pull it in to test that our use of it will work | 12:47 |
| *** yamamoto has joined #openstack-infra | 12:48 | |
| *** tdurakov_ has quit IRC | 12:49 | |
| *** baoli has joined #openstack-infra | 12:50 | |
| *** tdurakov_ has joined #openstack-infra | 12:52 | |
| *** baoli_ has joined #openstack-infra | 12:52 | |
| *** baoli has quit IRC | 12:54 | |
| *** markvoelker has joined #openstack-infra | 12:57 | |
| *** yamamoto has quit IRC | 12:59 | |
| *** markvoelker has quit IRC | 13:01 | |
| *** yamamoto has joined #openstack-infra | 13:05 | |
| *** kragniz has quit IRC | 13:12 | |
| *** mrmartin has quit IRC | 13:12 | |
| *** keedya has joined #openstack-infra | 13:12 | |
| *** salv-orlando has joined #openstack-infra | 13:12 | |
| *** dprince has joined #openstack-infra | 13:13 | |
| openstackgerrit | Mateusz Matuszkowiak proposed openstack-infra/project-config: Added three new repos for fuel-plugins https://review.openstack.org/269007 | 13:14 |
| *** gordc has joined #openstack-infra | 13:16 | |
| *** tdurakov_ has quit IRC | 13:17 | |
| *** tdurakov_ has joined #openstack-infra | 13:17 | |
| *** kragniz has joined #openstack-infra | 13:18 | |
| *** trown|outttypeww is now known as trown | 13:18 | |
| *** markvoelker has joined #openstack-infra | 13:24 | |
| *** piet has joined #openstack-infra | 13:24 | |
| *** tdurakov_ has quit IRC | 13:25 | |
| *** tdurakov_ has joined #openstack-infra | 13:25 | |
| *** electrofelix has joined #openstack-infra | 13:25 | |
| *** changbl has joined #openstack-infra | 13:27 | |
| *** daemontool has quit IRC | 13:29 | |
| *** yamamoto has quit IRC | 13:31 | |
| *** doug-fish has joined #openstack-infra | 13:31 | |
| *** doug-fish has quit IRC | 13:32 | |
| *** doug-fish has joined #openstack-infra | 13:32 | |
| *** binbincong_ has quit IRC | 13:33 | |
| *** jtomasek_ has joined #openstack-infra | 13:34 | |
| *** jtomasek has quit IRC | 13:34 | |
| *** Jeffrey4l has quit IRC | 13:34 | |
| *** Shrews has joined #openstack-infra | 13:35 | |
| *** Jeffrey4l has joined #openstack-infra | 13:36 | |
| *** yamamoto has joined #openstack-infra | 13:36 | |
| *** binbincong_ has joined #openstack-infra | 13:38 | |
| *** yamamoto has quit IRC | 13:38 | |
| *** tdurakov_ has quit IRC | 13:40 | |
| *** jcoufal has quit IRC | 13:40 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Add windmill project https://review.openstack.org/273702 | 13:41 |
| openstackgerrit | Vincent Françoise proposed openstack-infra/project-config: Enable multinode testing in Watcher https://review.openstack.org/274068 | 13:41 |
| *** agentle has joined #openstack-infra | 13:44 | |
| *** tdurakov_ has joined #openstack-infra | 13:45 | |
| *** amrith is now known as _amrith_ | 13:46 | |
| *** jcoufal has joined #openstack-infra | 13:47 | |
| *** yamamoto has joined #openstack-infra | 13:47 | |
| *** daemontool has joined #openstack-infra | 13:47 | |
| *** vgridnev has quit IRC | 13:47 | |
| *** agentle has quit IRC | 13:48 | |
| *** vgridnev has joined #openstack-infra | 13:48 | |
| *** agentle has joined #openstack-infra | 13:49 | |
| *** keedya has quit IRC | 13:49 | |
| *** tdurakov_ has quit IRC | 13:50 | |
| *** tdurakov_ has joined #openstack-infra | 13:51 | |
| *** pradk has quit IRC | 13:51 | |
| *** yamamoto has quit IRC | 13:53 | |
| *** yamamoto has joined #openstack-infra | 13:55 | |
| *** daemontool has quit IRC | 13:56 | |
| *** tdurakov_ has quit IRC | 13:57 | |
| *** tdurakov_ has joined #openstack-infra | 13:57 | |
| *** daemontool has joined #openstack-infra | 13:57 | |
| *** flepied has joined #openstack-infra | 13:57 | |
| *** esikachev has quit IRC | 13:58 | |
| *** tdurakov_ has quit IRC | 14:01 | |
| *** signed8b_ has joined #openstack-infra | 14:02 | |
| openstackgerrit | Alexey Shtokolov proposed openstack-infra/project-config: Add fuel-plugin-etckeeper repository for Fuel https://review.openstack.org/260208 | 14:03 |
| *** tdurakov_ has joined #openstack-infra | 14:03 | |
| jaosorior | Hello, does anyone know what's up with the fedora-23 node types? They seemed to be working, but now there are two barbican jobs that have been queuing several hours :/ | 14:04 |
| *** signed8b_ has quit IRC | 14:05 | |
| *** yamamoto has quit IRC | 14:05 | |
| *** esikachev has joined #openstack-infra | 14:06 | |
| *** jcoufal has quit IRC | 14:08 | |
| *** agentle has quit IRC | 14:08 | |
| *** tdurakov_ has quit IRC | 14:09 | |
| *** tdurakov_ has joined #openstack-infra | 14:09 | |
| openstackgerrit | Julien Danjou proposed openstack-dev/pbr: Add --sysconfdir option to install command https://review.openstack.org/274077 | 14:10 |
| *** hdd has joined #openstack-infra | 14:10 | |
| *** tiswanso has joined #openstack-infra | 14:11 | |
| *** agentle has joined #openstack-infra | 14:11 | |
| *** sdake has joined #openstack-infra | 14:11 | |
| *** tiswanso has quit IRC | 14:11 | |
| *** kgiusti has joined #openstack-infra | 14:12 | |
| *** rlandy has joined #openstack-infra | 14:12 | |
| *** tiswanso has joined #openstack-infra | 14:12 | |
| *** dkranz has joined #openstack-infra | 14:12 | |
| *** yamamoto has joined #openstack-infra | 14:14 | |
| *** tdurakov_ has quit IRC | 14:14 | |
| *** tdurakov_ has joined #openstack-infra | 14:14 | |
| *** peristeri has joined #openstack-infra | 14:17 | |
| *** yamamoto has quit IRC | 14:17 | |
| *** hdd has quit IRC | 14:17 | |
| *** tdurakov_ has quit IRC | 14:18 | |
| *** jcoufal has joined #openstack-infra | 14:18 | |
| *** tdurakov_ has joined #openstack-infra | 14:18 | |
| *** e0ne has joined #openstack-infra | 14:19 | |
| *** vgridnev has quit IRC | 14:19 | |
| *** jtomasek_ has quit IRC | 14:21 | |
| *** agentle is now known as notsogentle | 14:21 | |
| *** vgridnev has joined #openstack-infra | 14:21 | |
| *** keedya has joined #openstack-infra | 14:22 | |
| *** yamamoto has joined #openstack-infra | 14:22 | |
| *** ccrouch has quit IRC | 14:23 | |
| *** jcoufal has quit IRC | 14:23 | |
| *** zz_dimtruck is now known as dimtruck | 14:27 | |
| *** piet has quit IRC | 14:27 | |
| *** jcoufal has joined #openstack-infra | 14:27 | |
| *** _nadya_ has quit IRC | 14:28 | |
| *** tdurakov_ has quit IRC | 14:29 | |
| *** yamamoto has quit IRC | 14:29 | |
| *** jsavak has joined #openstack-infra | 14:29 | |
| *** maishsk_ has joined #openstack-infra | 14:30 | |
| *** tdurakov_ has joined #openstack-infra | 14:30 | |
| *** maishsk has quit IRC | 14:30 | |
| *** maishsk_ is now known as maishsk | 14:30 | |
| *** notsogentle has quit IRC | 14:31 | |
| *** maishsk has quit IRC | 14:31 | |
| AJaeger | jaosorior: do you have references? | 14:33 |
| *** amitgandhinz has joined #openstack-infra | 14:34 | |
| AJaeger | jaosorior: ah, found it. | 14:35 |
| *** maishsk has joined #openstack-infra | 14:35 | |
| openstackgerrit | Ivan Kolodyazhny proposed openstack-infra/project-config: Add functional test job for python-brick-cinderclient-ext https://review.openstack.org/265925 | 14:35 |
| AJaeger | if anybody is around to help jaosorior : zuul status shows two jobs for bandit running since 5 hours - and the fedora-23 job is queued... | 14:36 |
| *** piet has joined #openstack-infra | 14:39 | |
| *** ccrouch has joined #openstack-infra | 14:39 | |
| *** maishsk has quit IRC | 14:39 | |
| *** alivigni has joined #openstack-infra | 14:39 | |
| openstackgerrit | yolanda.robla proposed openstack-infra/puppet-jenkins: Increase timeout for yum install of development tools https://review.openstack.org/274087 | 14:40 |
| *** _amrith_ is now known as amrith | 14:42 | |
| *** yamamoto has joined #openstack-infra | 14:43 | |
| *** sabeen1 has joined #openstack-infra | 14:45 | |
| *** ccrouch1 has joined #openstack-infra | 14:46 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/shade: create_service() should normalize return value https://review.openstack.org/274091 | 14:47 |
| *** ccrouch has quit IRC | 14:48 | |
| openstackgerrit | David Moreau Simard proposed openstack-infra/project-config: Add Packstack integration test gate jobs https://review.openstack.org/269918 | 14:49 |
| *** baoli_ has quit IRC | 14:50 | |
| *** yamamoto has quit IRC | 14:50 | |
| cbader | I have a strange question, my nodepool at 1700 UTC last night started failing with Error communicating with server[http://localhost:8080/] anyone else see this issue last night? | 14:50 |
| *** baoli has joined #openstack-infra | 14:50 | |
| openstackgerrit | Derek Higgins proposed openstack-infra/tripleo-ci: Retry the ping test on failure https://review.openstack.org/274094 | 14:51 |
| cbader | I have tried restarting processes and even bounced the box but still have the issue. | 14:51 |
| *** ajo has quit IRC | 14:51 | |
| Zara | anyone about to help with the puppet-out-of-RAM issue? Also, sorry if I'm asking too frequently; it's Friday afternoon here so I'm wary of leaving it over the weekend. | 14:52 |
| *** rbrndt has joined #openstack-infra | 14:53 | |
| *** baoli has quit IRC | 14:55 | |
| *** notsogentle has joined #openstack-infra | 14:56 | |
| *** tdurakov_ has quit IRC | 14:59 | |
| *** jcoufal has quit IRC | 14:59 | |
| *** jcoufal has joined #openstack-infra | 15:00 | |
| *** hdd has joined #openstack-infra | 15:00 | |
| *** yamamoto has joined #openstack-infra | 15:03 | |
| *** yamamoto has quit IRC | 15:03 | |
| *** tdurakov_ has joined #openstack-infra | 15:05 | |
| *** sigmavirus24_awa is now known as sigmavirus24 | 15:05 | |
| *** fesp has joined #openstack-infra | 15:05 | |
| *** Qiming has quit IRC | 15:06 | |
| *** burgerk has joined #openstack-infra | 15:06 | |
| *** yamamoto has joined #openstack-infra | 15:08 | |
| *** yamamoto has quit IRC | 15:08 | |
| *** fesp has quit IRC | 15:09 | |
| *** vivekd has quit IRC | 15:09 | |
| *** dimtruck is now known as zz_dimtruck | 15:10 | |
| *** krotscheck has quit IRC | 15:10 | |
| *** aorourke-afk is now known as aorourke | 15:12 | |
| *** tdurakov_ has quit IRC | 15:13 | |
| *** mtanino has joined #openstack-infra | 15:13 | |
| *** tdurakov_ has joined #openstack-infra | 15:13 | |
| *** vivekd has joined #openstack-infra | 15:13 | |
| openstackgerrit | daisy-ycguo proposed openstack-infra/project-config: Create a puppet module for translation check website. https://review.openstack.org/274102 | 15:13 |
| AJaeger | I just saw a failure and log file contains: "MissingSchema: Invalid URL 'mirror.IAD.rax.openstack.org/pypi/simple/pyyaml/': No schema supplied. Perhaps you meant http://mirror.IAD.rax.openstack.org/pypi/simple/pyyaml/?" | 15:14 |
| AJaeger | jeblair, mordred ? ^ | 15:15 |
| *** uberjay has quit IRC | 15:15 | |
| vdrok | AJaeger, see the same, all tests fail because of it | 15:15 |
| *** uberjay has joined #openstack-infra | 15:15 | |
| AJaeger | yeah, seems everything fails - looks like the new images | 15:16 |
| *** jcoufal_ has joined #openstack-infra | 15:16 | |
| openstackgerrit | Simone Agresta proposed openstack-infra/jenkins-job-builder: Fixed downstream_ext comparison conditions https://review.openstack.org/274104 | 15:16 |
| AJaeger | infra-root, see http://eavesdrop.openstack.org/irclogs/%23openstack-infra/latest.log.html#t2016-01-29T00:32:25 | 15:16 |
| AJaeger | "otherwise, we should either (a) start using the afs mirrors some time after 1400 tomorrow, or (b) stop being able to build new nodes some time after 1400 tomorrow" | 15:17 |
| *** Daisy has quit IRC | 15:17 | |
| *** flepied has quit IRC | 15:18 | |
| *** jcoufal has quit IRC | 15:18 | |
| *** pradk has joined #openstack-infra | 15:18 | |
| *** pradk has quit IRC | 15:18 | |
| *** amotoki has joined #openstack-infra | 15:19 | |
| *** Daisy has joined #openstack-infra | 15:19 | |
| *** edmondsw has joined #openstack-infra | 15:20 | |
| *** ccrouch1 has quit IRC | 15:20 | |
| *** krotscheck has joined #openstack-infra | 15:21 | |
| *** ccrouch has joined #openstack-infra | 15:23 | |
| *** tdurakov_ has quit IRC | 15:23 | |
| *** sdake has quit IRC | 15:23 | |
| *** tdurakov_ has joined #openstack-infra | 15:23 | |
| krotscheck | Ooooh, network traffic on mirrors! | 15:24 |
| krotscheck | Looks like the old pypi mirrors are still being used though. | 15:24 |
| *** ildikov has quit IRC | 15:24 | |
| AJaeger | krotscheck: I just saw a failure and log file contains: "MissingSchema: Invalid URL 'mirror.IAD.rax.openstack.org/pypi/simple/pyyaml/': No schema supplied. Perhaps you meant http://mirror.IAD.rax.openstack.org/pypi/simple/pyyaml/?" | 15:25 |
| pabelanger | yolanda: no, upstream yet. Just need an empty repo | 15:25 |
| openstackgerrit | daisy-ycguo proposed openstack-infra/project-config: Create a puppet module for translation check website. https://review.openstack.org/274102 | 15:25 |
| yolanda | ++ | 15:25 |
| AJaeger | krotscheck: seems all of project-config tests are broken | 15:25 |
| krotscheck | AJaeger: Oooh, interesting catch | 15:25 |
| * krotscheck digs | 15:25 | |
| AJaeger | krotscheck: http://logs.openstack.org/97/270697/4/gate/gate-project-config-layout/2456710/ | 15:26 |
| russellb | seems tests all over are broken | 15:26 |
| AJaeger | yolanda: can you now send a #status alert? | 15:26 |
| *** pradk has joined #openstack-infra | 15:26 | |
| yolanda | let me try | 15:26 |
| nibalizer | oh did we break everything? | 15:26 |
| nibalizer | go us | 15:26 |
| AJaeger | nibalizer: seems to be the case according to vdrok and russelb - and those mails I just got... | 15:27 |
| yolanda | #status project-config tests are broken. We are working to fix the issue | 15:27 |
| *** tdurakov_ has quit IRC | 15:27 | |
| yolanda | ircbot doesn't want me | 15:27 |
| russellb | yeah, see all the red on the latest jobs on http://status.openstack.org/zuul/ | 15:27 |
| yolanda | nibalizer, can you try? | 15:27 |
| russellb | it's more than project-config | 15:27 |
| *** tdurakov_ has joined #openstack-infra | 15:27 | |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/project-config: Added schema to pypi mirror https://review.openstack.org/274113 | 15:28 |
| krotscheck | nibalizer: We did | 15:28 |
| krotscheck | AJaeger: ^^ You may have to force merge that. | 15:28 |
| *** vgridnev has quit IRC | 15:28 | |
| AJaeger | krotscheck: I can't ;( yolanda has to help ^ | 15:29 |
| nibalizer | #status project-config tests are broken. We are working to fix the issue | 15:29 |
| AJaeger | #status notice is the syntax | 15:29 |
| yolanda | ok let me see | 15:29 |
| yolanda | ah... | 15:29 |
| AJaeger | yolanda, try "notice" or "alert" | 15:29 |
| nibalizer | #status notice project-config tests are broken. We are working to fix the issue | 15:30 |
| nibalizer | no love | 15:30 |
| yolanda | #status alert jobs are broken, we are working to fix the issue | 15:30 |
| yolanda | !!! | 15:30 |
| openstack | yolanda: Error: "!!" is not a valid command. | 15:30 |
| yolanda | silly! | 15:30 |
| yolanda | ok going to see how to force it | 15:30 |
| AJaeger | nibalizer, yolanda : Both of you not setup? That's sad ;( | 15:30 |
| *** tdurakov_ has quit IRC | 15:31 | |
| AJaeger | mordred, are you around to explain how to use #status? | 15:31 |
| *** jpr has quit IRC | 15:31 | |
| krotscheck | Once that lands, we'll need an image rebuild | 15:31 |
| *** vgridnev has joined #openstack-infra | 15:32 | |
| AJaeger | krotscheck: want to send an email to openstack-dev and tell folks it's time for weekend? ;) | 15:32 |
| AJaeger | or how long will it take? | 15:32 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/shade: Add enabled flag to keystone service data https://review.openstack.org/274116 | 15:32 |
| krotscheck | AJaeger: Roger wilco | 15:32 |
| *** vgridnev has quit IRC | 15:33 | |
| yolanda | working on the force push | 15:34 |
| *** jpr has joined #openstack-infra | 15:34 | |
| *** dprince has quit IRC | 15:35 | |
| openstackgerrit | yolanda.robla proposed openstack-infra/project-config: Added schema to pypi mirror https://review.openstack.org/274113 | 15:35 |
| openstackgerrit | Merged openstack-infra/project-config: Added schema to pypi mirror https://review.openstack.org/274113 | 15:35 |
| yolanda | oh yes! | 15:36 |
| *** tdurakov_ has joined #openstack-infra | 15:36 | |
| *** zz_dimtruck is now known as dimtruck | 15:36 | |
| yolanda | can you recheck? | 15:37 |
| krotscheck | yolanda: We need to rebuild the slave images first. | 15:37 |
| krotscheck | jeblair: ^^ | 15:37 |
| krotscheck | Also, mordred ^^ | 15:37 |
| *** yolanda has quit IRC | 15:39 | |
| *** tonytan4ever has joined #openstack-infra | 15:40 | |
| krotscheck | Email has been sent | 15:40 |
| pavel_bondar | sdague: Hi! carl_baldwin mentioned that you might know some way to get grenade to run an external upgrade script. We were discussing it in https://bugs.launchpad.net/neutron/+bug/1516156 so comments are welcome. | 15:40 |
| openstack | Launchpad bug 1516156 in neutron "[RFE] IPAM migration from non-pluggable to pluggable" [Wishlist,In progress] - Assigned to Pavel Bondar (pasha117) | 15:40 |
| *** yolanda has joined #openstack-infra | 15:40 | |
| yolanda | nibalizer, you know more than me about nodepool status these days.. is ok to build images ? everything is on dib? | 15:41 |
| nibalizer | yolanda: uhhhhhhh | 15:41 |
| AJaeger | krotscheck: LOL ;) | 15:41 |
| nibalizer | I don't really know | 15:41 |
| zeih | hi, can you please have a look at this: http://logs.openstack.org/68/271968/2/check/gate-tacker-horizon-pep8/40c5405/console.html | 15:41 |
| *** Daisy has quit IRC | 15:41 | |
| nibalizer | no not everything is on dib no | 15:41 |
| AJaeger | zeih: everything's broken, we're on it... | 15:41 |
| AJaeger | zeih: http://lists.openstack.org/pipermail/openstack-dev/2016-January/085353.html | 15:41 |
| zeih | okay. thx | 15:41 |
| yolanda | any other infra-root to explain about nodepool images? | 15:42 |
| yolanda | i don't want to build something that cannot be built | 15:42 |
| *** amotoki has quit IRC | 15:42 | |
| krotscheck | SergeyLukjanov: ping? ^^ | 15:43 |
| krotscheck | mordred is, last I checked, in some kind of a 2 day management training. | 15:43 |
| * AJaeger will be back later... | 15:43 | |
| krotscheck | clarkb ? | 15:43 |
| *** jsavak has quit IRC | 15:44 | |
| *** jcoufal_ has quit IRC | 15:45 | |
| yolanda | also as i noticed that there is something with remote builders going on, is the same process now to build images? | 15:46 |
| * krotscheck is not of much help on this one, all that stuff was in jeblair's brain | 15:48 | |
| prometheanfire | wondering if someone can review this https://review.openstack.org/#/c/273795 | 15:48 |
| *** tdurakov_ has quit IRC | 15:48 | |
| *** baoli has joined #openstack-infra | 15:48 | |
| *** tdurakov_ has joined #openstack-infra | 15:49 | |
| *** zeih has quit IRC | 15:49 | |
| dims | krotscheck : haha (we give you time to submit talks!) | 15:49 |
| *** amotoki has joined #openstack-infra | 15:50 | |
| krotscheck | dims: There's a silver lining! | 15:50 |
| krotscheck | :D | 15:50 |
| dims | indeed! | 15:50 |
| *** jsavak has joined #openstack-infra | 15:50 | |
| jeblair | good morning! | 15:51 |
| pc_m | amotoki: ping | 15:51 |
| yolanda | jeblair, you come at the right time | 15:51 |
| krotscheck | jeblair: I broke it. | 15:51 |
| *** MCoLo has quit IRC | 15:51 | |
| * krotscheck didn't actually break it | 15:51 | |
| krotscheck | jeblair: But I'm going to claim credit anyway. | 15:52 |
| *** MCoLo has joined #openstack-infra | 15:52 | |
| *** MCoLo has quit IRC | 15:52 | |
| *** tdurakov_ has quit IRC | 15:52 | |
| *** MCoLo has joined #openstack-infra | 15:52 | |
| jeblair | yolanda, nibalizer: we have a standard procedure any time we end up with broken images, and that hasn't changed. | 15:52 |
| *** tdurakov_ has joined #openstack-infra | 15:52 | |
| jeblair | yolanda, nibalizer: that procedure is that we immediately delete the most recent images so that nodepool falls back to using the previous ones | 15:52 |
| yolanda | jeblair, for all image types? | 15:53 |
| jeblair | yolanda: for all image types | 15:53 |
| *** notsogentle has quit IRC | 15:54 | |
| jeblair | yolanda: that is why nodepool always keeps two of every image | 15:54 |
| jeblair | yolanda, nibalizer: would one of you like to perform that procedure now? | 15:54 |
| *** notsogentle has joined #openstack-infra | 15:54 | |
| yolanda | yep, i can do it | 15:55 |
| jeblair | yolanda: great, let me know if you have any questions | 15:55 |
| *** tdurakov_ has quit IRC | 15:55 | |
| krotscheck | Hrm, in that case, force merging the fix was probably not necessary. | 15:55 |
| *** yamahata has quit IRC | 15:55 | |
| yolanda | looks like not | 15:56 |
| *** yamahata has joined #openstack-infra | 15:56 | |
| jeblair | krotscheck: yeah, at the rate they're going, the rebuilds will probably take all of today. | 15:56 |
| *** jordanP has quit IRC | 15:57 | |
| krotscheck | jeblair: Better to let the old images take on the workload while that happens. | 15:57 |
| openstackgerrit | Valeriy Ponomaryov proposed openstack-infra/project-config: Add check-requirements job for manila-image-elements project https://review.openstack.org/274132 | 15:57 |
| *** amotoki has quit IRC | 15:58 | |
| openstackgerrit | Valeriy Ponomaryov proposed openstack/requirements: Add manila-image-elements to project list https://review.openstack.org/274135 | 15:58 |
| yolanda | jeblair, i see some images with only one entry, or others that have one entry and another building | 15:59 |
| *** tdurakov_ has joined #openstack-infra | 16:00 | |
| jeblair | yolanda: it should be fine to delete the ones that are building (though if that doesn't work, we should let greghaynes know) | 16:02 |
| jeblair | yolanda: which ones only have one entry? | 16:02 |
| yolanda | jeblair, i see ubuntu-trusty from rax for example, or fedora 23 | 16:02 |
| *** baoli has quit IRC | 16:02 | |
| jeblair | yolanda: all of the ubuntu-trusty rax images are between 6 and 23 days old, so no need to delete them. | 16:03 |
| jeblair | yolanda: you should only need to delete images that are less than a couple of hours old at the most | 16:04 |
| *** leakypipes has quit IRC | 16:04 | |
| yolanda | ah, noticed the time ok | 16:05 |
| yolanda | yep, i'm deleting quite new ones | 16:05 |
| jeblair | yeah, regular image building has been more or less broken for the past 3 weeks. | 16:05 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/project-config: zuul/layout: optimize puppet jobs https://review.openstack.org/274137 | 16:06 |
| SamYaple | whats the merge recheck command to for jenkins? | 16:07 |
| SamYaple | merge-recheck? | 16:07 |
| jeblair | SamYaple: 'recheck' | 16:07 |
| SamYaple | jeblair: that will recheck the merge jobs? | 16:07 |
| SamYaple | errr gate jobs | 16:07 |
| jeblair | SamYaple: it will run the check jobs | 16:08 |
| SamYaple | right i need to rerun gate jobs | 16:08 |
| jeblair | SamYaple: if it has an approved vote, it will go to gate when it finishes check | 16:08 |
| prometheanfire | SamYaple: hi :D | 16:09 |
| SamYaple | jeblair: I am aware. see PS in question https://review.openstack.org/#/c/272160/ | 16:09 |
| SamYaple | prometheanfire: hey! | 16:09 |
| prometheanfire | SamYaple: mind if I spam you with review requests? | 16:09 |
| SamYaple | prometheanfire: you can spam and ill add me as reviewer, but i won't be able to do them until tonight | 16:09 |
| jeblair | SamYaple: okay, that patch is in the check queue | 16:10 |
| prometheanfire | SamYaple: anything to get them in :P | 16:10 |
| SamYaple | jeblair: thats because someone typed recheck when its the gate jobs that need a recheck | 16:10 |
| prometheanfire | SamYaple: it starts with https://review.openstack.org/#/c/273795/ | 16:10 |
| jeblair | SamYaple: yes, that's the only choice | 16:10 |
| SamYaple | i am certain there is a way to recheck the gate jobs specifically | 16:10 |
| jeblair | SamYaple: there is not | 16:11 |
| prometheanfire | SamYaple: I'll be up tonight | 16:11 |
| prometheanfire | SamYaple: I split out the megapatch | 16:11 |
| SamYaple | prometheanfire: i saw | 16:11 |
| jeblair | SamYaple: like i said earlier, recheck goes into check, if it has a workflow+1, it will enter the gate after it passes check | 16:11 |
| jeblair | SamYaple: we used to have a reverify command that would do that, but some folks got tired of people sending unchecked patches into the gate queue that would then fail and slow everything down, so the gate pipeline was altered to require a +1 in check first | 16:12 |
| SamYaple | jeblair: its 'reverify' | 16:12 |
| SamYaple | thats the one i remembered | 16:12 |
| jeblair | SamYaple: yeah, it does the same thing as recheck now | 16:12 |
| SamYaple | ah ok | 16:12 |
| SamYaple | so im not crazy, that used to be a thing? | 16:12 |
| jeblair | yep | 16:12 |
| SamYaple | good to know. thanks jeblair! | 16:13 |
| jeblair | i miss it myself, but we can't have nice things :) | 16:13 |
| SamYaple | :) | 16:13 |
| openstackgerrit | Luigi Toscano proposed openstack-infra/project-config: Fix Sahara config for DevStack on stable branches https://review.openstack.org/274142 | 16:14 |
| jeblair | yolanda: have you finished deleting the images? | 16:15 |
| yolanda | double checking | 16:15 |
| yolanda | jeblair, i'm looking at the list and looks fine now | 16:16 |
| jeblair | it looks like centos7dib is still building | 16:16 |
| jeblair | i would like to stop the builder to abort that | 16:17 |
| jeblair | but last time i did that, i had to reboot the server | 16:17 |
| *** amotoki has joined #openstack-infra | 16:17 | |
| *** placko has quit IRC | 16:18 | |
| prometheanfire | ya, I thought it had to be reworkflowed | 16:18 |
| *** derekh has joined #openstack-infra | 16:18 | |
| *** placko has joined #openstack-infra | 16:18 | |
| *** fdegir has quit IRC | 16:19 | |
| *** fdegir has joined #openstack-infra | 16:19 | |
| jeblair | greghaynes: i shut down the builder and then killed the dib process that was running, that mostly cleaned up but it did leave /opt/dib_tmp/image.hJIa9ict/mnt mounted | 16:19 |
| *** _nadya_ has joined #openstack-infra | 16:19 | |
| jeblair | i unmounted it | 16:19 |
| *** salv-orlando has quit IRC | 16:19 | |
| eantyshev | yolanda, jeblair: Hello! can we look at https://review.openstack.org/#/c/272123 at the moment? | 16:20 |
| jeblair | eantyshev: not the best time for me | 16:22 |
| *** paramite has joined #openstack-infra | 16:22 | |
| jeblair | i've updated the ready script on the filesystem and am ready to restart the builder now | 16:22 |
| *** rajinir has joined #openstack-infra | 16:23 | |
| eantyshev | jeblair: ok, this can wait. This is a small fix for periodic pipelines in Zuul | 16:23 |
| paramite | greetings, what I should do to be able to delete branch in gerrit? | 16:24 |
| *** nt has joined #openstack-infra | 16:25 | |
| *** jpr has quit IRC | 16:25 | |
| prometheanfire | so, never did get an answer, does openstack infra use masterless puppet, puppetserver or the normal puppet-master (possibly with passenger)? | 16:25 |
| krotscheck | prometheanfire: The answer is "Yes". | 16:26 |
| prometheanfire | lol | 16:26 |
| nt | does JJB YAML require 4 spaces of indentation? | 16:26 |
| jeblair | prometheanfire: http://specs.openstack.org/openstack-infra/infra-specs/specs/ansible_puppet_apply.html | 16:26 |
| openstackgerrit | Merged openstack-infra/project-config: Add searchlight-specs to specs index https://review.openstack.org/273830 | 16:26 |
| jeblair | nt: no, but we might in our jjb configs | 16:26 |
| prometheanfire | so ansible managed puppet | 16:26 |
| *** placko has quit IRC | 16:26 | |
| *** ildikov has joined #openstack-infra | 16:26 | |
| prometheanfire | which is close to masterless | 16:27 |
| prometheanfire | neat | 16:27 |
| nt | jeblair: thx. is there a standard that i should follow? i'm doing a lot with ansible so two spaces is common there. just trying to keep things consistent. | 16:27 |
| krotscheck | yolanda, jeblair: lmk when it's safe to send an "all clear" email. | 16:27 |
| *** rcernin has quit IRC | 16:27 | |
| AJaeger | jeblair: could you check out why nibalizer and yolanda cannot use #status at all, please? | 16:27 |
| tosky | is it possible to test changes to jobs defined in project-config from another dummy review with Depends-On:? Or do jobs require a rebuild anyway? | 16:27 |
| AJaeger | tosky: not possible, jobs need to merge first. | 16:28 |
| tosky | AJaeger: as I thought; thanks | 16:28 |
| *** aopincar has joined #openstack-infra | 16:28 | |
| *** paramite is now known as paramite|afk | 16:28 | |
| tosky | so I need some experts to look at a certain change :) | 16:28 |
| jeblair | nt: actually we use 2 in our jjb config. we use the style in the spec: http://www.yaml.org/spec/1.2/spec.html | 16:29 |
| krotscheck | tosky: Most of the core/root brains are firefighting right now, can I help in the interim? | 16:29 |
| jeblair | nt: (sometimes it looks like 4, but that's only when we have a dict in a dict) | 16:30 |
| *** ccrouch1 has joined #openstack-infra | 16:30 | |
| tosky | krotscheck: sure, thanks; I saw the backlog, I didn't mean to push, it's about this (which tries to close another issue): https://review.openstack.org/274142 | 16:30 |
| nt | jeblair: how to you specify that in your config? i just checked the docs and i don't see a config option for indentation? or is that set somewhere in the JJB code? | 16:30 |
| tosky | SergeyLukjanov: also for you ^^ | 16:30 |
| prometheanfire | bib checks failing? passed previously and just ran recheck http://logs.openstack.org/76/273776/1/check/gate-diskimage-builder-python27/f92f6c8/console.html | 16:30 |
| jeblair | nt: it's written by humans so humans type the spacebar | 16:31 |
| prometheanfire | also, is this the channel for DIB stuff or is it trippleo? because there seems to be some crossover | 16:31 |
| *** jaosorior has quit IRC | 16:31 | |
| *** esker has joined #openstack-infra | 16:32 | |
| jeblair | 2016-01-29 15:27:16,199 DEBUG statusbot.bot: Ignoring message from unauthenticated user yolanda | 16:32 |
| jeblair | AJaeger, yolanda: ^ | 16:32 |
| *** ccrouch has quit IRC | 16:32 | |
| nt | jeblair: thank you! | 16:32 |
| jeblair | you must be authenticated with nickserv to use statusbot | 16:32 |
| jeblair | 2016-01-29 15:29:22,833 DEBUG statusbot.bot: Ignoring message from untrusted user nibalizer | 16:33 |
| *** HeOS has quit IRC | 16:33 | |
| AJaeger | yolanda: you said last time you were authenticated - are you really ? | 16:33 |
| AJaeger | thanks, jeblair | 16:33 |
| jeblair | i think nibalizer probably hasn't been added to the list of statusbot users | 16:33 |
| AJaeger | jeblair: he's in accessbot/channels.yaml | 16:33 |
| jeblair | AJaeger: there's a separate list for statusbot users | 16:34 |
| jeblair | manifests/site.pp: statusbot_auth_nicks | 16:34 |
| AJaeger | ah - you're right, nibalizer is missing there. | 16:34 |
| AJaeger | Shall I send a patch? | 16:34 |
| jeblair | AJaeger: nibalizer should send a patch :) | 16:35 |
| AJaeger | fine with me ;) | 16:35 |
| krotscheck | tosky: Checking. I suspect you'll get some pushabck from AJaeger, more comments on the patch. | 16:35 |
| jeblair | krotscheck: i believe you can send the all-clear | 16:36 |
| tosky | krotscheck: thanks | 16:36 |
| *** esikachev has quit IRC | 16:36 | |
| *** aopincar has quit IRC | 16:37 | |
| krotscheck | jeblair: Thanks | 16:37 |
| *** EricGonc_ has quit IRC | 16:37 | |
| *** piet has quit IRC | 16:38 | |
| krotscheck | jeblair: Done | 16:38 |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci: Make pingtest partially voting https://review.openstack.org/274151 | 16:40 |
| tosky | krotscheck: about the notes: I see some if-s based on the branch in other yaml files as well; wouldn't your suggestions mean also separate builders? | 16:40 |
| *** EricGonczer_ has joined #openstack-infra | 16:41 | |
| krotscheck | tosky: It would, I think. Note that my suggestion is tentative, and based on assumptions on what's going on in AJaeger's brain. | 16:41 |
| * krotscheck could be totally wrong. | 16:41 | |
| * krotscheck will remove his -1 to make sure things don't get stuck in a filter. | 16:41 | |
| *** aopincar has joined #openstack-infra | 16:42 | |
| *** andreykurilin has quit IRC | 16:42 | |
| jeblair | i have started image updates of all image types | 16:43 |
| jeblair | we should start to see results in maybe 30-60 minutes (but it will take many hours for them all to finish) | 16:43 |
| AJaeger | krotscheck: sdague is the real expert here, this behaviour is fine for different branches. | 16:43 |
| krotscheck | AJaeger, tosky: I stand corrected. | 16:44 |
| krotscheck | jeblair: Alright, I'll check around 10AM. | 16:44 |
| tosky | yeah, it seems like temporary fix due to the different behavior of devstack vs external plugins | 16:44 |
| jeblair | krotscheck: no one should ever filter on -1 code-reviews so there's no need to adjust your behavior there. | 16:44 |
| *** matrohon has quit IRC | 16:44 | |
| krotscheck | jeblair: I have found that "should" rarely reflects reality, but I agree with you. | 16:45 |
| jeblair | krotscheck: if anyone does do that, they're performing a disservice to the community and they can rot | 16:45 |
| *** EricGonczer_ has quit IRC | 16:46 | |
| *** scheuran has quit IRC | 16:47 | |
| *** EricGonczer_ has joined #openstack-infra | 16:48 | |
| *** harlowja_at_home has joined #openstack-infra | 16:49 | |
| jeblair | do we have any devstack-centos7 nodes on ovh yet? | 16:49 |
| jeblair | ah yes | 16:51 |
| jeblair | but maybe not rax yet | 16:51 |
| *** EricGonczer_ has quit IRC | 16:51 | |
| *** EricGonczer_ has joined #openstack-infra | 16:51 | |
| *** mikelk has quit IRC | 16:52 | |
| *** samuelBartel has quit IRC | 16:53 | |
| openstackgerrit | Merged openstack-infra/project-config: Prepare new translation functions for python projects https://review.openstack.org/270697 | 16:53 |
| *** paramite|afk is now known as paramite | 16:54 | |
| openstackgerrit | James E. Blair proposed openstack-infra/project-config: Nodepool: Remove hpcloud https://review.openstack.org/274155 | 16:54 |
| jeblair | krotscheck: shoulda done that before starting the image builds ^ | 16:55 |
| *** sridhar_ram1 has joined #openstack-infra | 16:55 | |
| krotscheck | jeblair: Probably. Especially since we've just been told that the super secret internal grace period is no longer a thing. | 16:55 |
| openstackgerrit | Merged openstack-infra/project-config: Add windmill project https://review.openstack.org/273702 | 16:55 |
| openstackgerrit | Merged openstack-infra/jenkins-job-builder: Add Test Selector Plugin support https://review.openstack.org/268795 | 16:55 |
| *** bpokorny has joined #openstack-infra | 16:56 | |
| *** tdurakov_ has quit IRC | 16:56 | |
| clarkb | I already have a change stack to remove hpcloud fwiw | 16:57 |
| jeblair | clarkb: did it account for centos7? | 16:57 |
| *** tdurakov_ has joined #openstack-infra | 16:57 | |
| thomasem | Can anyone tell me where the worker count for devstack gate jobs is configured? | 16:58 |
| clarkb | I rebased yesterday to pick up the recent changes | 16:58 |
| thomasem | Having a hard time finding the config | 16:58 |
| jeblair | clarkb: you and mordred | 16:58 |
| *** piet has joined #openstack-infra | 16:59 | |
| clarkb | Ialso have nodepool logging config patch to push up as soon aas I can get to my laptop | 16:59 |
| openstackgerrit | Simone Agresta proposed openstack-infra/jenkins-job-builder: Fixed downstream_ext comparison conditions https://review.openstack.org/274104 | 16:59 |
| openstackgerrit | Merged openstack-infra/project-config: Adds non voting ceilometer tests for oslo.messaging drivers https://review.openstack.org/273637 | 16:59 |
| jeblair | clarkb: good, because i'm not landing max_servers=-1 patches just to do it manually, i'd much rather land removals and cleanup. | 16:59 |
| *** sabeen1 has quit IRC | 17:00 | |
| *** ashleighfarnham has joined #openstack-infra | 17:01 | |
| *** flepied has joined #openstack-infra | 17:01 | |
| *** [1]Thelo has joined #openstack-infra | 17:02 | |
| *** e0ne has quit IRC | 17:03 | |
| tsufiev | folks, could you help with landing the change https://review.openstack.org/#/c/273750/ ? Horizon needs it to make integration tests consistently passing | 17:03 |
| *** dprince has joined #openstack-infra | 17:03 | |
| *** Thelo has quit IRC | 17:04 | |
| *** [1]Thelo is now known as Thelo | 17:04 | |
| *** baoli has joined #openstack-infra | 17:04 | |
| *** daemontool has quit IRC | 17:04 | |
| clarkb | thomasem: the openstack service worker count? devstack configures that | 17:04 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Use new translation setup for all horizon plugin repos https://review.openstack.org/273773 | 17:05 |
| *** jlanoux has quit IRC | 17:05 | |
| *** esikachev has joined #openstack-infra | 17:06 | |
| AJaeger | tsufiev: infra team does not normally review requirements, talk to the special core team, see https://review.openstack.org/#/admin/groups/131,members | 17:07 |
| *** tdurakov_ has quit IRC | 17:07 | |
| openstackgerrit | Valeriy Ponomaryov proposed openstack-infra/project-config: Add check-requirements job for manila-image-elements project https://review.openstack.org/274132 | 17:07 |
| *** vivekd has quit IRC | 17:08 | |
| thomasem | clarkb: Sorry, didn't mean to be vague. Uhhhm, I'm referring to the test worker count, what shows up in console.log in the summary for the tempest tests that ran. I suspect it's somewhere in Tempest, but for all of my googling and searching, I'm just missing it. | 17:08 |
| clarkb | thomasem: thst is set in tempest tox.ini iirc | 17:09 |
| thomasem | Okay, I'll go snoop around that. Thanks! :D | 17:09 |
| clarkb | default is half the number of cpus | 17:09 |
| thomasem | Ohhhhh, interesting | 17:10 |
| *** _ody_ has joined #openstack-infra | 17:10 | |
| *** unicell has quit IRC | 17:10 | |
| *** jpr has joined #openstack-infra | 17:10 | |
| *** esikachev has quit IRC | 17:11 | |
| *** _ody_ has quit IRC | 17:11 | |
| *** aopincar has quit IRC | 17:11 | |
| *** _nadya_ has quit IRC | 17:12 | |
| tsufiev | AJaeger, thanks for the info, didn't know that | 17:13 |
| openstackgerrit | James E. Blair proposed openstack-infra/project-config: Add devstack-centos7 to internap and blue box https://review.openstack.org/274161 | 17:13 |
| openstackgerrit | James E. Blair proposed openstack-infra/project-config: Remove devstack-centos7-dib in favor of devstack-centos7 https://review.openstack.org/274162 | 17:13 |
| openstackgerrit | James E. Blair proposed openstack-infra/project-config: Remove unused images https://review.openstack.org/274164 | 17:14 |
| jeblair | does fedora23 work? | 17:15 |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci: Set overcloud deploy timeout https://review.openstack.org/273716 | 17:16 |
| *** dprince has quit IRC | 17:16 | |
| *** apoorvad has joined #openstack-infra | 17:16 | |
| pabelanger | jeblair: it should yes, I tested it yesterday with system-config | 17:17 |
| *** harlowja_at_home has quit IRC | 17:17 | |
| *** dprince has joined #openstack-infra | 17:17 | |
| openstackgerrit | James E. Blair proposed openstack-infra/project-config: Add fedora-23 to bluebox, internap, and ovh https://review.openstack.org/274169 | 17:17 |
| *** yaume has quit IRC | 17:17 | |
| pabelanger | jeblair: if you want to approve: https://review.openstack.org/#/c/231681/ | 17:17 |
| pabelanger | jeblair: nevermind | 17:18 |
| pabelanger | already approved | 17:18 |
| jeblair | ok, we should definitely add it to more image types | 17:19 |
| *** signed8b_ has joined #openstack-infra | 17:20 | |
| jeblair | our big weak spot is now bare-precise and bare-trusty which are only running on one cloud | 17:20 |
| jeblair | my understanding is that's held up on the bindep work? | 17:21 |
| pabelanger | I am unsure | 17:21 |
| *** _ody has quit IRC | 17:21 | |
| *** achanda has joined #openstack-infra | 17:21 | |
| *** _ody has joined #openstack-infra | 17:21 | |
| jeblair | heh, good news! the centos7 hpcloud image builds are complete :) | 17:22 |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci: Enable undercloud ssl on nonha job https://review.openstack.org/273743 | 17:22 |
| openstackgerrit | Clark Boylan proposed openstack-infra/puppet-nodepool: Colocate a builder-worker with the service https://review.openstack.org/271541 | 17:22 |
| zaro | morning | 17:22 |
| jeblair | krotscheck: bare-precise in iad is ready | 17:22 |
| akscram | Guys, we have one unstable job in the fuel-octane project, please take a look https://review.openstack.org/#/c/272442/ | 17:22 |
| *** tiswanso has quit IRC | 17:22 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/system-config: Provide separate nodepool builder log config https://review.openstack.org/274173 | 17:22 |
| clarkb | jeblair: ^ I think that should do it | 17:22 |
| *** tiswanso has joined #openstack-infra | 17:23 | |
| jeblair | akscram: that's very strange | 17:24 |
| jeblair | the directory /srv/static/logs/42/272442 is owned by root | 17:25 |
| jeblair | which should not happen | 17:25 |
| jeblair | and the last modified times there don't make sense to me either | 17:26 |
| krotscheck | jeblair: cool, checking | 17:26 |
| *** ybathia has joined #openstack-infra | 17:28 | |
| *** notsogentle has quit IRC | 17:28 | |
| *** ybathia has quit IRC | 17:29 | |
| jeblair | akscram: only thing i can think of is that this is somehow related to the filesystem error we had earlier in the week | 17:29 |
| AJaeger | jeblair: fedora 23 has a problem | 17:29 |
| jeblair | i will manually fix the permissions | 17:29 |
| *** woodster_ has joined #openstack-infra | 17:29 | |
| *** ybathia has joined #openstack-infra | 17:29 | |
| jeblair | pabelanger: ^ see AJaeger | 17:29 |
| pabelanger | Was review.o.o taken offline by ansible or puppet? | 17:29 |
| yolanda | AJaeger, it maybe that, i'm looking at logs, i'm under a bouncer, so i may have something wrong | 17:29 |
| jeblair | akscram: should be fixed now. | 17:30 |
| pabelanger | AJaeger: what you seeing? | 17:30 |
| AJaeger | jeblair: check the two bandit projects in the top of check queue on status.o.o/zuul | 17:30 |
| *** ybathia has quit IRC | 17:30 | |
| AJaeger | I see "gate-barbican-dogtag-devstack-dsvm-f23: queued (non-voting)" - since 8 hours! | 17:30 |
| pabelanger | checking | 17:30 |
| *** ybathia has joined #openstack-infra | 17:30 | |
| jeblair | DibFailedError: DIB failed creating /opt/nodepool_dib/176231 | 17:30 |
| jeblair | :( | 17:30 |
| jeblair | (not related to f23) | 17:31 |
| *** zul has quit IRC | 17:31 | |
| *** achanda has quit IRC | 17:32 | |
| *** andymaier has joined #openstack-infra | 17:33 | |
| greghaynes | jeblair: I still need to release. It was a bit late yesterday for me | 17:33 |
| *** Swami has joined #openstack-infra | 17:33 | |
| greghaynes | probably unrelated to that fail, just FYI | 17:33 |
| *** ybathia has quit IRC | 17:33 | |
| pabelanger | jeblair: AJaeger I don't think there are any fedora-23 nodes online | 17:34 |
| *** ybathia has joined #openstack-infra | 17:34 | |
| pabelanger | if I am looking at graphite.o.o correctly | 17:34 |
| jeblair | Exception: Timeout waiting for ssh access | 17:34 |
| jeblair | fedora-23 ^ | 17:34 |
| jeblair | so that at least is what's happening with the nodes we're currently trying to launch | 17:34 |
| *** pblaho has quit IRC | 17:34 | |
| *** yamahata has quit IRC | 17:35 | |
| lifeless | mordred: ansiwen: when babel releases, make a g-r change to raise the minimum, then propogate that, then we can use the feature | 17:36 |
| akscram | jeblair: thanks, can you wave to me to send recheck there when problem will be solved? | 17:36 |
| jeblair | akscram: it's done | 17:36 |
| * greghaynes kind of wants to wait until monday to release dib | 17:37 | |
| akscram | jeblair: oh, good news, I'll try | 17:37 |
| jeblair | pabelanger, AJaeger: yeah they haven't been bootable in a while it looks like | 17:37 |
| greghaynes | I think it was just to add debugging stuff for infra, which can hopefully wait the weekend | 17:37 |
| jeblair | at least as far back as midnight utc | 17:37 |
| jeblair | 2016-01-29 17:28:54,975 INFO nodepool.image.build.devstack-trusty: + git --git-dir=/opt/dib_cache/source-repositories/windmill_e6cee2e9b526a299e60e90d89315eebd154afcaa/.git fetch --prune --update-head-ok git://git.openstack.org/openstack/windmill.git +master:master | 17:39 |
| jeblair | did windmill just get approved? | 17:39 |
| akscram | jeblair: yeah, it works now | 17:39 |
| *** Swami has quit IRC | 17:39 | |
| akscram | jeblair: how it can be prevented in the future? | 17:40 |
| jeblair | akscram: it can not | 17:40 |
| *** Swami has joined #openstack-infra | 17:40 | |
| jeblair | yes | 17:40 |
| jeblair | AJaeger: can you stop approving new projects for the day? | 17:40 |
| jeblair | also, anyone else ^ :) | 17:41 |
| *** abregman|afk has joined #openstack-infra | 17:42 | |
| AJaeger | jeblair: just new projects or everything? | 17:42 |
| AJaeger | jeblair: sure... | 17:42 |
| *** abregman|afk has quit IRC | 17:42 | |
| *** unicell has joined #openstack-infra | 17:42 | |
| jeblair | AJaeger: just new projects -- they race image builds and sometimes cause them to fail | 17:42 |
| *** abregman has joined #openstack-infra | 17:42 | |
| openstackgerrit | Timur Sufiev proposed openstack/requirements: Cap Selenium package to not exceed version 2.49 https://review.openstack.org/273750 | 17:42 |
| AJaeger | jeblair: windmill was approved in the last two hours or so | 17:42 |
| AJaeger | jeblair: will not even +2 any ;) | 17:42 |
| jeblair | AJaeger: thanks | 17:43 |
| *** andymaier has quit IRC | 17:43 | |
| *** ccrouch1 has quit IRC | 17:43 | |
| AJaeger | jeblair: is there a specific time we shouldn't do that in general? 14:00 to ? | 17:43 |
| jeblair | AJaeger: yes :) i can't fill in the ? right now though | 17:44 |
| *** abregman is now known as abregman|nb | 17:44 | |
| AJaeger | then let's tell yolanda and others as well... | 17:44 |
| jeblair | (though really it would be great if the build scripts were a little more robust) | 17:44 |
| AJaeger | definitely! | 17:44 |
| Zara | (I'm heading off for the night soon; if any infra-root has a chance to look at storyboard's host being out of RAM, we will buy you a cake if/when next we meet. (puppet keeps crashing so the api and webclient haven't been building, so the version on s.o.o is about a week out of date.)) | 17:45 |
| *** TechNick99 has quit IRC | 17:45 | |
| clarkb | Zara: looks like there are >130 storyboard worker processes | 17:46 |
| clarkb | each using about 43MB of memory | 17:46 |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config: Add storyboard.openstack.org to cacti https://review.openstack.org/274177 | 17:46 |
| pedroalvarez | nice one ^ | 17:46 |
| *** ccrouch has joined #openstack-infra | 17:46 | |
| *** ccrouch has quit IRC | 17:46 | |
| *** ccrouch has joined #openstack-infra | 17:46 | |
| fungi | okay, i'm sort of around again (unpacking and doing some minor post-travel chores) but have basically declared bankruptcy on ~10k lines of scrollback in channel. are we ready for the hpcloud turn-down now? if so, i'll go ahead with 264371 | 17:47 |
| jeblair | nobody touch nodepool | 17:47 |
| * fungi takes that as a "no" | 17:47 | |
| jeblair | it's been a disaster for a week, everything nodepool goes through me | 17:47 |
| fungi | ouch. i heard rumor of something in dib blowing up the nodepool image builder earlier in the week | 17:48 |
| AJaeger | welcome back, fungi! | 17:48 |
| fungi | thanks AJaeger! | 17:48 |
| Zara | clarkb: that sounds strange... | 17:49 |
| Zara | jeblair: thanks! | 17:49 |
| *** jistr has quit IRC | 17:49 | |
| AJaeger | fungi, highlights from backscroll would be jeblair and krotscheck working on the mirrors and nodepool, yolanda fixing some broken disks,... | 17:49 |
| clarkb | Zara: how many should be running? | 17:49 |
| fungi | i'm mostly caught up on the mailing lists, reviewing the meeting log shortly | 17:49 |
| *** acruz has joined #openstack-infra | 17:51 | |
| Zara | clarkb: I'm not sure, the storyboard worker daemon on my test instance is listed next to 7 with ps -aux | 17:52 |
| *** tdurakov_ has joined #openstack-infra | 17:52 | |
| pabelanger | jeblair: Hmm, will sync up with ianw and see what is going on | 17:52 |
| *** Daisy has joined #openstack-infra | 17:53 | |
| *** tdurakov_ has quit IRC | 17:54 | |
| Zara | clarkb: (erm, 7 processes, not a pid of 7...) | 17:54 |
| *** arxcruz has quit IRC | 17:54 | |
| clarkb | Zara: gotcha | 17:54 |
| *** paramite is now known as paramite|afk | 17:54 | |
| Zara | my test VM has 2GB of RAM | 17:55 |
| Zara | so something's odd. | 17:55 |
| openstackgerrit | Armando Migliaccio proposed openstack-infra/project-config: Restrict neutron-dsvm-api to running only during the check phase https://review.openstack.org/274182 | 17:55 |
| clarkb | I wonder if when we restart the service we aren't properly killing the workers off | 17:55 |
| clarkb | Zara: ^ so that when the service is updated and restarted we orphan processes | 17:56 |
| *** Guest96421 has quit IRC | 17:56 | |
| Zara | sounds probable. | 17:56 |
| *** zeih has joined #openstack-infra | 17:56 | |
| Zara | (or at least, I think I'd've seen something locally if it were storyboard itself doing strange things.) | 17:57 |
| *** tdurakov_ has joined #openstack-infra | 17:57 | |
| *** Daisy has quit IRC | 17:57 | |
| *** yamahata has joined #openstack-infra | 17:58 | |
| jeblair | krotscheck: ubuntu-trusty in ovh-bhs1 is ready | 17:58 |
| jeblair | krotscheck: as is bare-precise in rax-iad | 17:58 |
| krotscheck | jeblair: danke | 17:58 |
| jeblair | krotscheck: i'm a little worried about this failure: http://logs.openstack.org/41/271541/4/check/gate-openstackci-beaker-centos7-dsvm/6d3f889/console.html | 17:58 |
| *** tdurakov_ has quit IRC | 17:58 | |
| jeblair | but i don't have any hpcloud centos7 nodes to check right now | 17:59 |
| krotscheck | jeblair: Huhn. | 17:59 |
| krotscheck | jeblair: Lemme doublecheck the configure host script | 17:59 |
| krotscheck | jeblair: Ah | 17:59 |
| krotscheck | Sec | 17:59 |
| jeblair | krotscheck: yeah i see it too | 18:00 |
| Zara | clarkb: how would we fix it? guessing that's a task for infra-root? | 18:00 |
| jeblair | krotscheck: i found an iad bare-precise host: | 18:00 |
| jeblair | index-url = http://mirror.IAD.rax.openstack.org/pypi/simple | 18:00 |
| jeblair | trusted-host = http://mirror.IAD.rax.openstack.org/pypi/simple | 18:00 |
| krotscheck | jeblair: On it | 18:00 |
| jeblair | krotscheck: i'll stop the image builds | 18:00 |
| clarkb | Zara: first we should figure out why they leak, address that, then yes an root will need to stop al lthe unnecessary processes | 18:00 |
| *** derekh has quit IRC | 18:01 | |
| *** paramite|afk is now known as paramite | 18:01 | |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/project-config: Corrected entry for pip's trusted-host https://review.openstack.org/274184 | 18:01 |
| *** jpr has quit IRC | 18:02 | |
| krotscheck | jeblair: ^^ | 18:02 |
| *** sigmavirus24 is now known as sigmavirus24_awa | 18:02 | |
| *** sabeen has joined #openstack-infra | 18:03 | |
| Zara | clarkb: okay, I'm wondering how we'd figure it out without access to the machine, but I don't know that much about the tools infra provide for this sort of thing. | 18:03 |
| clarkb | Zara: you should be able to reproduce with the init scripts if that is indeed the problem | 18:03 |
| *** sabeen2 has joined #openstack-infra | 18:05 | |
| Zara | clarkb: ah, okay, thanks, that's straightforward! will try on Monday since I need to head off now; thanks so much for your help!. :) | 18:05 |
| pedroalvarez | wow 130 storyboard processes! | 18:06 |
| pedroalvarez | storyboard-worker* | 18:06 |
| *** amotoki has quit IRC | 18:06 | |
| jeblair | clarkb, fungi: i would like to land https://review.openstack.org/271541 https://review.openstack.org/274173 https://review.openstack.org/274155 https://review.openstack.org/274184 now | 18:06 |
| pabelanger | AJaeger: yolanda: do you mind adding me to https://review.openstack.org/#/admin/groups/1264,members and https://review.openstack.org/#/admin/groups/1265,members ? | 18:06 |
| Zara | pedroalvarez: what's it like over at s.b.o? | 18:07 |
| jeblair | clarkb, fungi: i have stopped all image builds and am currently cleaning up after them | 18:07 |
| pedroalvarez | Zara: 4 | 18:07 |
| AJaeger | pabelanger: I don't have permissions and guess yolanda has gone already. Better ask clarkb, jeblair, or fungi | 18:07 |
| jeblair | clarkb, fungi: i'd like to land all of those changes, and then re-enable puppet on nodepool and then start image builds again | 18:07 |
| openstackgerrit | Martin Oemke proposed openstack-infra/project-config: Enable check-requirements for tacker-horizon https://review.openstack.org/271980 | 18:07 |
| *** sabeen has quit IRC | 18:08 | |
| pabelanger | AJaeger: sure, np | 18:08 |
| jeblair | pabelanger: done | 18:08 |
| clarkb | jeblair: we should be setting the max servers to -1 first, let nodepool cleanup after itself then remove the config for those clouds | 18:08 |
| Zara | pedroalvarez: hahaha :) | 18:08 |
| fungi | jeblair: thanks for the heads-up. i'm shifting gears and reviewing those now, though it looks like the new patchset clarkb pushed for that first one is failing puppet apply and beaker jobs? | 18:09 |
| *** paramite has quit IRC | 18:09 | |
| pabelanger | jeblair: danke | 18:09 |
| clarkb | fungi: I will take a look at those in a sec | 18:09 |
| jeblair | fungi, clarkb: yeah, the centos7 at least is a (potential) false-negative, the others may be real | 18:09 |
| openstackgerrit | Merged openstack-infra/project-config: Fix Sahara config for DevStack on stable branches https://review.openstack.org/274142 | 18:10 |
| fungi | clarkb: Error: Could not find template 'nodepool/nodepool-buil | 18:10 |
| fungi | der.logging.conf.erb' at /etc/puppet/modules/nodepool/manifests/builder.pp:45 on | 18:10 |
| fungi | node bare-trusty-rax-iad-7586094 | 18:10 |
| fungi | grr, newlines | 18:10 |
| jeblair | clarkb: i understand -1, but *absolutely nothing* about nodepool and image builds has worked for 2 weeks. i *will* need to perform manual cleanup anyway, so i'd much rather just merge it and clean it up myself. | 18:10 |
| *** _nadya_ has joined #openstack-infra | 18:11 | |
| clarkb | fungi: ya I didn't use the right file suffix | 18:11 |
| clarkb | update in a sec | 18:11 |
| fungi | clarkb: looks like you forgot to add a .erb extension to the file, yeah | 18:11 |
| zeih | AJaeger: anteaya: I would like to ask to review https://review.openstack.org/#/c/271980/ | 18:11 |
| openstackgerrit | Clark Boylan proposed openstack-infra/puppet-nodepool: Colocate a builder-worker with the service https://review.openstack.org/271541 | 18:12 |
| jeblair | i have spent most of this week dealing with broken nodepool, deleting a couple more images manually is going to take 30 seconds, waiting for -1 to work is going to take much longer if it works at all | 18:12 |
| AJaeger | zeih: on my queue already... | 18:13 |
| fungi | clarkb: looks like both 271541 and 274173 add that file, but with different filenames (the first without the .erb extension, the second with) so will probably merge-conflict if both aren't adjusted? | 18:14 |
| *** jpr has joined #openstack-infra | 18:14 | |
| *** ujuc has quit IRC | 18:14 | |
| *** flepied has quit IRC | 18:15 | |
| clarkb | fungi: they are different repos | 18:15 |
| fungi | er, yes | 18:15 |
| clarkb | I am looking at why the system-config change filed now | 18:15 |
| fungi | i missed that *sigh* | 18:16 |
| clarkb | oh grrrrr | 18:16 |
| clarkb | we have 3 layers of indirection | 18:16 |
| *** SumitNaiksatam has quit IRC | 18:17 | |
| fungi | yep, invalid parameters because openstackci needs updating | 18:17 |
| clarkb | yup change for that in a minute | 18:17 |
| greghaynes | http://logs.openstack.org/84/272384/6/check/gate-nodepool-python27/bc95c36/console.html is a really odd nodepool test fail :/ | 18:17 |
| *** EricGonczer_ has quit IRC | 18:17 | |
| *** EricGonczer_ has joined #openstack-infra | 18:19 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/puppet-openstackci: Support nodepool builder logging config https://review.openstack.org/274192 | 18:20 |
| openstackgerrit | Clark Boylan proposed openstack-infra/system-config: Provide separate nodepool builder log config https://review.openstack.org/274173 | 18:21 |
| clarkb | fungi: jeblair ^ that rearrangement should make puppet happier | 18:21 |
| jeblair | i'm so glad every line of the class instantiation was changed :| | 18:22 |
| *** tdurakov_ has joined #openstack-infra | 18:22 | |
| *** zeih has quit IRC | 18:22 | |
| *** edmondsw has quit IRC | 18:23 | |
| greghaynes | we can undo it, I suspect the added threading logic is biting us though | 18:23 |
| jeblair | greghaynes: ? | 18:23 |
| jeblair | undo what? | 18:23 |
| greghaynes | the class reworking in that nodepool patch? | 18:24 |
| greghaynes | or are streams crossing | 18:24 |
| jeblair | greghaynes: no, i was talking about clarkb's patch | 18:24 |
| greghaynes | ah | 18:24 |
| jeblair | greghaynes: when you add a parameter in puppet, you must add spaces to untold numbers of lines so that the rocketships line up | 18:24 |
| greghaynes | wow | 18:24 |
| greghaynes | why not just... not do that | 18:24 |
| greghaynes | :p | 18:25 |
| jeblair | it's insane, but we've decided to go with it on the puppet-* modules because all of the puppet-* people do that | 18:25 |
| greghaynes | heh, ok | 18:25 |
| jeblair | we do not do it in system-config | 18:25 |
| jeblair | (at least, we don't enforce it) | 18:25 |
| *** rbrndt has quit IRC | 18:26 | |
| *** bhunter71 has joined #openstack-infra | 18:27 | |
| dmsimard | AJaeger: is that a difference between zuul cloner and zuul git prep ? | 18:27 |
| dmsimard | AJaeger: (assuming so) | 18:27 |
| AJaeger | dmsimard: shouldn't - but thinking more about it, let's just use gerrit-git-prep as we do elsewhere | 18:27 |
| dmsimard | so gerrit-git-prep and no cd ? | 18:28 |
| AJaeger | dmsimard: wait, let me check out of curiosity. | 18:28 |
| AJaeger | Yes, no cd and gerrit-git-prep | 18:28 |
| dmsimard | AJaeger: thanks a lot. | 18:28 |
| openstackgerrit | David Moreau Simard proposed openstack-infra/project-config: Add Packstack integration test gate jobs https://review.openstack.org/269918 | 18:29 |
| *** rossella_s has quit IRC | 18:31 | |
| *** rossella_s has joined #openstack-infra | 18:32 | |
| *** ccrouch1 has joined #openstack-infra | 18:32 | |
| bhunter71 | a dumb usage question; In gerrit ACL files, the wildcards specify a whole heirarchy/tree? Or, if I was to have two(2) ACL sections with identical configuration and names like 'refs/heads/*' and 'refs/heads/something/* - the latter section would be redundant and not necessary, right? Thanks! | 18:33 |
| jeblair | clarkb, fungi: are you still reviewing? | 18:33 |
| fungi | i am. was making sure the jobs were also passing this time | 18:34 |
| clarkb | bhunter71: refs/heads/* includes refs/heads/something/* they would not be redundant if the something/* path set the exclusive flag on its acls | 18:34 |
| openstackgerrit | Darragh Bailey proposed openstack-infra/jenkins-job-builder: Compatibility support for older maven project plugins https://review.openstack.org/274195 | 18:34 |
| clarkb | jeblair: yes (half the change sare mine so no reviews from me) | 18:34 |
| *** salv-orlando has joined #openstack-infra | 18:34 | |
| openstackgerrit | greghaynes proposed openstack-infra/nodepool: Cleanup bulder jobs and add disconnect support https://review.openstack.org/272384 | 18:34 |
| *** ccrouch has quit IRC | 18:35 | |
| openstackgerrit | Merged openstack-infra/project-config: Corrected entry for pip's trusted-host https://review.openstack.org/274184 | 18:35 |
| bhunter71 | interesting. thanks, as so many times before, clarkb! | 18:35 |
| *** kushal has quit IRC | 18:35 | |
| fungi | jeblair: clarkb: all those lgtm now too | 18:35 |
| fungi | jeblair: i did not approve any of them since you wanted those changes to go through you | 18:36 |
| jeblair | i have approved them all | 18:36 |
| jeblair | i want them all to land, then i will re-enable puppet | 18:36 |
| jeblair | once all of them are applied on the host, i'll clean up the centos7 images, restart the builder and kick off another set | 18:37 |
| fungi | sounds good--thanks for taking point on this | 18:37 |
| jeblair | np | 18:37 |
| fungi | sorry i'm still trying to get back up to speed after a week of minimal internets | 18:38 |
| jeblair | even though puppet will be restarted, i don't want to land any more nodepool-related changes until we have a good set of images | 18:38 |
| jeblair | once that happens, we should be able to return to normal | 18:38 |
| fungi | that seems exceedingly wise | 18:38 |
| fungi | we might want to avoid doing any non-urgent changes to nodepool configuration until next week at this point anyway | 18:38 |
| fungi | seeing as it's already friday | 18:39 |
| *** sabeen has joined #openstack-infra | 18:39 | |
| greghaynes | jeblair: I am pretty confident https://review.openstack.org/#/c/272384/ will pass tests now, but feel free to merge as you see fit since youre the one handling its fallout | 18:40 |
| *** sabeen2 has quit IRC | 18:40 | |
| *** sabeen3 has joined #openstack-infra | 18:41 | |
| openstackgerrit | greghaynes proposed openstack-infra/nodepool: Test for gear disconnect with queued builds https://review.openstack.org/273703 | 18:41 |
| *** sigmavirus24_awa is now known as sigmavirus24 | 18:41 | |
| *** flepied has joined #openstack-infra | 18:41 | |
| *** sabeen has quit IRC | 18:44 | |
| openstackgerrit | Merged openstack-infra/project-config: Nodepool: Remove hpcloud https://review.openstack.org/274155 | 18:44 |
| prometheanfire | greghaynes: hi, have time? | 18:44 |
| jeblair | greghaynes: thanks, i think i'm going to omit that in this intervention; i have a handle on the current behavior (which i think is sufficient to get the new image builds) | 18:45 |
| greghaynes | prometheanfire: not especially :( trying to catch up on all the stuff I put off this week | 18:45 |
| prometheanfire | booo | 18:45 |
| greghaynes | prometheanfire: why? | 18:45 |
| prometheanfire | reviews | 18:45 |
| prometheanfire | I split up that big review into pieces | 18:45 |
| greghaynes | prometheanfire: if they are dib ones then I'll get to them, I have a queue of stuff to review there | 18:45 |
| prometheanfire | greghaynes: k, thanks | 18:45 |
| *** kushal has joined #openstack-infra | 18:48 | |
| openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci: Minimise memory usage for deployed overcloud https://review.openstack.org/273431 | 18:50 |
| *** kushal has quit IRC | 18:50 | |
| *** kushal has joined #openstack-infra | 18:51 | |
| *** sdake has joined #openstack-infra | 18:51 | |
| *** nelsnelson has joined #openstack-infra | 18:53 | |
| *** signed8b_ has quit IRC | 18:54 | |
| *** tdurakov_ has quit IRC | 18:54 | |
| *** dkranz has quit IRC | 18:56 | |
| *** ZZelle_ has joined #openstack-infra | 18:56 | |
| *** pcaruana has joined #openstack-infra | 18:57 | |
| *** kzaitsev_mb has quit IRC | 18:58 | |
| *** SumitNaiksatam has joined #openstack-infra | 18:58 | |
| *** achanda has joined #openstack-infra | 18:58 | |
| *** daemontool has joined #openstack-infra | 18:58 | |
| *** gordc has quit IRC | 19:00 | |
| *** lezbar has quit IRC | 19:00 | |
| *** bpokorny_ has joined #openstack-infra | 19:01 | |
| *** tosky has quit IRC | 19:02 | |
| *** jpr has quit IRC | 19:04 | |
| *** jsavak has quit IRC | 19:04 | |
| *** bpokorny has quit IRC | 19:04 | |
| clarkb | the puppet stack should be merging in a few minutes according to zuul | 19:06 |
| *** daemontool has quit IRC | 19:10 | |
| *** jsavak has joined #openstack-infra | 19:13 | |
| *** dkranz has joined #openstack-infra | 19:13 | |
| *** achanda has quit IRC | 19:13 | |
| openstackgerrit | Merged openstack-infra/puppet-nodepool: Colocate a builder-worker with the service https://review.openstack.org/271541 | 19:14 |
| openstackgerrit | Merged openstack-infra/puppet-openstackci: Support nodepool builder logging config https://review.openstack.org/274192 | 19:14 |
| openstackgerrit | Merged openstack-infra/system-config: Provide separate nodepool builder log config https://review.openstack.org/274173 | 19:14 |
| jeblair | oh fun | 19:15 |
| krotscheck | jeblair: I'm going to be offline for a bit, have to move my bouncer to a non-hplcoud instance | 19:15 |
| jeblair | zombie ansible processes on puppetmaster | 19:15 |
| jeblair | krotscheck: gl | 19:15 |
| krotscheck | jeblair: thanks, I'll need it. | 19:16 |
| krotscheck | jeblair: At least they made me admin over my own quota this time ;) | 19:16 |
| *** rbrndt has joined #openstack-infra | 19:17 | |
| *** krotscheck has quit IRC | 19:17 | |
| *** eharney_ has joined #openstack-infra | 19:17 | |
| *** eharney_ is now known as eharney | 19:18 | |
| greghaynes | apparently https://review.openstack.org/#/c/204639/14 is not having its tests run because of a rebase conflict, but the rebase conflict isn't being shown in gerrit... | 19:19 |
| greghaynes | jgit vs zuul git mismatch? | 19:19 |
| greghaynes | although that should show us the gerrit merge conflict | 19:19 |
| AJaeger | I'm signing off now and wish everybody a great weekend! | 19:21 |
| jeblair | greghaynes: why do you think there is a merge conflict? | 19:22 |
| *** EricGonc_ has joined #openstack-infra | 19:23 | |
| *** EricGonc_ has quit IRC | 19:23 | |
| greghaynes | jeblair: I pulled it locally and rebased on master and got a conflict | 19:23 |
| fungi | greghaynes: jeblair: according to the zuul logs it looks like there's a reporting failure trying to expand a nonexistent LOG_PATH | 19:23 |
| *** EricGonc_ has joined #openstack-infra | 19:24 | |
| fungi | traceback in the logfile (both check and check-tripleo) | 19:24 |
| greghaynes | awesome | 19:24 |
| *** pcaruana has quit IRC | 19:24 | |
| *** Swami has quit IRC | 19:24 | |
| *** Swami has joined #openstack-infra | 19:24 | |
| *** EricGonczer_ has quit IRC | 19:25 | |
| fungi | greghaynes: i'm assuming you meant /24 not /14? | 19:25 |
| greghaynes | fungi: ah, yep | 19:25 |
| *** EricGonczer_ has joined #openstack-infra | 19:25 | |
| *** EricGonc_ has quit IRC | 19:25 | |
| jeblair | fungi: can you paste the tb? | 19:25 |
| fungi | jeblair: yep, working on it, will have it in just a sec | 19:26 |
| *** gordc has joined #openstack-infra | 19:26 | |
| *** dimtruck is now known as zz_dimtruck | 19:27 | |
| jeblair | i've re-enabled puppet on nodepool | 19:28 |
| fungi | jeblair: greghaynes: http://paste.openstack.org/show/485477 | 19:28 |
| fungi | the traceback for check-tripleo pipeline is similar | 19:29 |
| *** jsavak has quit IRC | 19:29 | |
| *** jsavak has joined #openstack-infra | 19:30 | |
| *** salv-orlando has quit IRC | 19:32 | |
| *** Jeffrey4l_ has joined #openstack-infra | 19:32 | |
| *** sdake has quit IRC | 19:33 | |
| *** Jeffrey4l has quit IRC | 19:34 | |
| *** sabeen1 has joined #openstack-infra | 19:35 | |
| *** pcaruana has joined #openstack-infra | 19:37 | |
| *** notsogentle has joined #openstack-infra | 19:37 | |
| *** jpr has joined #openstack-infra | 19:37 | |
| openstackgerrit | greghaynes proposed openstack/diskimage-builder: Perform a booting test for our images https://review.openstack.org/204639 | 19:37 |
| *** sabeen3 has quit IRC | 19:37 | |
| *** rguillebert has quit IRC | 19:37 | |
| *** jistr has joined #openstack-infra | 19:37 | |
| *** fesp has joined #openstack-infra | 19:38 | |
| *** ccrouch1 has quit IRC | 19:39 | |
| jeblair | this appears to be a defect in https://review.openstack.org/223298 | 19:40 |
| *** electrofelix has quit IRC | 19:40 | |
| jeblair | where the old reporting method would only format jobs if jobs had actually run | 19:40 |
| *** ybathia has quit IRC | 19:40 | |
| jeblair | but this new model seems to assume that each kind of failure has its own reporter | 19:40 |
| jeblair | however, the merge-failure reporter defaults to the job failure reporter | 19:41 |
| jeblair | which can't actually handle formatting merge-failures anymore | 19:41 |
| jeblair | we might be able to compensate by explicitly defining merge-failure reporters for all of our pipelines | 19:41 |
| *** ccrouch has joined #openstack-infra | 19:41 | |
| *** eil397 has joined #openstack-infra | 19:41 | |
| jeblair | but i believe jhesketh's intention was that people would not have to do that | 19:41 |
| jeblair | perhaps this is non-urgent enough we can leave it for him to untangle next week | 19:42 |
| *** zeih has joined #openstack-infra | 19:43 | |
| jeblair | puppet has run on nodepool | 19:43 |
| *** sdake has joined #openstack-infra | 19:44 | |
| jeblair | nodepool is deleting hpcloud images | 19:44 |
| jeblair | there are 0 hpcloud nodes | 19:44 |
| Clint | :/ | 19:45 |
| zeih | hi | 19:46 |
| fungi | i agree, non-urgent | 19:46 |
| zeih | sdague: are you around? | 19:46 |
| fungi | once we're confirmed no longer running with any remaining hpcloud nodes in use, i'll follow up to the ml thread about our sunsetting schedule | 19:47 |
| dmsimard | AJaeger: thanks for babysitting me throughout this, first time I add a job in project-config :) | 19:47 |
| jeblair | fungi: that's already the case | 19:48 |
| *** jpr has quit IRC | 19:49 | |
| jeblair | fungi: (we've only used it for centos7 for 3 days, so it's been almost entirely idle during that time) | 19:49 |
| *** jpr has joined #openstack-infra | 19:49 | |
| jeblair | it has not been a good week | 19:49 |
| fungi | oh, i missed that being the case | 19:49 |
| fungi | well then | 19:49 |
| *** zeih has quit IRC | 19:50 | |
| jeblair | at the rate we were able to upload images to hpcloud, we would not have been able to update them until after the sunset. | 19:50 |
| jeblair | so we stopped | 19:50 |
| jeblair | but kept centos7 running there until we could get it running on another cloud | 19:50 |
| jeblair | which happened yesterday | 19:50 |
| *** zz_dimtruck is now known as dimtruck | 19:51 | |
| fungi | ahh, actually i should have noticed in reviewing 274155 that it was only still for centos7 | 19:51 |
| *** EricGonczer_ has quit IRC | 19:52 | |
| jeblair | http://grafana.openstack.org/dashboard/db/zuul-status?panelId=8&fullscreen | 19:52 |
| *** EricGonczer_ has joined #openstack-infra | 19:52 | |
| jeblair | (until now, the graph has been misleading since we didn't actually have that many nodes available for all image typs) | 19:52 |
| *** ccrouch1 has joined #openstack-infra | 19:53 | |
| *** krtaylor has quit IRC | 19:54 | |
| *** ccrouch has quit IRC | 19:55 | |
| *** fesp has quit IRC | 19:56 | |
| jeblair | i've kicked off a full set of image builds | 19:57 |
| clarkb | jeblair: did we have to remove the working images from saturday for some reason? | 19:58 |
| * clarkb is confused as to why those wouldn't have continued to work | 19:58 | |
| SamYaple | prometheanfire: send those reviews over | 19:59 |
| jeblair | clarkb: i see 2 images that have the right timestamp to possibly have been from saturday | 19:59 |
| prometheanfire | SamYaple: https://review.openstack.org/273795 first | 19:59 |
| jeblair | clarkb: many of our images are newer than than, some older. | 20:00 |
| prometheanfire | SamYaple: then these in any order https://review.openstack.org/273764 https://review.openstack.org/273769 https://review.openstack.org/273772 https://review.openstack.org/273776 https://review.openstack.org/273782 https://review.openstack.org/273790 | 20:00 |
| clarkb | ok I must be missing some context on the urgency to rebuild then | 20:00 |
| jeblair | clarkb: we need new images to start using the new mirrors | 20:00 |
| *** achanda has joined #openstack-infra | 20:01 | |
| *** shashank_hegde has joined #openstack-infra | 20:01 | |
| *** ociuhandu has quit IRC | 20:01 | |
| *** jsavak has quit IRC | 20:03 | |
| *** Sukhdev has joined #openstack-infra | 20:03 | |
| *** bpokorny_ has quit IRC | 20:04 | |
| *** aysyd has quit IRC | 20:04 | |
| *** bpokorny has joined #openstack-infra | 20:04 | |
| jeblair | i'm going to get lunch; back in a bit | 20:05 |
| *** sabeen1 has quit IRC | 20:05 | |
| jeblair | if it all goes wrong, stop nodepool-builder and delete new images | 20:05 |
| clarkb | jeblair: it is ok to approve https://review.openstack.org/#/c/272717/1 right? we consume that from releases in nodepool/zuul so merging that won't affect us until we release | 20:06 |
| *** pcaruana has quit IRC | 20:06 | |
| *** jsavak has joined #openstack-infra | 20:06 | |
| *** krtaylor has joined #openstack-infra | 20:06 | |
| jeblair | clarkb: yep | 20:07 |
| openstackgerrit | Matthew Thode proposed openstack/diskimage-builder: add new cloud-init element https://review.openstack.org/273764 | 20:07 |
| prometheanfire | SamYaple: fixed https://review.openstack.org/273764 thought I caught them all | 20:07 |
| *** esker has quit IRC | 20:08 | |
| openstackgerrit | Matthew Thode proposed openstack/diskimage-builder: add gentoo support to growroot https://review.openstack.org/273769 | 20:09 |
| *** pcaruana has joined #openstack-infra | 20:09 | |
| SamYaple | prometheanfire: ok. all reviewed. Code looks good (and it was just a split of the other patchset) so I didn't see any changes needed | 20:10 |
| SamYaple | minus what was pointed out that you have already fixed | 20:11 |
| prometheanfire | yep | 20:11 |
| prometheanfire | as far as gate, as far as I can tell, notmyfault | 20:11 |
| SamYaple | yea i dont think it is, but i haven't dug into what those pariticular jobs are doing | 20:11 |
| *** acruz has quit IRC | 20:11 | |
| openstackgerrit | Merged openstack-infra/gear: Guard against double shutdown in client https://review.openstack.org/272717 | 20:11 |
| prometheanfire | SamYaple: about the usrbinenv thing | 20:12 |
| prometheanfire | https://review.openstack.org/#/c/273879/ | 20:12 |
| *** mrmartin has joined #openstack-infra | 20:12 | |
| *** diana_clarke has joined #openstack-infra | 20:15 | |
| *** ccrouch1 has quit IRC | 20:15 | |
| *** shardy has quit IRC | 20:18 | |
| *** pcaruana has quit IRC | 20:19 | |
| SamYaple | prometheanfire: cool | 20:19 |
| SamYaple | you know technically using env like this is abuse of the `env` command | 20:19 |
| *** ldnunes has quit IRC | 20:19 | |
| prometheanfire | is it? | 20:19 |
| SamYaple | but its such a common practice no one dares break the behaviour | 20:20 |
| prometheanfire | I thought it was making the code more portable | 20:20 |
| SamYaple | yea it won't work on older distros and OSes | 20:20 |
| SamYaple | some of them that is | 20:20 |
| prometheanfire | make it more portable by dropping support for older | 20:20 |
| prometheanfire | gotcha | 20:20 |
| SamYaple | its actually meant to run a command in a _different_ environemnt, so in that way its abuse | 20:20 |
| SamYaple | but yea its pretty common practice | 20:20 |
| SamYaple | yea prometheanfire, form the man page "env - run a program in a modified environment" | 20:21 |
| SamYaple | which we arent doing | 20:21 |
| prometheanfire | dunno where bash is on bsds | 20:22 |
| SamYaple | oh i know why we do it, and im not saying its a bad thing | 20:22 |
| *** ccrouch has joined #openstack-infra | 20:22 | |
| prometheanfire | but if it was /usb/bin/bash then everything would fail | 20:22 |
| *** amrith is now known as _amrith_ | 20:22 | |
| SamYaple | yup | 20:22 |
| *** derekh has joined #openstack-infra | 20:26 | |
| *** apoorvad has quit IRC | 20:28 | |
| *** jcoufal has joined #openstack-infra | 20:28 | |
| *** jcoufal has quit IRC | 20:28 | |
| *** rfolco has quit IRC | 20:29 | |
| *** vgridnev has joined #openstack-infra | 20:29 | |
| *** ybathia has joined #openstack-infra | 20:29 | |
| openstackgerrit | Derek Higgins proposed openstack-infra/tripleo-ci: Minimise memory usage for deployed overcloud https://review.openstack.org/273431 | 20:30 |
| *** rfolco has joined #openstack-infra | 20:32 | |
| *** salv-orlando has joined #openstack-infra | 20:32 | |
| *** thiagop has quit IRC | 20:33 | |
| clarkb | I am going to hold 2 nodes in bluebox to look at some multinode testing stuff real quick | 20:33 |
| *** ybathia has quit IRC | 20:36 | |
| *** ybathia has joined #openstack-infra | 20:36 | |
| clarkb | there is connectivity between the two private IPs for the nodes. I can ping between 10.220.2.160 and 10.220.2.161 but can't ping over the 172.24.4.0/23 tunneled addresses | 20:38 |
| *** salv-orlando has quit IRC | 20:39 | |
| *** kzaitsev_mb has joined #openstack-infra | 20:40 | |
| clarkb | and I rmeember now why i dislike ovs | 20:41 |
| clarkb | tcpdump! | 20:41 |
| *** rbrndt has quit IRC | 20:42 | |
| openstackgerrit | Merged openstack/requirements: Cap Selenium package to not exceed version 2.49 https://review.openstack.org/273750 | 20:43 |
| *** salv-orlando has joined #openstack-infra | 20:44 | |
| *** angdraug has joined #openstack-infra | 20:46 | |
| *** sridhar_ram1 is now known as sridhar_ram | 20:46 | |
| clarkb | like we had issues with a while back arp is failing, however I don't see the same error flows that we experienced then | 20:49 |
| clarkb | iptables is configured to accept all traffic between the two hosts using their private IPs | 20:50 |
| *** derekh has quit IRC | 20:51 | |
| clarkb | and both icmp and ssh should be allowed for all IPs | 20:51 |
| clarkb | but that don't work | 20:51 |
| jeblair | back | 20:53 |
| clarkb | I suspect something at the tunnel layer | 20:53 |
| *** piet has quit IRC | 20:54 | |
| *** apoorvad has joined #openstack-infra | 20:55 | |
| clarkb | mgagne: jlk would there be any thing at a cloud level that would filter gre tunnels between VMs on internap/bluebox? | 20:55 |
| jlk | oh man. | 20:56 |
| jlk | that's way above my knowledge base | 20:56 |
| jlk | it's a GRE tunnel? | 20:56 |
| jlk | not just, the tenant network bits themselves? | 20:56 |
| clarkb | jlk: correct, its a GRE tunnel within the existing tenant networks whatever they may be | 20:57 |
| *** Sukhdev has quit IRC | 20:57 | |
| *** piet has joined #openstack-infra | 20:57 | |
| jlk | so, I'm not entirely sure that will work | 20:57 |
| jlk | in SoftLayer, we're using l2pop to manage ARP, and there's vxlan encapsulation going on, and other... things, that I don't know if that'l lwork | 20:57 |
| jlk | but I'm no network expert | 20:58 |
| clarkb | doing multiple layers should be fine as long as you don't explicitly filter the gre ip packet type I think | 20:58 |
| *** bpokorny_ has joined #openstack-infra | 20:58 | |
| jlk | what's the GRE tunnel buying you? | 20:58 |
| clarkb | jlk: IP address space managed by the clouds under test | 20:59 |
| jlk | you're saying that, privately, the two can talk to each other? Two instances within the same cloud on the same tenant network? | 20:59 |
| jlk | oh, this is cloud on cloud stuff | 20:59 |
| clarkb | specifically so that tempest can ssh into VMs on compute hosts external to where tempest is running | 20:59 |
| clarkb | yes | 20:59 |
| jlk | oh oh! | 20:59 |
| jlk | yeah there is some special stuff that has to happen | 20:59 |
| jlk | I think | 20:59 |
| jlk | let me see if I can summon somebody who knows more about this than me | 21:00 |
| clarkb | at a security group level we look ok, but I woulnd't be surprised if there was some additional filtering going on (which is bad bad bad imo) | 21:00 |
| clarkb | jlk: that would be great, thank you | 21:00 |
| fungi | as long as nothing is filtering ip protocol 47 between instances, there shouldn't really be anything else needed at the provider level right? | 21:00 |
| clarkb | fungi: that is what I would expect | 21:01 |
| clarkb | but when you start talking about ovs and friends it may be more subtle than that I suppose | 21:02 |
| jlk | right, because you're trying ot do OVS on top of what's already there | 21:02 |
| *** bpokorny has quit IRC | 21:02 | |
| fungi | trying to do ovs inside gre though | 21:02 |
| clarkb | yes, though thats the joy of a tunnel the encapsulating layer shouldn't really care | 21:02 |
| fungi | so the ovs part ought not be visible at the provider's layer | 21:02 |
| clarkb | fungi: right | 21:03 |
| fungi | yeah, that | 21:03 |
| jeblair | some of the images are finished, and i suspect they've gone wrong again | 21:03 |
| jeblair | trying to confirm now | 21:03 |
| jeblair | omg | 21:04 |
| *** ajmiller has quit IRC | 21:04 | |
| jeblair | the pip trusted hosts setting is CASE SENSITIVE | 21:04 |
| clarkb | does pip know that dns isn't? | 21:05 |
| jeblair | apparently not | 21:05 |
| jeblair | for the love of | 21:05 |
| jeblair | okay, i'm stopping all of the image builds again | 21:05 |
| *** ajmiller has joined #openstack-infra | 21:06 | |
| jeblair | (and i verified on-host that if we downcase both of those variables we should be okay) | 21:06 |
| fungi | wow, WHAT? | 21:07 |
| fungi | dstufft: ^ is that for any particular reason, or just a bug? | 21:07 |
| dstufft | just a bug | 21:08 |
| fungi | okay, cool | 21:08 |
| dstufft | -> goes to get alyssa from bus | 21:08 |
| fungi | i was simply having a tough time imagining under what circumstances that would have been at all necessary | 21:08 |
| fungi | anybody writing up that pip issue? if not, i'll get that going now | 21:08 |
| jeblair | fungi: not i | 21:09 |
| *** dims has quit IRC | 21:09 | |
| jeblair | i'm unwinding images | 21:09 |
| fungi | thanks | 21:09 |
| *** _nadya_ has quit IRC | 21:09 | |
| *** jistr has quit IRC | 21:10 | |
| openstackgerrit | Merged openstack-dev/hacking: Updated from global requirements https://review.openstack.org/272226 | 21:11 |
| *** baoli has quit IRC | 21:11 | |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci: Pin to mirror.centos.org https://review.openstack.org/274241 | 21:11 |
| *** baoli has joined #openstack-infra | 21:12 | |
| jeblair | okay, images all cleaned up | 21:12 |
| *** achanda has quit IRC | 21:12 | |
| clarkb | fungi: its probably a straightforwrad fix to compare .lower()'d values too, if you want to do a pull request | 21:13 |
| fungi | yeah, i'm already cloning the repo | 21:14 |
| *** achanda has joined #openstack-infra | 21:14 | |
| fungi | surprised i didn't have it cloned before now | 21:14 |
| *** rbrndt has joined #openstack-infra | 21:14 | |
| clarkb | heads up, if your US passport expires this year the US state department is saying you should renew it now (they expect a huge surge in renewals this year for raisins) | 21:15 |
| * Clint squints. | 21:16 | |
| *** baoli has quit IRC | 21:16 | |
| openstackgerrit | James E. Blair proposed openstack-infra/project-config: Lowercase mirror host https://review.openstack.org/274242 | 21:16 |
| jeblair | clarkb, fungi, others: if you could examine that very carefully, that may save us some hours :) | 21:17 |
| *** baoli has joined #openstack-infra | 21:17 | |
| jeblair | (though i have individually tested the components to the best of my ability) | 21:17 |
| *** sabeen1 has joined #openstack-infra | 21:17 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Add gate tests for windwill https://review.openstack.org/274243 | 21:18 |
| *** sabeen2 has joined #openstack-infra | 21:19 | |
| nibalizer | ohai | 21:19 |
| clarkb | jeblair: I have tested that ocmmand against echo $NODEPOOL_MIRROR_HOST|tr '[:upper:]' '[:lower:]' and it prints all lower | 21:19 |
| clarkb | lgtm | 21:19 |
| * nibalizer has the jetlag | 21:19 | |
| clarkb | er | 21:19 |
| clarkb | I used UPPER.lower.MiXeD | 21:20 |
| *** rfolco has quit IRC | 21:20 | |
| *** abregman|nb has quit IRC | 21:20 | |
| fungi | checked for obvious non-lower-case uses of any of our pypi mirror names in obvious places too (like pypi.DFW getting set for our long-lived servers for example), and see nothing concerning | 21:22 |
| *** sabeen1 has quit IRC | 21:22 | |
| *** baoli has quit IRC | 21:22 | |
| mrmartin | nibalizer: Brussels? | 21:23 |
| *** baoli has joined #openstack-infra | 21:23 | |
| nibalizer | mrmartin: yep | 21:24 |
| *** kzaitsev_mb has quit IRC | 21:24 | |
| mrmartin | nibalizer: cool, I think we can sit down tomorrow or on sunday if you have a little time and you could help me to finish the askbot.o.o openstackid integration. | 21:25 |
| mrmartin | 30min max. | 21:25 |
| mrmartin | I can buy you a beer in exchange | 21:25 |
| nibalizer | sure | 21:26 |
| mrmartin | unfortunately no palinka this time. | 21:26 |
| *** rguillebert has joined #openstack-infra | 21:26 | |
| nibalizer | baw | 21:26 |
| mrmartin | but I can take on for you to Austin if you like | 21:26 |
| mrmartin | on / one | 21:26 |
| *** kzaitsev_mb has joined #openstack-infra | 21:27 | |
| *** peristeri has quit IRC | 21:28 | |
| *** kgiusti has left #openstack-infra | 21:28 | |
| *** sdake has quit IRC | 21:29 | |
| nibalizer | :) | 21:30 |
| *** _amrith_ is now known as amrith | 21:31 | |
| *** SumitNaiksatam has quit IRC | 21:32 | |
| *** rfolco has joined #openstack-infra | 21:32 | |
| openstackgerrit | Spencer Krum proposed openstack-infra/puppet-statusbot: Allow statusbot to take a hash of auth nicks https://review.openstack.org/274252 | 21:32 |
| dstufft | fungi: back now, but I suspect the issue is just that the most obvious thing to do when I wrote the code was a string equality check :| | 21:33 |
| openstackgerrit | Merged openstack-infra/project-config: Lowercase mirror host https://review.openstack.org/274242 | 21:33 |
| jeblair | nibalizer: 'respond_to' is duck typing in puppet? | 21:33 |
| *** zeih has joined #openstack-infra | 21:33 | |
| fungi | clarkb: jeblair: dstufft: i have what i suspect is the case sensitivity fix for pip (along with extending the test to catch regression of this), now i just need to refresh my memory on github's workflow | 21:33 |
| nibalizer | jeblair: ya | 21:34 |
| *** rfolco has quit IRC | 21:34 | |
| clarkb | fungi: click the fork button on pip, then add your fork as a remote to your repo, git push fungi my-branch-name, then go to gui and click the pull request button | 21:34 |
| fungi | clarkb: thanks, that's roughly what i remember doing before | 21:34 |
| jeblair | or email a patch ;) | 21:35 |
| fungi | heh | 21:35 |
| nibalizer | heh | 21:35 |
| *** baoli has quit IRC | 21:35 | |
| *** thorst_ has joined #openstack-infra | 21:36 | |
| *** baoli has joined #openstack-infra | 21:36 | |
| *** notsogentle has quit IRC | 21:36 | |
| jeblair | okay, i manually copied that into place on nodepool and will kick off image builds again | 21:36 |
| *** rcernin has joined #openstack-infra | 21:36 | |
| jeblair | Nth time's the charm | 21:36 |
| jeblair | for sufficiently large values of N | 21:37 |
| *** achanda has quit IRC | 21:38 | |
| *** zeih has quit IRC | 21:38 | |
| openstackgerrit | Spencer Krum proposed openstack-infra/system-config: Add nibalizer to statusbot nicks https://review.openstack.org/274256 | 21:38 |
| openstackgerrit | Spencer Krum proposed openstack-infra/system-config: Move statusbot_auth_nicks to hiera https://review.openstack.org/274257 | 21:38 |
| *** dims has joined #openstack-infra | 21:39 | |
| *** thorst_ has quit IRC | 21:39 | |
| *** thorst_ has joined #openstack-infra | 21:41 | |
| fungi | dstufft: i probably did it all thumbs, but https://github.com/pypa/pip/pull/3435 | 21:42 |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Add gate tests for windmill https://review.openstack.org/274243 | 21:42 |
| dstufft | fungi: looks fine to me, will merge when tests pass | 21:42 |
| *** ashleighfarnham has quit IRC | 21:43 | |
| *** jgriffith is now known as jgriffith_away | 21:43 | |
| *** sdake has joined #openstack-infra | 21:43 | |
| *** thorst__ has joined #openstack-infra | 21:43 | |
| *** notsogentle has joined #openstack-infra | 21:43 | |
| fungi | dstufft: awesome, thanks for looking | 21:44 |
| *** notsogentle has quit IRC | 21:44 | |
| *** vgridnev has quit IRC | 21:44 | |
| *** notsogentle has joined #openstack-infra | 21:45 | |
| *** thorst_ has quit IRC | 21:46 | |
| openstackgerrit | James E. Blair proposed openstack/gertty: Support fetching more than 1k changes post 2.8 https://review.openstack.org/274260 | 21:47 |
| jeblair | Clint: ^ computers are hard | 21:47 |
| *** jsavak has quit IRC | 21:49 | |
| *** jsavak has joined #openstack-infra | 21:49 | |
| *** mrmartin has quit IRC | 21:50 | |
| fungi | oh wow. need to see if i've made the same pagination error anywhere | 21:50 |
| Clint | jeblair: woot! | 21:51 |
| *** vgridnev has joined #openstack-infra | 21:53 | |
| *** AndyU has joined #openstack-infra | 21:53 | |
| *** derekh has joined #openstack-infra | 21:53 | |
| *** mrmartin has joined #openstack-infra | 21:53 | |
| nibalizer | fungi: di you attach this 'reviewable' thing to that? or some bot? | 21:54 |
| *** notsogentle has quit IRC | 21:55 | |
| *** notnownikki has quit IRC | 21:55 | |
| *** vgridnev has quit IRC | 21:56 | |
| *** notsogentle has joined #openstack-infra | 21:56 | |
| *** achanda has joined #openstack-infra | 21:57 | |
| *** mrmartin has quit IRC | 21:58 | |
| AndyU | Newby here trying to gether some info. I'd like to know what openstack infra requires concerning how donated infrastucture is provided. I saw the recent announcement concerning the HPE Helion cloud. If by chance it were possible to persuade a company to donate infrastructure what does openstack infra require? For example, do you need hands on site access? Is the donor expected to provide the hardware maintenance and support? I | 22:00 |
| AndyU | Any published info? | 22:00 |
| clarkb | AndyU: there is published info, let me get a link | 22:01 |
| AndyU | thx | 22:01 |
| *** pradk has quit IRC | 22:01 | |
| clarkb | AndyU: http://docs.openstack.org/infra/system-config/contribute-cloud.html | 22:01 |
| clarkb | AndyU: happy to answer questions about that once you have a chance to read it over | 22:01 |
| jeblair | i think i just found a way to perform local mergability checks without touching the git working tree | 22:02 |
| AndyU | I saw that.That's actually what prompted me to ask here. | 22:03 |
| *** kzaitsev_mb has quit IRC | 22:04 | |
| clarkb | AndyU: typically the ask is that you provide a cloud we can talk to that can run the instances described on that page | 22:04 |
| AndyU | ok, do you require the ability to come on site and be hands on? | 22:05 |
| clarkb | AndyU: in that case we wouldn't need/want on site access and the donor would be expected to care for the cloud. Much like using a public cloud | 22:05 |
| clarkb | (though the cloud need not be made available to the general public) | 22:05 |
| AndyU | ok, good to know. that would make the idea easier to sell | 22:05 |
| nibalizer | AndyU: the idea is that the openstack api is the service interface | 22:06 |
| nibalizer | so if you want us to stay below 200 nodes, set our quota for instance | 22:06 |
| nibalizer | other than that, create cloud credentials, and we use those + openstack to do our thing :) | 22:06 |
| AndyU | ok good. I'd heard that separate physical space had to be established, site access, network isolation, site access, etc was required. I wanted to fact check that | 22:07 |
| jeblair | AndyU: where did you hear that? | 22:08 |
| AndyU | That could have been a pretty tough sell. What you describe is good | 22:08 |
| AndyU | Just from a colleague | 22:08 |
| jeblair | AndyU: okay, well, anything you can share about where your colleague heard that would be helpful as it could help us correct a source of misinformation that might prevent other people from being interested. | 22:09 |
| jeblair | AndyU: feel free to privately message one of us on that subject if you wish | 22:10 |
| *** rlandy has quit IRC | 22:10 | |
| AndyU | Understood. Thanks! Gonna try and get some traction on the idea and with any luck I may have cause to ask more quewstions ;-) | 22:10 |
| *** doug-fis_ has joined #openstack-infra | 22:10 | |
| jeblair | AndyU: thank you! | 22:11 |
| AndyU | <fingers crossed> | 22:11 |
| fungi | nibalizer: i know not what i did, so suspect that is a bottish thingamajigger | 22:11 |
| *** dprince has quit IRC | 22:12 | |
| *** doug-fi__ has joined #openstack-infra | 22:14 | |
| *** gordc has quit IRC | 22:14 | |
| *** doug-fish has quit IRC | 22:14 | |
| *** thorst_ has joined #openstack-infra | 22:14 | |
| fungi | AndyU: awesome! also as we bring more donors into production we list them in an "infrastructure donors" section at http://www.openstack.org/foundation/companies/ for some additional exposure/thanks | 22:16 |
| *** angdraug has quit IRC | 22:16 | |
| clarkb | fungi: we need to update that to include bluebox | 22:16 |
| * clarkb will ping people | 22:17 | |
| fungi | clarkb: yep, if we're full-on using bb now, please do | 22:17 |
| *** doug-fis_ has quit IRC | 22:17 | |
| *** thorst__ has quit IRC | 22:17 | |
| clarkb | and I suppose leave up hpe because they host the infra cloud stuff | 22:17 |
| *** tdurakov_ has joined #openstack-infra | 22:17 | |
| clarkb | (thought we aren't using it yet) | 22:17 |
| *** salv-orlando has quit IRC | 22:18 | |
| *** doug-fi__ has quit IRC | 22:18 | |
| *** changbl has quit IRC | 22:18 | |
| *** kzaitsev_mb has joined #openstack-infra | 22:19 | |
| *** dmsimard has quit IRC | 22:19 | |
| fungi | clarkb: well, we're "using" it, just not for nodepool yet (though it could be argued some on our team are heavily testing openstack with it) | 22:20 |
| jeblair | er | 22:20 |
| fungi | but yeah, production grey area there | 22:20 |
| *** sdake has quit IRC | 22:20 | |
| fungi | we're getting some useful insights into what openstack installation at a modest scale is like | 22:21 |
| *** burgerk has quit IRC | 22:22 | |
| *** Qiming has joined #openstack-infra | 22:22 | |
| *** dmsimard has joined #openstack-infra | 22:22 | |
| *** ashleighfarnham has joined #openstack-infra | 22:24 | |
| *** e0ne has joined #openstack-infra | 22:24 | |
| *** kzaitsev_mb has quit IRC | 22:25 | |
| cody-somerville | <3s grafana.openstack.org -- so much easier to read and interpret and also very pretty. | 22:25 |
| *** Qiming has quit IRC | 22:26 | |
| *** doug-fish has joined #openstack-infra | 22:26 | |
| *** tonytan4ever has quit IRC | 22:27 | |
| *** thorst_ has quit IRC | 22:28 | |
| *** HeOS has joined #openstack-infra | 22:28 | |
| cody-somerville | What is internap btw? | 22:28 |
| *** kzaitsev_mb has joined #openstack-infra | 22:29 | |
| *** dimtruck is now known as zz_dimtruck | 22:29 | |
| *** sbalukoff has quit IRC | 22:29 | |
| *** doug-fish has quit IRC | 22:31 | |
| fungi | cody-somerville: an isp and hosting company | 22:31 |
| fungi | i used to use them for a lot of tier-2 connectivity in the northeast usa | 22:32 |
| fungi | but they've recently-ish acquired a cloud provider who's running openstack | 22:32 |
| *** ybathia has quit IRC | 22:32 | |
| *** trown is now known as trown|outttypeww | 22:33 | |
| clarkb | they have a fairly large worldwide presence too, multiple datacenters in north america, europe, and asia | 22:33 |
| clarkb | (its not just NE USA) | 22:33 |
| *** kzaitsev_mb has quit IRC | 22:35 | |
| jeblair | https://jenkins05.openstack.org/job/gate-gnocchi-pep8/822/console | 22:35 |
| jeblair | that just used the new mirror successfully | 22:36 |
| *** kzaitsev_mb has joined #openstack-infra | 22:36 | |
| jeblair | so i think the new images should be okay | 22:36 |
| clarkb | fungi: your pip change failed tests, looks like you don't always get a string that cna be lowered | 22:37 |
| jeblair | and we can declare the afs mirrors to be in production | 22:37 |
| *** e0ne_ has joined #openstack-infra | 22:37 | |
| fungi | clarkb: ooh, fun! | 22:37 |
| jeblair | http://cacti.openstack.org/cacti/graph.php?action=zoom&local_graph_id=3104&rra_id=5&view_type=&graph_start=1454092663&graph_end=1454107063 | 22:37 |
| fungi | jeblair: awesome! | 22:38 |
| *** dims has quit IRC | 22:38 | |
| *** e0ne has quit IRC | 22:39 | |
| jeblair | we need to add another read-only site for the pypi mirror | 22:40 |
| jeblair | (it's currently only in dfw) | 22:40 |
| jeblair | http://paste.openstack.org/show/485499/ is the command for anyone interested | 22:40 |
| openstackgerrit | Merged openstack-infra/tripleo-ci: Pin to mirror.centos.org https://review.openstack.org/274241 | 22:41 |
| jeblair | anyone feel like flexing their afs admin muscles to add the secondary RO site? | 22:42 |
| nibalizer | jeblair: haha im game | 22:42 |
| nibalizer | basically need to add server afs01.ord.openstack.org partition /vicepa RO Site but to the mirror.pypi | 22:42 |
| *** cody-somerville has quit IRC | 22:43 | |
| jeblair | nibalizer: yep; according to http://docs.openstack.org/infra/system-config/afs.html#creating-a-volume i think 'vos addsite' and 'vos release' are all you need to do | 22:43 |
| *** diana_clarke has quit IRC | 22:43 | |
| *** rajinir has quit IRC | 22:45 | |
| jeblair | nibalizer: oh, you need to make yourself an afs admin first :) | 22:45 |
| *** e0ne has joined #openstack-infra | 22:45 | |
| nibalizer | heh | 22:45 |
| jeblair | nibalizer: so follow the trail that starts here: http://docs.openstack.org/infra/system-config/afs.html#adding-a-user | 22:45 |
| nibalizer | so addprinc? | 22:45 |
| *** e0ne_ has quit IRC | 22:45 | |
| clarkb | comparing time to install deps on that job before and after new mirror is a pretty consistent 7 minutes | 22:46 |
| jeblair | nibalizer: yeah; http://docs.openstack.org/infra/system-config/kerberos.html#addprinc then http://docs.openstack.org/infra/system-config/afs.html#adding-a-user then http://docs.openstack.org/infra/system-config/afs.html#adding-a-user | 22:46 |
| *** e0ne_ has joined #openstack-infra | 22:46 | |
| jeblair | clarkb: yeah, we ran the numbers monday and with a warm cache, it's very comparable. | 22:47 |
| jeblair | clarkb: with a cold cache, it's generally not bad, except for ovh, where it can be slow enough that we had to update the pip timeout. but the cold cache situation shouldn't last for long. | 22:48 |
| clarkb | and in theory a wheel mirror is going to speed that up considerably so overall should be a win | 22:48 |
| jeblair | yup | 22:48 |
| *** kzaitsev_mb has quit IRC | 22:49 | |
| *** e0ne has quit IRC | 22:49 | |
| prometheanfire | DIB cores, can you look at | 22:54 |
| prometheanfire | DIB cores, can you look at https://review.openstack.org/#/c/273769 upstream won't release a fix without core reviews on it | 22:55 |
| *** amitgandhinz has quit IRC | 22:55 | |
| *** sbalukoff has joined #openstack-infra | 22:57 | |
| *** sabeen2 has quit IRC | 22:57 | |
| greghaynes | upstream? | 22:57 |
| prometheanfire | https://github.com/openstack-packages/diskimage-builder/blob/rpm-master/diskimage-builder.spec#L26 | 22:58 |
| prometheanfire | that is in their repo but not deployed | 22:58 |
| prometheanfire | it's annoying | 22:58 |
| *** sridhar_ram has quit IRC | 22:59 | |
| greghaynes | ah, downstream | 22:59 |
| prometheanfire | derekh: hi | 22:59 |
| prometheanfire | derekh: you might be able to explain it more | 22:59 |
| nibalizer | jeblair: sorry I'm trying to find a bite before I sit down to do this, and belgium is not exactly cooperating | 22:59 |
| *** eharney has quit IRC | 23:00 | |
| *** bpokorny_ has quit IRC | 23:00 | |
| jeblair | nibalizer: no rush; do you want to get back to it later or would you rather i do it? | 23:00 |
| *** sigmavirus24 is now known as sigmavirus24_awa | 23:00 | |
| clarkb | nibalizer: I can recommend a couple places if you are in brussells | 23:01 |
| *** bpokorny has joined #openstack-infra | 23:01 | |
| *** diana_clarke has joined #openstack-infra | 23:01 | |
| *** diana_clarke has quit IRC | 23:01 | |
| jeblair | also, liege is the right kind of waffle to get | 23:01 |
| *** kzaitsev_mb has joined #openstack-infra | 23:01 | |
| nibalizer | jeblair: i really want to do it | 23:02 |
| nibalizer | clarkb: the problem is its late and I want something fast, but please do reccomend | 23:02 |
| jeblair | nibalizer: ok, all yours :) | 23:03 |
| *** jsavak has quit IRC | 23:03 | |
| *** notsogentle has quit IRC | 23:03 | |
| clarkb | nibalizer: Toukoul | 23:04 |
| greghaynes | yes | 23:04 |
| greghaynes | that place | 23:04 |
| clarkb | trying to find that other place we ate at that had belgian good and was decent but google is not helping | 23:05 |
| jeblair | closed 1hr ago? | 23:05 |
| clarkb | oh time differences :( | 23:05 |
| greghaynes | clarkb: oh, I bet I can find it | 23:06 |
| *** EricGonczer_ has quit IRC | 23:07 | |
| *** baoli has quit IRC | 23:07 | |
| greghaynes | hrm, not seeing a lot of places in that area that are also open :/ | 23:08 |
| *** yamamoto has joined #openstack-infra | 23:08 | |
| greghaynes | also, were there that many streets when we were there? ;) | 23:08 |
| *** derekh has quit IRC | 23:08 | |
| *** EricGonczer_ has joined #openstack-infra | 23:09 | |
| clarkb | greghaynes: my taxi driver last ight was from ethiopia and asked if I liked ethiopian food and had to tell him about brussels | 23:11 |
| *** yamamoto has quit IRC | 23:12 | |
| *** bryan_att has joined #openstack-infra | 23:12 | |
| nibalizer | i have found late night gyros | 23:12 |
| *** tdurakov_ has quit IRC | 23:13 | |
| *** notsogentle has joined #openstack-infra | 23:14 | |
| *** sabeen1 has joined #openstack-infra | 23:16 | |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci: Provide fallback mirrors for centos pin https://review.openstack.org/274287 | 23:17 |
| clarkb | nibalizer: win | 23:18 |
| *** salv-orlando has joined #openstack-infra | 23:18 | |
| *** EricGonczer_ has quit IRC | 23:23 | |
| *** EricGonczer_ has joined #openstack-infra | 23:23 | |
| nibalizer | jeblair: okay so how do I get up to kadmin? | 23:25 |
| jeblair | nibalizer: use kadmin.local on the kdc | 23:26 |
| nibalizer | ah! | 23:26 |
| jeblair | nibalizer: (after you bootstrap yourself, you can use plain old kadmin remotely in the future) | 23:26 |
| nibalizer | ok | 23:27 |
| nibalizer | ami creating nibz@OPENSTACK.ORG or nibz/admin@OPENSTACK.ORG | 23:27 |
| nibalizer | I think the later | 23:28 |
| fungi | you need both, yeah? | 23:30 |
| *** dkranz has quit IRC | 23:31 | |
| jeblair | nibalizer: yeah, do both | 23:32 |
| jeblair | nibalizer: but for the rest of the steps you'll use nibz/admin | 23:33 |
| *** alivigni has quit IRC | 23:33 | |
| *** EricGonczer_ has quit IRC | 23:34 | |
| nibalizer | cool | 23:34 |
| nibalizer | i have achieved principals | 23:34 |
| *** HeOS has quit IRC | 23:34 | |
| jeblair | woot! you're a kerberos admin now! | 23:35 |
| jlk | clarkb: so my guy says that "it should work assuming you are using sub 1500 MTUs" | 23:36 |
| *** EricGonczer_ has joined #openstack-infra | 23:37 | |
| *** EricGonczer_ has quit IRC | 23:37 | |
| *** AndyU has quit IRC | 23:38 | |
| *** notsogentle has quit IRC | 23:38 | |
| clarkb | jlk ok probably something else then. (we do reduce mtu) | 23:38 |
| *** zul has joined #openstack-infra | 23:41 | |
| fungi | clarkb: jlk: just for my own edification, shouldn't it also work with 1500 (or greater) mtu, but just result in ip fragments on the provider's network when the additional gre header overhead ticks that over the real network's mtu? | 23:41 |
| jlk | fungi: I do not know | 23:41 |
| fungi | not that i expect that to be happening, but still, fragmentation is a very real part of the internet protocol | 23:41 |
| clarkb | fungi except thst neutron has many l2 interfaces to traverse that wont fragmdnt | 23:41 |
| clarkb | so fragmentation cant really work with neutron | 23:42 |
| fungi | oh, right, if it's going through neutron and hits an mtu drop somewhere that isn't a layer-3 router | 23:42 |
| *** notsogentle has joined #openstack-infra | 23:42 | |
| fungi | hence the ml thread(s) recently on fixing neutron to support the internet | 23:43 |
| fungi | er, i mean, to dtrt with mtus ;) | 23:43 |
| prometheanfire | so, why isn't dib a port of openstack-infra? | 23:43 |
| fungi | prometheanfire: i have no idea what you mean by "port" in this context | 23:44 |
| *** ybathia has joined #openstack-infra | 23:44 | |
| prometheanfire | part, under infra's umbrella | 23:45 |
| prometheanfire | not part of tripleo | 23:45 |
| fungi | prometheanfire: dib sprung out of the tripleo team's work. the infra team simply started to make use of it | 23:45 |
| fungi | but tripleo was using it long before we did | 23:45 |
| prometheanfire | ah | 23:46 |
| fungi | and it was written by them for a use case they had specifically | 23:46 |
| greghaynes | That's the great thing about making unix-ey tools | 23:46 |
| greghaynes | they can be useful for a lot of differnt use cases | 23:47 |
| fungi | and also shows a success of one of openstack's teams reusing a project written to scratch an itch in another one | 23:47 |
| prometheanfire | true | 23:47 |
| fungi | which we try to do whenever possible | 23:47 |
| fungi | (i.e., avoid reinventing the wheel) | 23:48 |
| prometheanfire | I'm just annoyed at being behind their constantly broken gate :( | 23:48 |
| greghaynes | well, I think tripleo has very little to do with that - the tripleo job is not voting, even | 23:48 |
| greghaynes | although we do try to not land stuff unless it passes there | 23:48 |
| *** salv-orlando has quit IRC | 23:49 | |
| prometheanfire | they said it'd be fixed next week :| | 23:49 |
| *** e0ne_ has quit IRC | 23:49 | |
| greghaynes | I dont remember saying that... | 23:50 |
| greghaynes | We are under the influence of a lot of outside forces under our control, which is really the crux of the problem | 23:50 |
| openstackgerrit | Derek Higgins proposed openstack-infra/tripleo-ci: Minimise memory usage for deployed overcloud https://review.openstack.org/273431 | 23:50 |
| greghaynes | er, outside forces *not* under our control | 23:50 |
| prometheanfire | 15:16 < prometheanfire > bnemec: I know it's being worked on, but mind letting me know when I can run recheck? | 23:50 |
| prometheanfire | 15:16 < bnemec > prometheanfire: I wouldn't count on it happening before next week. :-( | 23:50 |
| greghaynes | That seems like a good lower bound | 23:51 |
| prometheanfire | having gate fail for that long? | 23:51 |
| greghaynes | well, in the ideal world we would never break, I just mean given the level of breakage | 23:52 |
| prometheanfire | that bad :P | 23:52 |
| prometheanfire | just don't like being stalled is all, it means I can't continue work in other areas that depend on this | 23:53 |
| fungi | though this points out that better direct test coverage of expectations in dib might avoid having to indirectly test it by running other tripleo stuff to make sure dib doesn't break it | 23:53 |
| greghaynes | Yep, openstack is pretty much a high latency high bandwidth development process | 23:53 |
| prometheanfire | very async | 23:53 |
| fungi | prometheanfire: most of us have a dozen or more things we work on in parallel and do a sort of event-driven workflow of switching context to whatever's unblocked at a given point in time | 23:54 |
| greghaynes | fungi: ya, there are a few patches in dib specifically for that | 23:54 |
| greghaynes | which have been blocked on merging because of CI fails these past couple weeks | 23:54 |
| greghaynes | but soon! (TM) | 23:54 |
| *** notsogentle has quit IRC | 23:54 | |
| prometheanfire | greghaynes: that will be a time for rejoicing :P | 23:55 |
| *** sridhar_ram has joined #openstack-infra | 23:56 | |
| *** apoorvad_ has joined #openstack-infra | 23:57 | |
| *** uberjay_ has joined #openstack-infra | 23:57 | |
| *** cody-somerville has joined #openstack-infra | 23:58 | |
| *** cody-somerville has quit IRC | 23:58 | |
| *** cody-somerville has joined #openstack-infra | 23:58 | |
| *** yolanda_ has joined #openstack-infra | 23:59 | |
| *** bhunter_ has joined #openstack-infra | 23:59 | |
| *** uberjay has quit IRC | 23:59 | |
| *** hughsaunders has quit IRC | 23:59 | |
| *** apoorvad has quit IRC | 23:59 | |
| *** bhunter71 has quit IRC | 23:59 | |
| *** yolanda has quit IRC | 23:59 | |
| *** rubasov has quit IRC | 23:59 | |
| *** yolanda_ is now known as yolanda | 23:59 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!