Friday, 2018-10-12

*** ijw has quit IRC00:00
*** Swami has quit IRC00:01
tonybHow do I set required for all jobs in a repo/project (without creating my own jobs?)  I thought I could do somethign like: https://review.openstack.org/#/c/609823/2/.zuul.yaml but that didn't work00:09
ianwtonyb: i think it's a job-specific option00:14
ianwtonyb: to be more specific, the answer is probably to make a base template with that, and inherit00:19
tonybianw: Okay I think I understand00:26
*** gyee has quit IRC00:29
pabelangerianw: mind looking at https://review.openstack.org/609855/ so when clarkb gets back we can get a +300:30
pabelangercurrently nodepool release jobs are failing00:30
pabelangerhttp://lists.openstack.org/pipermail/release-job-failures/2018-October/000972.html00:31
* fungi looks too00:32
pabelangerthanks!00:32
*** longkb has joined #openstack-infra00:33
fungido we need to reenqueue a tag once that merges?00:36
fungior do you have it covered?00:36
*** mattoliverau has joined #openstack-infra00:37
pabelangerI can, if needed. Other wise, happy to defer to someone else00:39
openstackgerritMerged openstack-infra/project-config master: Fix syntax error with zuul-tarball job  https://review.openstack.org/60985500:48
openstackgerritIan Wienand proposed openstack/diskimage-builder master: Minor documentation updates  https://review.openstack.org/60963600:52
pabelangerzuul enqueue-ref --tenant=openstack --trigger=gerrit --pipeline=release --project=openstack-infra/nodepool --ref=refs/tags/3.3.0 --newrev=907251740903ed8c44eb9556d7a34a1e32abd47600:54
pabelangerfungi: clarkb: ianw: how does ^ look00:54
*** eernst has quit IRC00:55
fungilgtm. you can also feed the inventory.yaml url from the failed run to the reenqueue.py script in my homedir on the zuul server00:56
pabelangeroh, neat. let me try that00:56
fungiand then run the command it spits out00:57
pabelanger++00:58
pabelangerthanks00:58
pabelangerokay, jobs reenqueued00:59
ianwsorry, missed this, but LGTM :)00:59
pabelangeryah, reenqueue.py works great01:00
pabelangerokay, worked. nodepool 3.3.0 on tarballs.o.o and pypi01:03
pabelangertristanC: ^01:03
pabelangerOh01:04
pabelangerHmm01:04
pabelangerfungi: clarkb: I think we build a python201:04
pabelangerhttp://logs.openstack.org/90/907251740903ed8c44eb9556d7a34a1e32abd476/release/release-zuul-python/dbd3fb201:05
openstackgerritIan Wienand proposed openstack/diskimage-builder master: Add a pre-finalise.d phase  https://review.openstack.org/60986301:06
clarkbyou have to set setup.cfg info to build a python3 wheel with pbr iirc01:06
clarkbpabelanger: you should be able to install thr sdist though01:06
pabelangerclarkb: http://git.openstack.org/cgit/openstack-infra/project-config/tree/playbooks/zuul-tarball/run.yaml#n501:07
pabelangerI guess that is python2 on the node01:07
fungi[wheel] universal = 101:07
clarkbthe python version that makes the wheel shouldnt matter01:08
clarkbits an output problem01:08
pabelangeryah, we don't seem to set that in nodepool01:08
clarkbbut the sdist wont care01:08
fungioh, or 3-only01:08
clarkbso your pip install will still work01:08
pabelangernodepool-3.3.0-py2-none-any.whl01:08
pabelangerI would expect that to be nodepool-3.3.0-py3-none-any.whl01:09
pabelangerlike nodepool-3.2.001:09
fungiif you don't set universal then need to build wheel under a python3 interpreter01:09
pabelangerokay, if that is the case, then our run.yaml playbook is not correct01:09
pabelangerwe likey should use python301:09
clarkbpabelanger: the versiom of python that makes the wheel doesnt matter  iirc01:11
clarkbpbr uses config to determine what the output is01:11
clarkboh fungi says the version does matter01:12
clarkbin any case the release that was just made should work via sdist01:12
fungiyeah, sdist will  be fine01:12
openstackgerritPaul Belanger proposed openstack-infra/project-config master: Force python3 for zuul-tarball job  https://review.openstack.org/60986601:13
pabelangerfungi: clarkb: okay, so if I follow, ^ should build the wheel as python3. And today, if we did pip3 nodepool, we won't get the wheel since it only is python201:14
clarkbright you get the sdist01:14
pabelangerokay01:14
clarkbso you can install nodepool just fine01:14
clarkbbut it will take a little more time01:14
*** bobh has joined #openstack-infra01:24
fungisorry, wasn't really able to type earlier01:25
fungithe major interpreter version determines whether wheel (or setup.py bdist_wheel) creates a whl filename with py or py301:26
fungipy2 or py301:26
fungiunless you set [bdist_wheel] universal = 1 in setup.cfg then it will say any instead (implying it works with both)01:27
fungiformerly [wheel] but that's deprecated in favor of [bdist_wheel] in the most recent wheel release01:27
*** rh-jelabarre has joined #openstack-infra01:29
*** mrsoul has joined #openstack-infra01:30
tonybCan I Depends-On a chnage in openstack-zuul-jobs and have it do what I expect? or is openstack-zuul-jobs a trusted repo list project-config?01:36
fungiyou should be able to depends-on a change for it and see the effects01:40
tonybfungi: Thanks.01:42
openstackgerritTony Breeds proposed openstack-infra/openstack-zuul-jobs master: [DNM] Testing a new project-template for ceilometer  https://review.openstack.org/60990101:43
openstackgerritCorey Bryant proposed openstack-infra/zuul-jobs master: Add ensure-python role  https://review.openstack.org/60976101:45
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool master: Implement a Kubernetes driver  https://review.openstack.org/53555701:47
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool master: Implement an OpenShift resource provider  https://review.openstack.org/57066701:49
*** aidin has joined #openstack-infra01:55
*** eernst has joined #openstack-infra01:58
*** apetrich has quit IRC02:15
*** zigo has quit IRC02:29
*** yamamoto has quit IRC02:31
*** yamamoto has joined #openstack-infra02:31
*** roman_g has quit IRC02:39
*** aidin has quit IRC02:44
*** hongbin has joined #openstack-infra02:45
*** lbragstad has joined #openstack-infra02:45
*** aidin has joined #openstack-infra02:46
*** armax has joined #openstack-infra02:48
*** psachin has joined #openstack-infra02:53
*** rlandy|bbl is now known as rlandy03:00
*** rlandy has quit IRC03:02
*** bobh has quit IRC03:02
*** armax_ has joined #openstack-infra03:03
*** armax has quit IRC03:04
*** bobh has joined #openstack-infra03:05
*** dave-mccowan has quit IRC03:05
*** armax has joined #openstack-infra03:07
*** armax_ has quit IRC03:08
*** aidin has quit IRC03:10
*** bobh has quit IRC03:16
*** aidin has joined #openstack-infra03:37
*** armax has quit IRC03:49
*** udesale has joined #openstack-infra03:54
*** hongbin has quit IRC04:01
*** aidin has quit IRC04:02
*** dayou has quit IRC04:06
*** smcginnis has quit IRC04:06
*** lathiat has quit IRC04:06
*** lathiat has joined #openstack-infra04:06
*** felipemonteiro has joined #openstack-infra04:06
*** dayou has joined #openstack-infra04:06
*** EmilienM is now known as EvilienM04:13
openstackgerritMerged openstack-infra/zuul-sphinx master: Add attr-overview directive  https://review.openstack.org/60498004:16
*** bhavikdbavishi has joined #openstack-infra04:17
openstackgerritTony Breeds proposed openstack-infra/project-config master: Use root user for editing pip.conf  https://review.openstack.org/60990904:18
tonybianw: Can you look over ^^04:19
*** carl_cai has joined #openstack-infra04:20
*** lbragstad has quit IRC04:21
*** ykarel__ has joined #openstack-infra04:22
*** abhishekk has joined #openstack-infra04:27
abhishekkhi, in gate environment, while executing gate jobs is httpd server is running, if yes then which port it uses?04:28
*** icey has quit IRC04:30
*** mpeterson has quit IRC04:30
*** icey has joined #openstack-infra04:30
*** mpeterson has joined #openstack-infra04:30
fungiinfra-root: (and anyone else) just a heads up, i may not be available tomorrow through the weekend04:34
*** ykarel__ is now known as ykarel04:38
tonybabhishekk: I'm far from an expert but I don't think there is a web server running.04:46
tonybabhishekk: I guess a dsvm job with wsgi must have a virtual server for each endpoint04:47
tonybabhishekk: what do you need one for? and in which jobs?04:47
*** ramishra has joined #openstack-infra04:48
openstackgerritMerged openstack-infra/zuul master: Add a quick-start test job  https://review.openstack.org/60951404:50
*** lathiat_ has joined #openstack-infra04:51
chkumar|offShrews: go ahead, I am happy to review the code04:53
*** lathiat has quit IRC04:54
abhishekktonyb, thank you04:54
abhishekktonyb, I am doing something like downloading image from external network for functional test04:54
abhishekkbut those tests fails most of the time due to network issues04:55
abhishekkSotK, I am doing something like this https://review.openstack.org/#/c/608856/3/glance/tests/functional/v2/test_images.py04:55
tonybabhishekk: Ahh okay.04:55
tonybabhishekk: So yeah short term you'd be best placed to provide your own (either as you're doing) or as a stand-alone role/task in your functional setups04:57
*** eernst has quit IRC04:57
tonybabhishekk: There are per provider mirrors (for apt/yum/images) but I'm not sure exactly how you'd plumb them into your functaional jobs04:57
tonybabhishekk: It'd be worth discussin this on os-dev04:58
abhishekktonyb, yes, I am doing that but those tests are timing out now, where as they are working properly in local environment :(04:58
tonybabhishekk: which 'that' do you mean in the above statement?04:58
abhishekkthat means, running local http server and downloading file from that04:59
tonybabhishekk: for example the the link you gave you're pulling from https://www.openstack.org/assets/openstack-logo/ which isn't local to the provider05:00
abhishekktonyb, sorry look at the PS 5 from that patch05:00
tonybabhishekk: so even talkign to 127.0.0.1:8111 they're still failing?05:00
abhishekktonyb, yes05:00
*** eernst has joined #openstack-infra05:00
tonybabhishekk: Oh :( No idea then05:02
abhishekktonyb, ok, thank you for inputs though05:03
tonybabhishekk: I guess requests-mock isn't an option?05:04
abhishekktonyb, yes that's the last one, I guess in functional tests of glance we are not mocking anything05:04
tonybabhishekk: Yeah having a real sever is the best option05:05
abhishekktonyb, yes, lets see, I will post it on os-dev if not able to encounter the issue, thank you again05:05
tonybabhishekk: I havea  half formed idea abiyt providing a really basic one and launching that before the stestr run.05:06
* tonyb will stall on that until you post to os-dev to see if I can solidify it05:06
abhishekktonyb, sounds good05:06
*** toabctl has joined #openstack-infra05:07
*** ssbarnea has quit IRC05:11
*** janki has joined #openstack-infra05:15
*** pcaruana has joined #openstack-infra05:15
*** rh-jelabarre has quit IRC05:22
openstackgerritTony Breeds proposed openstack-infra/openstack-zuul-jobs master: Add ceilometer specific templates  https://review.openstack.org/60990105:23
*** ssbarnea has joined #openstack-infra05:30
*** olivierb has joined #openstack-infra05:31
*** felipemonteiro has quit IRC05:31
abhishekktonyb, looks like, I manage to pulled it off, I will make sure by recheking 3-4 times05:31
tonyb\o/05:32
abhishekk:D,05:32
*** chkumar|off is now known as chandankumar05:34
*** kjackal has joined #openstack-infra05:34
openstackgerritSimon Westphahl proposed openstack-infra/zuul master: Check paused parent on node failure of child job  https://review.openstack.org/60991105:38
*** jesusaur has quit IRC05:41
*** eernst has quit IRC05:45
*** aidin has joined #openstack-infra05:49
*** ykarel has quit IRC06:01
*** ykarel has joined #openstack-infra06:02
*** kopecmartin|ruck has quit IRC06:04
*** apetrich has joined #openstack-infra06:04
*** kopecmartin has joined #openstack-infra06:05
*** aidin has quit IRC06:10
*** jesusaur has joined #openstack-infra06:11
AJaegertonyb: you can do something like http://git.openstack.org/cgit/openstack-infra/project-config/tree/zuul.d/projects.yaml#n5226 as well - no need for a new template, just add the individual jobs06:12
prometheanfireif people have time, this dib change would be nice https://review.openstack.org/60986306:14
prometheanfirejust a couple lines06:14
*** aidin has joined #openstack-infra06:15
*** janki has quit IRC06:19
*** dpawlik has joined #openstack-infra06:30
openstackgerritAndreas Jaeger proposed openstack-infra/project-config master: Small cleanups for zuul.d/projects.yaml  https://review.openstack.org/60977806:31
*** aidin has quit IRC06:33
*** janki has joined #openstack-infra06:34
AJaegerconfig-core, three small cleanups for review, please: https://review.openstack.org/609793 https://review.openstack.org/609795 https://review.openstack.org/60977806:39
tonybAJaeger: I did try that but is seems like it'd be more prone to drift ?06:47
tonybAJaeger: Given we can use the new templates in multiple repos do you still think that adding the overritgh per job is ther right way?06:48
AJaegertonyb: if you have multiple users, I'm fine taking the template06:49
AJaegertonyb: I would like to give jd_ time to review https://review.openstack.org/609901 as well so that the telemetry team is aware of it.06:50
AJaegertonyb: I added jd_ now to the review...06:50
tonybAJaeger: Okay cool.  I'll work on it next week.  There isn't a great rush.  The powervm team have a work around06:51
tonybAJaeger: so did I ;P06:51
AJaegertonyb: so, go ahead and change neutron to ceilometer - and then let's wait a few days...06:51
AJaeger;)06:51
tonybAJaeger: Will do.06:51
*** kopecmartin is now known as kopecmartin|ruck06:52
openstackgerritTony Breeds proposed openstack-infra/openstack-zuul-jobs master: Add ceilometer specific templates  https://review.openstack.org/60990106:53
AJaegertonyb: still a few neutron left in your change ^06:54
tonybAJaeger: Next week I'll sync up with dhellmann to find out if/when he;ll migreat the ceilometer-zvm zuul config in repo so I can update it06:54
tonybAJaeger: :(06:54
AJaegertonyb: he will not update it - it's not an official repo06:55
AJaegertonyb: so do it yourself, please.06:55
AJaegertonyb: and dhellmann is on PTO next week06:55
tonybAJaeger: Ahh okay.  I'll find the tools and run them myself then06:56
AJaegertonyb: repo openstack/goal-tools - this checks governance repo for release teams, no idea whether it works with single repo...06:56
openstackgerritTony Breeds proposed openstack-infra/openstack-zuul-jobs master: Add ceilometer specific templates  https://review.openstack.org/60990106:57
tonybAJaeger: Thanks, and I think I got them all this time06:57
tonybAJaeger: I can probably hack it to work for a single repo as needed ;P06:58
tonybAJaeger: Thanks for your help I think I'm going knock off now before I mess anythign else up ;P06:58
AJaegernp, enjoy your evening/night!06:59
*** rcernin has quit IRC07:03
openstackgerritMerged openstack-infra/zuul-jobs master: Install docker from distro properly  https://review.openstack.org/60984407:03
*** jesusaur has quit IRC07:08
*** ykarel_ has joined #openstack-infra07:12
*** ykarel_ has quit IRC07:13
*** jesusaur has joined #openstack-infra07:13
*** ykarel_ has joined #openstack-infra07:14
*** ykarel has quit IRC07:14
*** bhavikdbavishi has quit IRC07:16
*** apetrich has quit IRC07:17
*** shardy has joined #openstack-infra07:18
*** jtomasek has joined #openstack-infra07:19
*** ykarel_ is now known as ykarel07:22
openstackgerritMerged openstack-infra/zuul master: Fixes for quick-start playbooks  https://review.openstack.org/60955107:23
openstackgerritMerged openstack-infra/zuul master: Remove disable-recommends in quick-start test  https://review.openstack.org/60955307:23
*** ginopc has joined #openstack-infra07:27
*** ginopc has quit IRC07:30
*** apetrich has joined #openstack-infra07:30
*** tosky has joined #openstack-infra07:45
*** jpich has joined #openstack-infra07:46
openstackgerritTobias Henkel proposed openstack-infra/zuul master: Retry jobs failed with MERGER_FAILURE  https://review.openstack.org/60994507:53
*** e0ne has joined #openstack-infra07:59
*** electrofelix has joined #openstack-infra08:04
*** d0ugal has quit IRC08:07
*** d0ugal has joined #openstack-infra08:08
*** ginopc has joined #openstack-infra08:08
*** rpittau has quit IRC08:09
*** rpittau has joined #openstack-infra08:10
openstackgerritAndreas Jaeger proposed openstack-infra/project-config master: Use tempest-pg-full  https://review.openstack.org/60995808:10
*** rpittau has quit IRC08:10
*** rpittau has joined #openstack-infra08:11
*** roman_g has joined #openstack-infra08:28
*** bauzas is now known as PapaOurs08:30
*** zigo has joined #openstack-infra08:34
*** derekh has joined #openstack-infra08:41
*** e0ne has quit IRC08:44
*** dpawlik has quit IRC08:46
*** e0ne has joined #openstack-infra08:46
*** slaweq has quit IRC08:46
*** dpawlik has joined #openstack-infra08:51
*** psachin has quit IRC08:55
openstackgerritMerged openstack-infra/zuul-jobs master: Add ensure-python role  https://review.openstack.org/60976108:56
*** janki has quit IRC08:58
*** janki has joined #openstack-infra08:58
*** janki has quit IRC09:03
*** janki has joined #openstack-infra09:04
*** janki has quit IRC09:05
*** janki has joined #openstack-infra09:05
*** tomaw has quit IRC09:07
*** tomaw has joined #openstack-infra09:08
*** ramishra has quit IRC09:09
*** ramishra has joined #openstack-infra09:09
*** e0ne has quit IRC09:13
*** e0ne has joined #openstack-infra09:16
*** dtantsur|afk is now known as dtantsur09:33
*** gnuoy has quit IRC09:35
*** gnuoy has joined #openstack-infra09:37
*** gnuoy has quit IRC09:38
*** gnuoy has joined #openstack-infra09:38
*** xinliang has quit IRC09:42
*** psachin has joined #openstack-infra09:45
*** gnuoy has quit IRC09:50
openstackgerritMerged openstack-infra/openstack-zuul-jobs master: Ensure python3.7 installed for openstack-tox-py37  https://review.openstack.org/60976609:51
*** imacdonn has quit IRC09:53
*** imacdonn has joined #openstack-infra09:53
*** gnuoy has joined #openstack-infra09:55
*** xinliang has joined #openstack-infra09:56
*** slaweq has joined #openstack-infra09:56
*** ykarel is now known as ykarel|lunch10:04
*** rkukura has quit IRC10:13
openstackgerritTristan Cacqueray proposed openstack-infra/zuul master: wip: add status graph rendering  https://review.openstack.org/61000110:27
*** psachin has quit IRC10:33
*** e0ne has quit IRC10:34
*** udesale has quit IRC10:36
*** pabelanger has quit IRC10:40
*** jpena|off has quit IRC10:41
*** rascasoft has quit IRC10:42
*** jpena|off has joined #openstack-infra10:42
*** agopi has quit IRC10:46
*** abhishekk has quit IRC10:46
*** psachin has joined #openstack-infra10:49
*** sshnaidm is now known as sshnaidm|off10:57
*** dayou has quit IRC11:09
*** apetrich has quit IRC11:10
*** ykarel_ has joined #openstack-infra11:11
*** ykarel|lunch has quit IRC11:14
*** dayou has joined #openstack-infra11:14
*** longkb has quit IRC11:20
*** e0ne has joined #openstack-infra11:27
*** psachin has quit IRC11:28
*** apetrich has joined #openstack-infra11:29
*** psachin has joined #openstack-infra11:33
*** bhavikdbavishi has joined #openstack-infra11:37
*** dave-mccowan has joined #openstack-infra11:38
*** bhavikdbavishi has quit IRC11:46
*** rh-jelabarre has joined #openstack-infra11:46
*** bhavikdbavishi has joined #openstack-infra11:46
*** bobh has joined #openstack-infra12:01
*** kjackal has quit IRC12:04
openstackgerritTobias Henkel proposed openstack-infra/nodepool master: Fix skipping of max-servers if not set  https://review.openstack.org/61001912:06
*** kjackal has joined #openstack-infra12:07
*** bobh has quit IRC12:09
*** sthussey has joined #openstack-infra12:11
dhellmanntonyb , AJaeger : I think I made the migration tools take a list of repos as argument, but if not patches to do so would be fine12:15
*** e0ne has quit IRC12:16
AJaegerdhellmann: indeed, you do - the scripts handle teams, but underlying tool uses repos. Great!12:17
*** e0ne has joined #openstack-infra12:17
dhellmanncool12:17
dhellmannin other news, until my isp comes back I have IRC and web but my dev box can't see the internet :-/12:18
AJaegerdhellmann: start your vacation a day earlier ;)12:18
dhellmannI like the way you think, AJaeger12:18
ssbarneadhellmann AJaeger : I need some help with one infra issue, I need a permalink to the fedora qcow image, one that current has a floating name at https://nb02.openstack.org/images/ -- how can I get one?12:19
AJaegerssbarnea: both of us are the wrong persons to help you with that.12:20
dhellmannyeah, unfortunately I don't know much about how images are managed12:20
dhellmannwhat causes the name to keep changing?12:20
ssbarneai don't know who to ask,...12:20
AJaegerI doubt that there's an easy way - so, best explain the use case...12:20
AJaegerssbarnea: any of the admins - fungi, clarkb, pabelanger, mordred, Shrews might be able to help12:20
ssbarneai don't want to add extra heuristics for looking for the last image, is better to update the process that is udpating it to create a symlink.12:21
ssbarneathanks, lets hope they will be able to tell me where it can be done. usually is not hard to create a symlink.12:21
dhellmannhow do we pick the latest image for the other types of images?12:21
AJaegerssbarnea: explain them your use case ;) The names are only used by nodepool and are not meant for you to consume at all...12:22
*** smarcet has joined #openstack-infra12:23
*** smarcet has quit IRC12:24
*** ansmith has joined #openstack-infra12:24
ssbarneaAJaeger: dhellmann : we are trying to use the same image for undercloud deploymentm see https://review.openstack.org/#/c/591540/57/config/environments/standalone_fedora_libvirt.yml12:25
AJaegerssbarnea: and those images are available uploaded in our clouds - we push them from nb02 inside each cloud12:26
AJaegerssbarnea: are you aware of http://nl01.openstack.org/image-list ?12:27
AJaegerssbarnea: might be best to write an email to openstack-infra explaining the use case, showing the review and asking for help. Using nb02.openstack.org/images/ is the wrong answer ;)12:28
ssbarneaAJaeger: I think that we already know that but i need a temporary solution, so we can continue with f28 enablement. Still, if the image is changing every day, it becomes impossible to test.12:30
ssbarneaweshay: ^^ any ideas?12:31
AJaegerinfra-root, please see above ^12:32
AJaegerssbarnea: can't help constructively...12:32
*** ramishra has quit IRC12:34
*** rlandy has joined #openstack-infra12:40
weshayssbarnea, we don't need to use that image locally man12:41
weshayhowever the links changing all the time w/o softlinks to a perm link is understandable12:41
weshayhard12:41
weshayssbarnea, use the fedora cloud qcow man12:41
Shrewsssbarnea: AJaeger: the image name format contains a unique identifier (generated by zookeeper) that's necessary for nodepool operations. That's how the name is stored there. I'm not inclined to make large changes in nodepool to alter that for an external entity12:41
*** psachin has quit IRC12:45
fricklerssbarnea: I may have missed some earlier discussion on that, but can you remind us why you prefer an infra-built image to the default upstream cloud image?12:49
*** jtomasek has quit IRC12:49
Shrewswe might be able to offer a new web endpoint to expose the latest. still seems rather fragile to depend on our images (we could remove it at any time, and break you)12:50
frickleronly possible approach would be to create a job that builds an image and publishes it on tarballs.o.o, independent from what nodepool does12:51
*** kgiusti has joined #openstack-infra12:51
*** yamamoto has quit IRC12:54
*** yamamoto has joined #openstack-infra12:54
Shrewsssbarnea: here are the dib elements we use in nodepool to build our fedora-28 image: http://git.openstack.org/cgit/openstack-infra/project-config/tree/nodepool/nodepool.yaml#n16212:56
ssbarneaShrews: thanks!12:57
*** mriedem has joined #openstack-infra12:58
*** yamamoto has quit IRC12:59
*** agopi has joined #openstack-infra13:02
*** bobh has joined #openstack-infra13:06
*** elod has quit IRC13:06
*** elod has joined #openstack-infra13:06
openstackgerritTobias Henkel proposed openstack-infra/nodepool master: Ensure that completed handlers are removed frequently  https://review.openstack.org/61002913:07
*** dtantsur is now known as dtantsur|afk13:07
*** agopi has quit IRC13:07
*** carl_cai has quit IRC13:10
*** yamamoto has joined #openstack-infra13:10
*** eharney has joined #openstack-infra13:10
*** bhavikdbavishi1 has joined #openstack-infra13:13
mordredssbarnea: yah - I think bulding one using the same elements for your use would be better - because then you can exclude the element that includes every git repo in infra - which should make the resulting images a much more reasonable size without having a functional impact13:13
*** bhavikdbavishi has quit IRC13:13
*** markvoelker has quit IRC13:15
*** bhavikdbavishi1 has quit IRC13:17
*** ssbarnea_ has quit IRC13:18
*** mriedem is now known as hansmoleman13:22
*** munimeha1 has joined #openstack-infra13:22
*** lbragstad has joined #openstack-infra13:23
odyssey4mefolks, I see network configuration was added to glean in https://github.com/openstack-infra/glean/commit/fa417426ae4c1892f6a855be142b0060fe946a77 - except it's only added for gentoo... is there a reason to hold that back, rather than have it apply to a broader set of distributions?13:24
odyssey4mewhen using glean on bionic we're finding that the DNS isn't configured, so this is a bit of a hindrence13:25
odyssey4methe issue doesn't apply in infra because infra is using unbound and disables any other way to configure DNS on the nodes13:25
*** ijw has joined #openstack-infra13:26
*** ijw has quit IRC13:26
*** ijw has joined #openstack-infra13:27
*** kjackal has quit IRC13:28
*** kjackal has joined #openstack-infra13:31
*** agopi has joined #openstack-infra13:32
*** munimeha1 has quit IRC13:38
openstackgerritDavid Shrewsbury proposed openstack-infra/nodepool master: Implement a Kubernetes driver  https://review.openstack.org/53555713:39
openstackgerritDavid Shrewsbury proposed openstack-infra/nodepool master: Add tox functional testing for drivers  https://review.openstack.org/60951513:39
*** boden has joined #openstack-infra13:40
mordredodyssey4me: we should be doing network config for ubuntu as well ... OH - networkd, not network13:40
*** dansmith is now known as SteelyDan13:41
mordredodyssey4me: no, I don't think there is a specific reason, we just haven't had anybody interested in it yet13:41
*** ijw has quit IRC13:41
*** stephenfin is now known as finucannot13:41
odyssey4memordred: well, I think I've figured out the actual issue - networkd only does the ip bits, no dns13:42
odyssey4mecurrently glean puts down a resolv.conf for the resolvers13:42
openstackgerritTobias Henkel proposed openstack-infra/nodepool master: Ensure that completed handlers are removed frequently  https://review.openstack.org/61002913:42
odyssey4mebut if resolved is enabled at all, it will overwrite that, and DNS resolution fails because resolved doesn't have any DNS configuration done13:42
odyssey4meso I guess the solution here would be to allow glean to configure resolved or modify resolve.conf13:43
Shrewsianw: i hope the port thing *is* a neutron bug. are you suggesting we wait to find out if it is before continuing with the nodepool change?13:44
Shrewsi guess that's probably the sanest thing to do13:45
mordredodyssey4me: yeah. although - avoiding resolved is the thing I'd personally rather do, given that it doesn't know how to properly do dns13:45
*** felipemonteiro has joined #openstack-infra13:46
mordredbut if it's unavoidable for some reason, I imagine the better thing to do would be to have glean actually configure it13:46
*** ijw has joined #openstack-infra13:47
mordredstrigazi: heya - if you have a sec, the magnum functional test job for openstacksdk has stopped working and I don't know why ... http://logs.openstack.org/17/608317/3/check/openstacksdk-functional-devstack-magnum/11eaab9/job-output.txt.gz13:52
*** rpittau has quit IRC13:54
openstackgerritDavid Shrewsbury proposed openstack-infra/nodepool master: Add tox functional testing for drivers  https://review.openstack.org/60951513:54
*** munimeha1 has joined #openstack-infra13:57
*** bdodd has quit IRC14:03
*** efried is now known as fried_rice14:05
*** ijw has quit IRC14:05
*** bdodd has joined #openstack-infra14:06
*** ijw has joined #openstack-infra14:07
*** eharney has quit IRC14:09
*** smcginnis has joined #openstack-infra14:09
*** ijw_ has joined #openstack-infra14:10
*** ijw has quit IRC14:12
prometheanfireodyssey4me: mordred, so, add resolvd support to gleans dns config for networkd?14:14
prometheanfiremordred: I think we should define supported OS versions, my question is what we should enable networkd on14:15
prometheanfirealso, does this look like it's checking the wrong file to anyone else (not looking at /etc/systemd/network) https://github.com/openstack-infra/glean/commit/fa417426ae4c1892f6a855be142b0060fe946a77#diff-1b42769ab0f2150100d2c2c1c090b512R54014:16
*** pabelanger has joined #openstack-infra14:16
*** yamamoto has quit IRC14:18
*** yamamoto has joined #openstack-infra14:18
*** lewo has joined #openstack-infra14:19
*** ykarel__ has joined #openstack-infra14:20
*** ykarel_ has quit IRC14:22
openstackgerritSorin Sbarnea proposed openstack/gertty master: Change the default location for gertty config file  https://review.openstack.org/58186814:22
*** yamamoto has quit IRC14:24
*** yamamoto has joined #openstack-infra14:24
*** lbragstad is now known as elbragstad14:25
*** munimeha1 has quit IRC14:32
openstackgerritMonty Taylor proposed openstack-infra/nodepool master: Add universal flag to setup.cfg  https://review.openstack.org/61005714:35
*** ijw_ has quit IRC14:35
*** ijw has joined #openstack-infra14:35
*** xinliang has quit IRC14:36
*** haleyb has quit IRC14:38
*** smarcet has joined #openstack-infra14:39
*** ansmith has quit IRC14:40
*** markvoelker has joined #openstack-infra14:41
*** bnemec is now known as beekneemech14:44
*** eharney has joined #openstack-infra14:44
openstackgerritMatthew Thode proposed openstack-infra/glean master: check networkd files  https://review.openstack.org/61006414:44
openstackgerritJames E. Blair proposed openstack-infra/system-config master: Add opendev nameservers (1/2)  https://review.openstack.org/60509214:49
openstackgerritJames E. Blair proposed openstack-infra/system-config master: Add opendev nameservers (2/2)  https://review.openstack.org/61006614:49
*** felipemonteiro has quit IRC14:56
*** ansmith has joined #openstack-infra14:57
*** e0ne has quit IRC14:58
prometheanfiremordred: so, for networkd and resolved, we can just enable resolved if networkd is enabled and it will just work (if the symlink exists), should I make the glean-networkd unit file depend/start resolved?14:59
openstackgerritMerged openstack-infra/project-config master: Force python3 for zuul-tarball job  https://review.openstack.org/60986615:02
clarkbwhy wouldnt you let the image build decide if resolvd should be enabled?15:03
clarkbI dont think glean should do that?15:03
prometheanfireclarkb: glean writes out /etc/resolv.conf, which overrides resolved15:04
prometheanfiresee the last bullet here https://www.freedesktop.org/software/systemd/man/systemd-resolved.service.html#/etc/resolv.conf15:04
*** ansmith has quit IRC15:05
pabelanger--skip-dns is a thing for glean15:05
pabelangerthat shouldn't touch resolv.conf15:05
clarkbya glean doesnt set dns for infra15:05
clarkbwe configure nameserver to localhost and run unbound and glean doesnt touch it15:05
clarkbI think you can do similar with resolved15:06
prometheanfiretrue, can just add that to ExecStart=%%GLEANSH_PATH%%/glean.sh --distro networkd15:06
*** e0ne has joined #openstack-infra15:06
prometheanfirethough resolved would interfere if you wish to use unbound locally15:07
pabelangerhttp://git.openstack.org/cgit/openstack-infra/project-config/tree/nodepool/elements/nodepool-base/glean@.service.d/override.conf15:07
pabelangeris the default ARGS we use for glean15:07
pabelangerI guess gentoo isn't using --interface?15:07
clarkbprometheanfire: yes I dont think infra would want resolved15:08
clarkbIm saying configuring dns like that should be image build choice not glean decision15:08
clarkbthen I can choose to use unbound or resolved15:08
clarkbor point at google or cloudflare directly15:09
prometheanfireso it'd have to be optional, somehow, maybe two diferent glean-networkd.service files, one disabling dns/resolved and one enabling it, toggled by the --skip-dns flag15:10
clarkb(I think default is whatever dhcp says to do and however your dhcp client configures it)15:10
clarkbprometheanfire: why does glean have to know? I guess the case where dhcp isnt used?15:10
clarkbcan we jus say that isnt supported and more  people should use dhcp?15:11
openstackgerritMonty Taylor proposed openstack-infra/zuul master: Add __init__.py to web directory  https://review.openstack.org/61008015:11
prometheanfireI doubt it, iirc some providers only do static still15:11
AJaegerconfig-core, please review https://review.openstack.org/609958 https://review.openstack.org/609795 https://review.openstack.org/60977815:12
clarkbprometheanfire: actually it would work to set the statically configured resolvers in resolv.conf right?15:14
clarkbyou dont get resolved but dns will work via libc15:14
*** rpioso|afk is now known as rpioso15:14
prometheanfireglean-networkd as it esists now prepares stuff for resolved, if the end user wishes to use resolved they need to both set an override.conf for glean-networkd.service and symlink /etc/resolv.conf to /run/systemd/resolve/stub-resolv.conf15:15
openstackgerritMonty Taylor proposed openstack-infra/zuul master: Add __init__.py to web directory  https://review.openstack.org/61008015:15
prometheanfireclarkb: ya, if /etc/resolv.conf is a file then it 'wins' and resolvd just uses the contents from /etc/resolv.conf15:15
clarkbthen it is already functional?15:16
prometheanfireyes, as long as we are fine with my previous comment15:16
prometheanfireglean-networkd as it esists now prepares stuff for resolved, if the end user wishes to use resolved they need to both set an override.conf for glean-networkd.service and symlink /etc/resolv.conf to /run/systemd/resolve/stub-resolv.conf15:16
*** bhavikdbavishi has joined #openstack-infra15:16
clarkbI dont really care as long as dns is functional on boot which it should be15:17
*** dpawlik has quit IRC15:17
prometheanfireya, it would be15:17
clarkbif you want something more that seems out of scope for glean15:17
clarkblike infra using unbound is independnt of glean15:17
*** jtomasek has joined #openstack-infra15:18
prometheanfiremy point with bringing this up is that using glean means you need to do even more extra work to allow resolved to work15:18
prometheanfirebut it sounds like everyone is fine with that15:18
prometheanfireso meh15:18
*** ansmith has joined #openstack-infra15:18
*** gyee has joined #openstack-infra15:19
*** adriancz has quit IRC15:20
*** ijw has quit IRC15:20
prometheanfireodyssey4me: so, looks like glean doesn't need to care about resolved, at least for now it will set up dns for /etc/resov.conf but not systemd-resolved15:21
*** ijw has joined #openstack-infra15:21
clarkbprometheanfire: one of the reasons glean exists is that cloud-init does far too much for you. Things that should be handled by your image build or configuration management. I'm trying to sort out the balance of working networking (and dns) with glean in this context and I personally err on the side of dns will work on boot so glean has done its job15:23
clarkbprometheanfire: if users want something different (like infra using unbound) that is an image build/configuration management responsibility so that you can make the correct decisions for you15:24
prometheanfireclarkb: and that's fine, it's easier for me to not write code than to write code :P15:25
openstackgerritMonty Taylor proposed openstack-infra/zuul master: Use zuul/web/static in setup_hook  https://review.openstack.org/60976415:26
clarkbprometheanfire: one thing we might be able to assume is that if you are explicitly using networkd (so gentoo today and not the other distros glean supports) maybe we write the symlink with glean (and don't make it a set of options?15:30
clarkbprometheanfire: I think the distros that aren't resolved by default would find it really odd to get rsolved support generally15:30
clarkbbut I also don't know how common resolved is if networkd is used? I know resolved requires networkd but not the other way around iirc15:32
odyssey4meclarkb prometheanfire I was thinking more along the lines of if systemd-resolved is enabled, then write the right config for it. If not, then do what it does now (write /etc/resolv.conf). And do much the same for systemd-networkd.15:32
clarkbodyssey4me: that also seems like a reasonable approach if glean can determine that easily (I think it can at least via systemctl)15:33
clarkbthen your image build or config mgmt can explicitly enable a feature and glean will just honor that15:33
odyssey4meIt's more to make glean work with whatever is in place, rather than have it compete - which is exactly what we just found happened on bionic - in bionic resolved is on by default.15:33
clarkb(really what we want to avoid doing is surprising the user like cloud init can do)15:33
odyssey4me*systemd-resolved15:33
clarkbodyssey4me: oh is it? I may avoid upgrading to bionic at home then :P15:33
* clarkb grumbles something about dns over dbus being bad15:34
clarkbat least they fixed the known security bugs with it :(15:34
prometheanfireodyssey4me: the config for which nameservers to use is already set up if using systemd-networkd15:34
odyssey4meyep, that's pretty much what I'm thinking - glean should just respect whatever's in the image... for now we've disabled systemd-resolved in our image, because meh15:34
odyssey4meprometheanfire: alright, in that case then writing out the systemd-networks config should be based on whether that service is enabled IMO15:35
prometheanfirethe main question is if glean should use resolved if also using networkd, I think people don't want resolved to be tied to networkd like that15:35
prometheanfireodyssey4me: more words (less pronouns) :P15:35
odyssey4meprometheanfire: no, I think it should only write the config if the service is present and enabled15:35
prometheanfireodyssey4me: right now it only writes the networkd files if networkd is enabled15:36
prometheanfireit already does what you ask15:36
prometheanfirethe config for resolved is just the DNS= entries in the network files, glean already writes that info to /etc/resov.conf15:37
*** e0ne has quit IRC15:37
prometheanfireif resolved is disabled that DNS= info does nothing iirc15:37
odyssey4meprometheanfire: well, on a bionic host it wrote the config, and DNS was totally broken - I had to mask systemd-resolved and re-run glean to get it operational again15:37
prometheanfireso we are already doing the right thing15:38
prometheanfireodyssey4me: what is 'the config'15:38
prometheanfireand what is 'it'15:38
prometheanfirepronoun games15:38
odyssey4meprometheanfire: this was the rsulting set for eth0 on the bionic host (note no DNS present): http://paste.openstack.org/show/731976/15:39
clarkbodyssey4me: was /etc/resolv.conf written though?15:40
*** e0ne has joined #openstack-infra15:40
prometheanfireodyssey4me: oh, I guess we don't write out the DNS= lines then15:40
*** e0ne has quit IRC15:40
openstackgerritMonty Taylor proposed openstack-infra/zuul master: Add line to setup.cfg limiting to python >=3.5  https://review.openstack.org/61008915:42
odyssey4meclarkb: it was written, then systemd-resolved realised that /etc/resolv.conf was different to what it liked and changed it... and DNS resolution broke because there were no resolvers configured any more15:42
clarkbah15:42
prometheanfireclarkb: odyssey4me mordred, do you think that the networkd config should write out those lines then?15:42
prometheanfireodyssey4me: that part (overwriting resolv.conf) is the OS's fault, not glean15:42
prometheanfireit's not overwritten on gentoo at least15:42
openstackgerritMonty Taylor proposed openstack-infra/nodepool master: Add line to setup.cfg limiting to python >=3.5  https://review.openstack.org/61009015:42
odyssey4meprometheanfire: it's systemd-resolved's fault, yes15:43
odyssey4mesystemd-resolved changes /etc/resolv.conf to contain 'nameserver 127.0.0.53', and then expects '/etc/systemd/resolved.conf' to contain the resolvers I think... although it may get those from systemd-networkd's interface configs, I don't know15:45
*** rlandy is now known as rlandy|biab15:45
mordredclarkb: yeah - I think having glean write the info to all the various places is a good idea15:45
clarkbodyssey4me: ok given that (dns not working on boot as I expected) I think the best option is to attempt to configure resolved if enabled15:46
odyssey4me++15:46
prometheanfireodyssey4me: it can read from networkd config too, let me check15:46
*** dpawlik has joined #openstack-infra15:47
prometheanfirewhich is what I'd prefer to configure, leave the distro resolved.conf file alone15:47
*** kjackal_v2 has joined #openstack-infra15:48
*** kjackal has quit IRC15:48
openstackgerritDoug Hellmann proposed openstack-infra/project-config master: update the branching script to include a semver bump  https://review.openstack.org/60982715:49
*** markvoelker has quit IRC15:51
*** dpawlik has quit IRC15:51
*** ansmith has quit IRC15:51
*** markvoelker has joined #openstack-infra15:52
prometheanfiremordred: clarkb it looks like most of the time dns info is given globally, should those dns servers be configured for all interfaces?15:53
clarkbssbarnea: following up on testing images, you should be able to add jobs in zuul that run on the fedora-28 image and have zuul do much of the heavy lifting for you15:54
clarkbprometheanfire: I'm not sure I understand since dns is a layer 7 service that runs on top of the network stack15:55
ssbarneaclarkb: yep, i know. we already have one CR that is creating such job. Still, there is a long chain to fixes we need to do till this will merge.15:55
prometheanfireclarkb: it's based on how the info is handed off to glean in /mnt/config/openstack/latest/network_data.json15:55
clarkbprometheanfire: right and dns info isn't interface specific there. And the dns service on the host isn't interface specific either?15:56
prometheanfirewhile openstack CAN specify the dns service per interface, it usually doesn't15:56
*** ansmith has joined #openstack-infra15:57
prometheanfireand resolved can also do dns per interface15:57
prometheanfiresee https://paste.pound-python.org/show/gM5ezN6lqofVD2aFbq5t/ for an example of dns per interface15:57
clarkbhow does that work for a process resolving a name?15:57
clarkbroutes should determine how you talk to the resolver that is configured, interfaces don't directly come into it15:58
clarkbI guess the analog is dhcp on interface foo says dns server is xyz and dhcp on another interface cna say somethin different15:59
prometheanfireya15:59
clarkbin the case of global config from config-drive I would set it globally if possible otherwise setting it on each interface is probably the closest behavior match?15:59
prometheanfireit also allows for having multiple default routes and allows for complex routing15:59
clarkbwith debuntu you can set the dns config globally iirc16:00
*** ssbarnea has quit IRC16:00
clarkbnot sure if resolvd/networkd do that16:00
prometheanfireclarkb: ya, globally is done in /etc/systemd/resolved.conf though (which is likely fine if we only replace #FallbackDNS= with the values from glean)16:00
prometheanfirenetworkd doesn't do dns on it's own16:01
*** derekh has quit IRC16:01
*** finucannot is now known as stephenfin16:01
*** panda has quit IRC16:01
prometheanfireactually it has a DNS= there, we could use that16:01
prometheanfire           A space-separated list of IPv4 and IPv6 addresses to use as system DNS servers. DNS requests are sent to one of the listed DNS servers in parallel to suitable per-link DNS servers acquired from systemd-networkd.service(8) or set at runtime by external applications. For compatibility reasons, if this16:02
prometheanfire           setting is not specified, the DNS servers listed in /etc/resolv.conf are used instead, if that file exists and any servers are configured in it. This setting defaults to the empty list.16:02
*** panda has joined #openstack-infra16:02
clarkbamorin are you still around? (it is late on a friday I know). Wondering if we should disable nodepool launches on ovh gra1 while its a bit unstable and if that is a known issue on your side (happy to help debug from our side if we can)16:02
prometheanfireso, write to both /etc/resolv.conf and the DNS= line in /etc/systemd/resolved.conf16:02
clarkbprometheanfire: that sounds like what we want for the global values16:03
*** ginopc has quit IRC16:04
prometheanfireyep16:04
*** janki has quit IRC16:04
prometheanfiretoo bad I can't do lineinfile in glean :P16:04
clarkbthe systemd ini configs are probably readable by config parser? (I have never tested this)16:05
clarkbif that works its stdlib so no deps and should make editing the configs simpler than direct string manipulation16:05
pabelangerclarkb: at one point, devstack jobs (I think) had a custom index.html file on logs.o.o, but I cannot seem to find that. Is that still a thing in zuulv3?16:06
prometheanfirehmm, probably, I'll have to see16:08
clarkbpabelanger: yes it is still athing, looks like devstack-gate jobs have it but not the new jobs16:08
clarkbpabelanger: I'm guessing that was missed in the zuulv3ification16:09
pabelangerthanks16:09
clarkbpabelanger: I think it has to do with file paths16:09
pabelangerhttp://logs.openstack.org/53/606853/2/check/legacy-tempest-dsvm-neutron-full-centos-7/8fa2241/16:09
clarkbpabelanger: the legacy jobs write logs to different paths than the new jobs so the conditional sin't hit? I'm still looking for where we do that16:09
openstackgerritMerged openstack-infra/zuul master: Check paused parent on node failure of child job  https://review.openstack.org/60991116:10
clarkbpabelanger: the content comes from devstack-gate/help/*16:10
openstackgerritMerged openstack-infra/zuul master: Optionally disable service worker in zuul-web  https://review.openstack.org/60962616:10
pabelangerclarkb: got it thanks! I think once we switch openstack to use upload-logs-swift role, we maybe look to see how we can allow jobs to add their own html content too.16:11
clarkbpabelanger: http://git.openstack.org/cgit/openstack-infra/system-config/tree/modules/openstack_project/manifests/static.pp#n210 is the config that consumes those "logs"16:11
pabelanger++16:12
*** bobh has quit IRC16:12
*** eernst has joined #openstack-infra16:14
hogepodgeI was hoping to get some infra eyes on this patch. It changes behavior in the gate that's mean to "save time and bandwidth". I want to make sure it's not putting undue load on infra if it merges. https://review.openstack.org/#/c/609646/816:15
openstackgerritJames E. Blair proposed openstack-infra/zuul master: WIP: Run dstat and generate graphs in unit tests  https://review.openstack.org/61010016:17
*** jpich has quit IRC16:17
*** hansmoleman is now known as mriedem_away16:18
clarkbhogepodge: it should be fine particularly if you are using the pypi caching mirrors in the local cloud region16:19
*** ykarel__ has quit IRC16:19
evrardjpclarkb: infra has reverse proxy for PyPI now, or do we still mirror?16:22
evrardjpmaybe the reverse proxy are caching? I am confused by your sentence :)16:23
clarkbevrardjp: we switched to a reverse proxy for pypi when it became untenable to keep mirroring pypi with the gigabytes of machine learning packages being pushed there every day16:23
evrardjpbut yeah it seems it is using the infra mirrors anyway16:23
clarkbevrardjp: and yes the reverse proxies cache16:23
*** bobh has joined #openstack-infra16:23
evrardjpok that clarifies my understanding :)16:24
*** rlandy|biab is now known as rlandy16:24
evrardjpor confirms whatever the right word is.16:24
clarkbevrardjp: mirroring pypi requires more than 2TB of disk now and is growing at many gigabytes per day.16:24
clarkbso we gave up16:24
*** tknp has joined #openstack-infra16:24
*** tknp has left #openstack-infra16:24
*** ssbarnea has joined #openstack-infra16:25
clarkbhttps://pypi.org/project/tf-nightly-gpu/#files packages like that16:25
clarkbthat one package is about a gigabyte per day16:25
*** fried_rice is now known as fried_rolls16:27
evrardjpinternet scale!16:29
*** eernst has quit IRC16:32
openstackgerritMonty Taylor proposed openstack-infra/zuul master: Add reference to svg icon in the PWA manifest  https://review.openstack.org/61010516:33
*** eernst has joined #openstack-infra16:34
*** eernst has quit IRC16:34
*** eernst has joined #openstack-infra16:34
*** irclogbot_2 has joined #openstack-infra16:35
prometheanfireok, got the networkd side done16:35
*** shardy has quit IRC16:37
openstackgerritMatthew Thode proposed openstack-infra/glean master: write dns info to networkd  https://review.openstack.org/61010716:42
*** irclogbot_2 has quit IRC16:42
prometheanfireclarkb, odyssey4me: part one ^16:42
clarkbprometheanfire: that doesn't handle the global case?16:44
openstackgerritMonty Taylor proposed openstack-infra/zuul master: Add icons in the PWA manifest  https://review.openstack.org/61010516:46
prometheanfireclarkb: not yet, I just wanted to get some code up, that's why I -W'd it16:47
clarkbgotcha16:47
prometheanfirethat's also the hard part, but if config parser works it should be easier16:48
clarkblooks like packethost mirror crashed again. This implies it isn't a disk issue, or the attempt to fully provision the disk did not work16:56
clarkbthe last thing in syslog is ansible running16:58
openstackgerritMerged openstack-infra/nodepool master: Add universal flag to setup.cfg  https://review.openstack.org/61005717:04
clarkbjobs are failing in pre when that happens so I think the impact is not catastrophic, but still would be good to understand it17:05
*** Bhujay has joined #openstack-infra17:07
*** SpamapS has quit IRC17:13
*** SpamapS has joined #openstack-infra17:15
*** trown is now known as trown|lunch17:16
prometheanfireI suppose I should only write out to /etc/systemd/resolved.conf if that file already exists (aka if resolved is installed)17:17
*** e0ne has joined #openstack-infra17:28
openstackgerritClark Boylan proposed openstack-infra/zuul master: WIP: Run dstat and generate graphs in unit tests  https://review.openstack.org/61010017:32
*** lujinluo has joined #openstack-infra17:38
lujinluoHi, does anyone know who I should contact regarding my account in review.openstack.org? please kindly advise, thanks17:44
clarkblujinluo: I can help. What would you like to know?17:44
lujinluothanks clarkb!17:45
lujinluoi have an account with email address lujinluo at gmail.com17:45
lujinluobut people are having trouble adding that account as reviewers on review.openstack.org17:45
lujinluothe error message is "Lujin Luo <lujinluo@gmail.com> does not identify a registered user or group"17:46
clarkblujinluo: the issue is you have two accounts with that email address so that key isn't unique17:46
clarkblujinluo: if you give me a few minutes I can dig up details on the two accounts. While I am doing that can you tell me what account id number you have at https://review.openstack.org/#/settings/17:47
lujinluoI see. on my side, the one that i am using is with Account ID 2858117:47
lujinluothanks! clarkb17:48
*** Bhujay has quit IRC17:50
clarkblujinluo: is the username that you push code with 'lujin' ?17:51
lujinluoyes17:51
clarkbok 28765 is the other account. What we can do is update the openid on that account to be for 28581, then disable 2876517:52
clarkbI'll work on getting that done17:52
lujinluothank you so much for your help! clarkb17:52
clarkbmordred: ^ maybe you can stick around for a few minute to be extra eyeballs on that since its sql update. I am going to get prepped and have a thing for you in a few minutes17:52
AJaegerconfig-core, please review https://review.openstack.org/609909 https://review.openstack.org/609958 https://review.openstack.org/609795 https://review.openstack.org/60977817:52
*** armax has joined #openstack-infra17:55
clarkbmordred: corvus: UPDATE account_external_ids SET account_id = '28581' WHERE account_id = '28765';17:56
clarkbdoes that look right?17:56
mordredclarkb: yes, I thnk so?17:58
clarkbAJaeger: on https://review.openstack.org/#/c/609958/1 the new zuulv3 job shouldn't use devstack-gate at all can we just remove that job from d-g?17:59
clarkbmordred: its cool to update the column being selected on?17:59
*** ssbarnea_ has joined #openstack-infra18:00
Shrewsclarkb: yup18:00
clarkbShrews: mordred ok cool running that now18:00
clarkblooks like it worked and only updated the one row18:02
clarkblujinluo: I think you should be all set now18:03
lujinluoawesome. clarkb let me do a quick double check18:03
lujinluocool clarkb yeah! i can now add that email address18:04
*** armax has quit IRC18:05
lujinluoclarkb: if it is not too much, would it be possible to delete/hide my old account? i no long have access to that account18:05
lujinluoit would be fine if it is too hard18:05
clarkblujinluo: yes I disabled the old account, that was necessary to get the email to work as a key18:07
clarkblujinluo: I moved the openid from the old ccount to the current account with the db update, then disabled the old account18:07
lujinluosorry i was not clear enough. i have another account with another email address, luo.lujin at jp.fujitsu.com18:08
lujinluoall three of them share the same "Lujin Luo"18:08
lujinluoentry18:08
clarkboh18:08
lujinluoso people always get confused which one to pick18:09
openstackgerritMerged openstack-infra/project-config master: Use root user for editing pip.conf  https://review.openstack.org/60990918:09
clarkblujinluo: that account is 1749118:09
lujinluosince i no longer have access to the previous company email address, i miss out all the review requests go to that account18:09
lujinluoyes clarkb18:10
clarkblujinluo: do we need to move the openid for that one too? that one is a bit harder because the email addresses differ18:10
clarkbor is it enough to just disable the account? we can reenable it later if necessary too18:10
cgoncalveshi. looking for a DIB core to review https://review.openstack.org/#/c/609169/ :)18:10
lujinluoif it is possible let's move the openid18:10
clarkblujinluo: its possible just not sure if I have to sanitize the email too18:11
*** eernst has quit IRC18:11
clarkbfungi probably knows but isn't around today. Maybe we can double check with him when he is back (hopefully next week)18:12
lujinluoclarkb: ok, got it.18:12
lujinluothanks for your help today clarkb! :-)18:12
lujinluohave a good day!18:13
clarkblujinluo: you too18:13
*** eernst has joined #openstack-infra18:13
*** trown|lunch is now known as trown18:14
*** eernst has quit IRC18:14
clarkbnot sure if anyone else has noticed this but firefox seems to be rejecting ssl certs on some sites and it isn't clear why. The UI renders a green padlock. I haven't noticed this on any of our sites, but if found should probably let us know18:14
pabelangerclarkb: which version of firefox? I remember seeing something about latest version dropping old cert providers18:15
clarkb63 beta 1118:16
AJaegerclarkb: sure, can remove it...18:17
openstackgerritMerged openstack-infra/zuul master: Add __init__.py to web directory  https://review.openstack.org/61008018:20
openstackgerritAndreas Jaeger proposed openstack-infra/project-config master: Remove legacy-tempest-dsvm-neutron-pg-full  https://review.openstack.org/60995818:20
AJaegerclarkb: done ^18:20
clarkbAJaeger: thanks18:20
*** olivierb has quit IRC18:21
*** eernst has joined #openstack-infra18:22
*** eernst has quit IRC18:23
openstackgerritJames E. Blair proposed openstack-infra/zuul master: Simplify Builds page  https://review.openstack.org/61013718:30
openstackgerritJames E. Blair proposed openstack-infra/zuul master: Add more information to build page  https://review.openstack.org/61013818:30
openstackgerritMerged openstack-infra/openstack-zuul-jobs master: Remove obsolete legacy-tempest-dsvm-py35-full-devstack-plugin-ceph  https://review.openstack.org/60979518:30
*** eharney has quit IRC18:34
*** e0ne has quit IRC18:37
openstackgerritAndreas Jaeger proposed openstack-infra/project-config master: Cleanup devstack-gate jobs  https://review.openstack.org/60995818:40
AJaegerclarkb: found some more jobs that we can remove and update my change ^18:40
*** smarcet has quit IRC18:41
*** eharney has joined #openstack-infra18:45
*** ijw has quit IRC18:45
*** kmalloc is now known as needscoffee18:48
*** bhavikdbavishi has quit IRC18:51
openstackgerritJames E. Blair proposed openstack-infra/zuul master: WIP: Run dstat and generate graphs in unit tests  https://review.openstack.org/61010018:51
clarkbneedscoffee: ++ to healthcheck by default rather than configurable addon. That is all18:55
needscoffeeclarkb: ^_^18:57
needscoffeeclarkb: i hate it when things are "configurable" like that. and it seemed useful.18:58
needscoffeethat email thread just confirmed my choice was a good one.18:58
clarkbit seems odd to support such a feature and not turn it on by default18:59
needscoffeewell we had it in our default paste-ini19:02
needscoffee100% agree19:02
openstackgerritMerged openstack-infra/zuul-jobs master: Create ansible-galaxy-import role  https://review.openstack.org/60847219:02
AJaegerclarkb: could you review https://review.openstack.org/#/c/609778/ as well, please?19:08
openstackgerritMonty Taylor proposed openstack-infra/zuul master: Remove unneeded cruft from bdist_wheel fix attempts  https://review.openstack.org/61014319:08
*** fried_rolls is now known as fried_rice19:10
*** rkukura has joined #openstack-infra19:16
AJaegerthanks, clarkb19:18
openstackgerritDavid Moreau Simard proposed openstack-infra/zuul-jobs master: Add a role to install and configure dstat to run in the background  https://review.openstack.org/51837419:20
openstackgerritDavid Moreau Simard proposed openstack-infra/zuul-jobs master: Add a role to install and configure dstat to run in the background  https://review.openstack.org/51837419:20
openstackgerritMerged openstack-infra/project-config master: Small cleanups for zuul.d/projects.yaml  https://review.openstack.org/60977819:21
*** lujinluo has quit IRC19:27
*** eernst has joined #openstack-infra19:32
mwhahahaI'm looking to convert a legacy job to v3, what's the prefered way to copy the repo data into the workspace or consume the stuff that was checked out?  Or is it best to just keep using zuul-cloner19:34
clarkbmwhahaha: zuul does that automatically for you in v319:35
AJaegermwhahaha: get rid of zuul-cloner!19:35
mwhahahaclarkb: yea i know it checks it out, but does it drop in in /home/zuul/workspace/<repo> ?19:35
clarkbmwhahaha: /home/zuul/src/$hostname/path/to/repo (it is encoded in a zuul ansible var too)19:35
* mwhahaha is trying to understand the pathing19:35
*** eernst has quit IRC19:35
clarkbit might be workspace/src. Its a ansible variable /me looks it up19:35
mordredmwhahaha: it's done in the src dir of the zuul home dir on the remote node in golang format19:36
mordredmwhahaha: so - /home/zuul/src/git.openstack.org/openstack/nova home/zuul/src/git.openstack.org/openstack/swift etc19:36
clarkbmwhahaha: http://logs.openstack.org/07/599607/2/check/openstack-zuul-jobs-linters/a3ecca5/zuul-info/inventory.yaml I loke to look at an example inventory file. zuul.project.src_dir19:37
*** electrofelix has quit IRC19:37
AJaeger"{{ ansible_user_dir }}/{{ zuul.projects['git.openstack.org/openstack/tempest'].src_dir }}"19:37
mwhahahayea it's kinda annoying that you have to know the full url in the zuul.projects[] data19:37
*** dhill_ has quit IRC19:37
mwhahahabut thanks i'll try to use that instead19:37
*** rkukura has quit IRC19:38
clarkbmwhahaha: not sure I understand? its given to you in the src_dir var?19:38
mordredwell, it's also predictable, so you don't _have_ to use that zuul.projects[] data19:38
mordredclarkb: src_dir only works for a single repo19:38
clarkbmordred: no its in there for each projects repo too19:38
clarkbso you can loop over it19:38
mordredgood point19:38
mwhahahazuul.projects['git.openstack.org/openstack/python-tripleoclient'].src_dir19:38
AJaegerclarkb: or use the above as I pointed out above19:39
mwhahahathat's one hell of a variable :D19:39
mordredyah. I guess it just depends on what you're wanting to do19:39
mordredif you find yourself writing zuul.projects['git.openstack.org/openstack/python-tripleoclient'].src_dir in a playbook - you're writing too many characters :)19:39
mordredbut yes - it's totally there for the iterating19:39
AJaegermwhahaha: feel free to show a change and ask for review here...19:40
mordred++19:40
mwhahahai shall, just trying to figure out where the data is :)19:40
AJaegermordred: time for a quick cleanup change review, please? https://review.openstack.org/60995819:40
* AJaeger waves good bye19:41
*** rkukura has joined #openstack-infra19:41
*** dhill_ has joined #openstack-infra19:41
mordredAJaeger: done! have a good weekend19:42
AJaegermordred: thanks - and thanks ;)19:43
clarkbloop: zuul.projects | dict2items # this doesn't exist until ansible 2.6 :/ then item.value.src_dir is your path?19:44
clarkbin 2.5 which zuul runs I guess you use with_something19:45
*** apetrich has quit IRC19:48
*** apetrich has joined #openstack-infra19:49
smcginnisIf anyone other than Andreas has a moment to take a look at https://review.openstack.org/#/c/609827/ - I have some follow up work I want to do once I've been able to fully test that.19:49
openstackgerritMerged openstack-infra/project-config master: Cleanup devstack-gate jobs  https://review.openstack.org/60995819:51
openstackgerritMonty Taylor proposed openstack-infra/zuul master: Add line to setup.cfg limiting to python >=3.5  https://review.openstack.org/61008919:54
*** rkukura has quit IRC19:55
clarkbsmcginnis: done. Keep in mind its hard to undo that with pbr once you've take the plunge19:55
clarkb(so maybe test with a repo where specific version numbers don't matter too much)19:56
smcginnisclarkb: Thanks!19:56
smcginnisYeah, going to test with release-test before I attempt anything else there.19:56
smcginnisI suppose an attempt to revert the commit would just double the problem.19:56
smcginnisAssuming they don't remove the text from the commit.19:56
clarkbya I want to say when zuul ran into an improper/unwated pbr version tag that we had to do a hard reset?19:57
clarkbcorvus probably remembers19:57
smcginnisSeems like the only way to clear something like that out.19:57
smcginnisKinda scary a little inconspicuous text in a commit message has the potential to cause so much trouble. :)19:58
*** eernst has joined #openstack-infra20:02
corvusyes.  we took the extraordinary step of altering git history.20:10
corvussmcginnis, dhellmann: i can't recommend *against* using the Sem-Ver headers enough.  in short, i think it's a really bad idea to encode actionable content like that in commit messages.  the primary benefit of version control systems is that any change can be undone.  but *that* change *can not* be undone.  it's terrifying.20:12
smcginniscorvus: Any suggestions how we can increment version for those that want to be able to test upgrades to master after we cut stable branches?20:13
clarkbsmcginnis: pip install -U? I'm not sure why the version matters for that20:14
smcginniscorvus: The idea with this is when we create stable branches, we have the auto generated patches that update reno and the stable .gitreview, etc. This would just be included with that.20:14
smcginnisclarkb: I don't think distro packagers can do that.20:14
corvussmcginnis: nope, i'm sorry i haven't engaged with the problem and i have to run now.  i'll be happy to chat more next week, i just felt that since clarkb invoked my experience with the semver headers, i should share my 2-cents in case the lack of "undo" functionality had not occurred to you (it certainly hadn't to others, including me, until i saw the results)20:15
smcginniscorvus: No problem, thanks for that.20:15
clarkbsmcginnis: ya its probably harder for distros since their versions have proper meaning typically. I'm not sure how you'd tackle that in a distro context20:16
openstackgerritMerged openstack-infra/project-config master: update the branching script to include a semver bump  https://review.openstack.org/60982720:16
smcginnisclarkb: Doug detailed some of the challenges with this on the ML. Let me see if I can dig that up.20:16
*** rkukura has joined #openstack-infra20:16
smcginnishttp://lists.openstack.org/pipermail/openstack-dev/2018-October/135706.html20:17
*** ssbarnea_ has quit IRC20:17
smcginnisI'm certainly open to other suggestions, but even with the inherent risks, this seemed like the best solution to the issue.20:17
*** needscoffee is now known as kmalloc20:20
openstackgerritMerged openstack-infra/nodepool master: Add line to setup.cfg limiting to python >=3.5  https://review.openstack.org/61009020:21
openstackgerritMerged openstack-infra/zuul master: Quick-start: remove ansible Dockerfile  https://review.openstack.org/60955520:21
openstackgerritClark Boylan proposed openstack-infra/nodepool master: Run zuul-quick-start job  https://review.openstack.org/61015920:35
*** ianychoi has joined #openstack-infra20:35
*** rkukura has quit IRC20:37
*** lujinluo has joined #openstack-infra20:37
*** ansmith has quit IRC20:40
*** eharney has quit IRC20:40
*** rkukura has joined #openstack-infra20:41
*** bobh has quit IRC20:48
openstackgerritMerged openstack-infra/zuul master: Build new container images for quick-start  https://review.openstack.org/60976320:51
*** mriedem_away is now known as mriedem20:51
openstackgerritMerged openstack-infra/zuul master: Add line to setup.cfg limiting to python >=3.5  https://review.openstack.org/61008920:52
*** rkukura has quit IRC20:56
*** dave-mccowan has quit IRC20:57
*** rkukura has joined #openstack-infra21:00
*** trown is now known as trown|outtypewww21:00
openstackgerritPaul Belanger proposed openstack-infra/zuul-jobs master: Fix typo with ansible_galaxy_info.token  https://review.openstack.org/61016421:00
openstackgerritMerged openstack-infra/zuul master: Quick-start: switch to polygerrit  https://review.openstack.org/60982821:08
tonybdhellmann: Thanks.21:22
*** jamesdenton has quit IRC21:24
openstackgerritMerged openstack-infra/zuul master: Remove unneeded cruft from bdist_wheel fix attempts  https://review.openstack.org/61014321:27
openstackgerritMatt McEuen proposed openstack-infra/project-config master: New Airship project - Tempest Plugin  https://review.openstack.org/61017521:30
*** PapaOurs is now known as bauzas21:32
smcginnisgerritbot appears to be dead in #openstack-cinder21:39
*** rkukura has quit IRC21:39
clarkbsmcginnis: it connects to channels in a least recently used rotation because there are too many channels for it to connect to directly anymore21:40
clarkbsmcginnis: have there been missed events? or is it just gone? if no missed events and just gone it should join and emit the next event21:40
clarkband I think the sasl auth is in place now so it shouldn't have the auth race issues with freenode +r rules anymore21:40
smcginnisMissed events. Or at least severely delayed.21:40
smcginnisNot a big deal, just noticed there wasn't an announcement for a patch I pushed up.21:41
smcginnisI'll check again on Monday.21:41
*** lujinluo has quit IRC21:42
clarkbsmcginnis: 2018-10-12 21:38:32,626 INFO gerritbot: Sending "Sean McGinnis proposed openstack/cinder master: Allow setting volume readonly during creating  https://review.openstack.org/610177" to #openstack-cinder21:42
clarkbit thinks it is sending things to the channel21:42
smcginnisHuh, nothing's shown up in channel. Unless irssi is blocking it somehow.21:42
smcginnisAnyway, gotta run. Thanks.21:43
*** ansmith has joined #openstack-infra21:43
*** boden has quit IRC21:43
*** rkukura has joined #openstack-infra21:43
clarkbya eavesdrop doesn't see it either21:43
clarkbmy current guess is we didn't restart it after the sasl change went in because that happened after the config updates21:47
clarkbbut finding in the logs where it started is not easy21:47
*** agopi_ has joined #openstack-infra21:47
*** lujinluo has joined #openstack-infra21:49
*** agopi has quit IRC21:49
clarkbno ps says it started after the install should've happened21:50
clarkbfungi: ^ sasl may not have solved the problems. I wonder if there is something buggy about the channel joiing21:51
*** rkukura has quit IRC21:52
prometheanfireclarkb: is it ok if comments are lost from the resolved.conf file?21:53
clarkbprometheanfire: probably? usually the comments are "don't edit this by hand"21:54
*** rlandy has quit IRC21:54
clarkbgrepping gerritbot logs I see it joining the cinder channel multiple times but never parting it (could be due to restarts) it should be connected to the channel?21:54
*** jistr has quit IRC21:55
prometheanfireclarkb: ok, because configparser drops all comments which sucks, and configobj isn't a python builtin21:56
*** jistr has joined #openstack-infra21:56
*** tosky has quit IRC22:02
*** jistr has quit IRC22:05
*** agopi_ is now known as agopi22:05
*** mriedem is now known as mriedem_away22:05
*** jistr has joined #openstack-infra22:06
clarkbI'm pretty stumped on this channel issue since the join happens long after the connection to the server but it would've been auth'd via sasl anyway22:12
clarkbthe only thing I can figure is that freenode didn't get the message?22:12
*** agopi is now known as agopi|brb22:13
openstackgerritMatthew Thode proposed openstack-infra/glean master: write dns info to networkd  https://review.openstack.org/61010722:16
*** agopi|brb has quit IRC22:17
prometheanfireclarkb, mordred, odyssey4me: ^ reviewable22:18
prometheanfirehuh, no zuul dir for glean22:19
*** pcaruana has quit IRC22:20
openstackgerritMerged openstack-infra/zuul-jobs master: Fix typo with ansible_galaxy_info.token  https://review.openstack.org/61016422:26
*** eernst has quit IRC22:30
*** pbourke has quit IRC22:32
*** pbourke has joined #openstack-infra22:32
*** eernst has joined #openstack-infra22:33
*** eernst has quit IRC22:37
*** eernst has joined #openstack-infra22:37
*** eernst_ has joined #openstack-infra22:39
*** eernst has quit IRC22:39
*** eernst_ has quit IRC22:43
*** kjackal_v2 has quit IRC22:56
*** elbragstad has quit IRC22:56
*** sthussey has quit IRC23:03
*** elbragstad has joined #openstack-infra23:15
*** elbragstad has quit IRC23:15
*** owalsh_ has joined #openstack-infra23:18
*** kgiusti has left #openstack-infra23:18
*** owalsh has quit IRC23:21
*** agopi|brb has joined #openstack-infra23:25
openstackgerritClark Boylan proposed openstack-infra/project-config master: Disable packethost due to mirror outages  https://review.openstack.org/61018723:28
clarkbthe mirror has been shutdown again. I think we should go ahead and do ^23:28
*** slaweq has quit IRC23:32
*** agopi|brb has quit IRC23:41
mwhahahaspeaking of mirrors, do we have the cloud images (like the centos one) available via a mirror?23:51
*** gyee has quit IRC23:52
mwhahahai see the fedora one, but not centos23:52
clarkbmwhahaha: the fedora atomic images at http://mirror.dfw.rax.openstack.org/fedora/atomic/stable/ are the only ones I know of23:52
prometheanfirenot sure who added gentoo to the glean gate, but, it's not passing23:52
prometheanfirehttp://logs.openstack.org/07/610107/2/check/nodepool-functional-py35-gentoo-src/ccdd739/job-output.txt.gz#_2018-10-12_22_42_09_82995523:52
prometheanfirestuck on waitforimage23:53
clarkbprometheanfire: check the nodepool builder logs23:53
mwhahahaclarkb: yea the regular fedora one is there too http://mirror.bhs1.ovh.openstack.org/fedora/releases/28/Cloud/x86_64/images/23:53
prometheanfireclarkb: images built successfully23:54
prometheanfireif that's what you are talking about23:54
clarkbwaitforimage is waiting for the image to build and upload.23:55
clarkbhttp://logs.openstack.org/07/610107/2/check/nodepool-functional-py35-gentoo-src/ccdd739/controller/nodepool/nodepool-image-list.txt does not show the image being ready23:56
clarkbhttp://logs.openstack.org/07/610107/2/check/nodepool-functional-py35-gentoo-src/ccdd739/controller/logs/screen-nodepool-builder.txt.gz#_Oct_12_23_42_31_463377 that makes me think the build failed23:56
*** EvilienM is now known as EmilienM23:57
clarkbhttp://logs.openstack.org/07/610107/2/check/nodepool-functional-py35-gentoo-src/ccdd739/controller/logs/builds/gentoo-17-0-systemd-0000000003_log.txt.gz#_2018-10-12_23_42_24_086 is the error maybe?23:57
prometheanfireI think I have a review for that...23:58
prometheanfiresomewhere, I added sudo to packages.yaml23:58

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!