| *** hamzy has joined #oooq | 00:28 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001 @ (2 more messages) | 00:31 |
|---|---|---|
| *** dtrainor_ has joined #oooq | 01:11 | |
| *** dtrainor has quit IRC | 01:12 | |
| *** rascasoft has quit IRC | 01:16 | |
| *** rascasoft has joined #oooq | 01:17 | |
| *** dtantsur|afk has quit IRC | 01:24 | |
| *** chem has quit IRC | 01:56 | |
| *** chem has joined #oooq | 01:57 | |
| *** ykarel|away has joined #oooq | 02:06 | |
| *** dsneddon has quit IRC | 02:15 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039 @ https://review.openstack.org/602248, master: tripleo-ci-centos-7 (2 more messages) | 02:31 |
| *** gkadam has joined #oooq | 02:37 | |
| *** dsneddon has joined #oooq | 02:39 | |
| *** vinaykns has quit IRC | 02:46 | |
| *** dsneddon has quit IRC | 02:55 | |
| *** chem has quit IRC | 02:57 | |
| *** chem has joined #oooq | 02:58 | |
| *** dsneddon has joined #oooq | 03:18 | |
| *** dsneddon has quit IRC | 03:24 | |
| *** dsneddon has joined #oooq | 04:00 | |
| *** udesale has joined #oooq | 04:00 | |
| *** dsneddon has quit IRC | 04:05 | |
| *** ykarel|away has quit IRC | 04:26 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039 @ https://review.openstack.org/602248, master: tripleo-ci-centos-7 (2 more messages) | 04:31 |
| *** dsneddon has joined #oooq | 04:41 | |
| *** ykarel|away has joined #oooq | 04:42 | |
| *** dsneddon has quit IRC | 04:47 | |
| *** chem has quit IRC | 04:58 | |
| *** chem has joined #oooq | 04:58 | |
| *** dsneddon has joined #oooq | 05:18 | |
| *** ratailor has joined #oooq | 05:18 | |
| *** ykarel|away has quit IRC | 05:18 | |
| *** dsneddon has quit IRC | 05:24 | |
| *** ykarel|away has joined #oooq | 05:24 | |
| *** marios has joined #oooq | 05:31 | |
| *** marios_|ruck has joined #oooq | 05:32 | |
| *** quiquell|off is now known as quiquell | 05:42 | |
| *** dsneddon has joined #oooq | 05:45 | |
| *** jfrancoa has joined #oooq | 05:51 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039 @ https://review.openstack.org/602248, master: tripleo-ci-centos-7 (2 more messages) | 06:31 |
| *** holser_ has joined #oooq | 06:48 | |
| *** dsneddon has quit IRC | 06:50 | |
| *** holser_ has quit IRC | 06:52 | |
| *** holser_ has joined #oooq | 06:53 | |
| *** jpena|off is now known as jpena | 06:55 | |
| *** quiquell is now known as quiquell|brb | 06:56 | |
| *** kopecmartin|off is now known as kopecmartin | 06:57 | |
| *** chem has quit IRC | 07:00 | |
| *** chem has joined #oooq | 07:02 | |
| marios | sshnaidm|rover: preparing the press release... http://paste.openstack.org/show/748770/ agree? | 07:03 |
| marios | sshnaidm|rover: done :D | 07:08 |
| *** tosky has joined #oooq | 07:09 | |
| *** dsneddon has joined #oooq | 07:25 | |
| *** amoralej|off is now known as amoralej | 07:26 | |
| *** skramaja has joined #oooq | 07:32 | |
| *** yolanda has joined #oooq | 07:33 | |
| *** dsneddon has quit IRC | 07:41 | |
| sshnaidm|rover | marios, maybe we'll have fixed today.. | 07:49 |
| marios | sshnaidm|rover: yeah but for that call /press release ;) they dont really care if it isn't OSP15 | 07:49 |
| marios | so for that we are green | 07:49 |
| marios | just mentioned the queens stuff for completeness | 07:49 |
| marios | sshnaidm|rover: damien is working on it https://review.openstack.org/#/c/649287/3 | 07:50 |
| sshnaidm|rover | marios, yeah, keeping eye on it | 07:50 |
| sshnaidm|rover | marios, good work in total | 07:51 |
| marios_|ruck | https://www.timeanddate.com/countdown/generic?iso=20190403T18&p0=680&font=cursive&csz=1$ | 07:51 |
| marios_|ruck | sshnaidm|rover: ^ | 07:51 |
| zbr | marios: sshnaidm|rover hi! here is something strange: https://review.openstack.org/#/c/645166/ -- i see that tripleo-ci-centos-7-standalone did run only on gate and if if remember well, we had a rule saying that everyting that runs on gate must be on check too. right? | 07:52 |
| marios_|ruck | zbr: yeah should be so looking | 07:53 |
| *** quiquell|brb is now known as quiquell | 07:53 | |
| zbr | the reason for that being to avoid: failure to merge at gate. maybe in this case it was a transient case but i still do not see gate being a subset of check. | 07:53 |
| marios_|ruck | zbr: so must be a bug in the tripleo-quickstart zuul layout | 07:54 |
| marios_|ruck | zbr: will you check and post that? | 07:54 |
| marios_|ruck | zbr: or would you like us to | 07:54 |
| marios_|ruck | zbr: i mean i could if you wnat but in a bit doing reviews now | 07:54 |
| sshnaidm|rover | zbr, https://github.com/openstack/tripleo-quickstart/blob/master/zuul.d/layout.yaml#L18 | 07:54 |
| sshnaidm|rover | zbr, it's limited for specific files | 07:54 |
| marios_|ruck | sshnaidm|rover: yeah but not in gate | 07:55 |
| marios_|ruck | sshnaidm|rover: its only in check there | 07:55 |
| sshnaidm|rover | marios_|ruck, it's both in gate and in check, but in check it's limited for this file | 07:55 |
| sshnaidm|rover | marios_|ruck, maybe worth to limit it in gate too | 07:55 |
| marios_|ruck | sshnaidm|rover: but it didn't run in gate | 07:55 |
| sshnaidm|rover | marios_|ruck, it did | 07:56 |
| zbr | marios_|ruck: so, the definition in check overrides the job from template so the gate endsup running it due to template but the check does not due to override.... well... i guess soon we will need zuul certifications ;) | 07:57 |
| marios_|ruck | tripleo-ci-centos-7-standaloneFAILURE in 1h 32m 24s | 07:57 |
| marios_|ruck | zbr: it did run sshnaidm|rover is right ^ | 07:57 |
| sshnaidm|rover | zbr, you can override it in same way in gate | 07:57 |
| zbr | sshnaidm|rover: yeah. i know. not sure which approach is better. | 07:58 |
| sshnaidm|rover | zbr, the other way is to remove this limit, but then it will run on every job | 07:59 |
| sshnaidm|rover | not sure if we want this, but I'm fine with it too | 08:00 |
| zbr | sshnaidm|rover: yep. not a big deal here but I can see this issue recurring again and again, where override in one gate/check does get out of sync. i wonder if we can find a way to prevent it (happened several times already). | 08:02 |
| zbr | maybe if we use yaml anchors? sadly they do no not work with lists, only with dictionaries. | 08:03 |
| sshnaidm|rover | zbr, I don't think it's a biggie, the other direction is a problem | 08:03 |
| sshnaidm|rover | zbr, if we have more jobs in gate that's fine, it can catch something we missed in checks | 08:04 |
| sshnaidm|rover | zbr, and without running a lot of jobs in check every time, because gate jobs run once | 08:04 |
| zbr | debatable but lets not pursuit any change regarding this yet, we have more important things to do | 08:05 |
| sshnaidm|rover | quiquell, what is difference between libvirt images in ci and not in ci? Seems like libvirt image in ci doesn't have lvm2 installed, no idea why.. | 08:07 |
| quiquell | sshnaidm|rover: they get downloaded from same place | 08:08 |
| *** dsneddon has joined #oooq | 08:08 | |
| quiquell | sshnaidm|rover: they use both fetch-images role | 08:08 |
| quiquell | no difference | 08:08 |
| sshnaidm|rover | quiquell, yeah, weird | 08:08 |
| quiquell | but lvm2 has to be in the jost not in the image itself | 08:09 |
| quiquell | s/jost/host/ | 08:09 |
| quiquell | at CI host is nodepool imag e | 08:09 |
| zbr | chandankumar: do you know what could have caused this tempest test to fail? http://logs.openstack.org/66/645166/4/gate/tripleo-ci-centos-7-standalone/bb49871/logs/tempest.html.gz | 08:09 |
| quiquell | non CI is whenever you are running it | 08:09 |
| quiquell | is that it ? | 08:09 |
| sshnaidm|rover | quiquell, it fails in standalone deploy when missing lvm2 | 08:09 |
| sshnaidm|rover | quiquell, while trying to run vgdisplay etc | 08:10 |
| quiquell | pass me the link | 08:10 |
| sshnaidm|rover | quiquell, http://logs.rdoproject.org/59/19859/21/check/tripleo-ci-reproducer-fedora-28-libvirt-standalone/885275c/tripleo-ci-reproducer/logs/01/1001/1/check/tripleo-ci-centos-7-standalone-dlrn-hash-tag/458ee46/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz | 08:11 |
| quiquell | sshnaidm|rover, marios_|ruck: Can I stop the promoter ? | 08:11 |
| sshnaidm|rover | quiquell, it's standalone logs from ci reproducer logs | 08:11 |
| quiquell | want to check something from stein | 08:11 |
| marios_|ruck | quiquell: i am ok with that sshnaidm|rover ? | 08:11 |
| sshnaidm|rover | quiquell, sure | 08:11 |
| marios_|ruck | we had promotion today | 08:11 |
| quiquell | ok thanks | 08:12 |
| *** dsneddon has quit IRC | 08:13 | |
| quiquell | sshnaidm|rover: https://cloud.centos.org/centos/7/images/CentOS-7-x86_64-GenericCloud-1802.qcow2 | 08:25 |
| quiquell | sshnaidm|rover: this is the image is using | 08:25 |
| *** ccamacho has joined #oooq | 08:25 | |
| quiquell | sshnaidm|rover: lvm2 shoudl be installed in the libvirt role | 08:25 |
| arxcruz | kopecmartin: ping | 08:27 |
| kopecmartin | arxcruz, yes? | 08:27 |
| arxcruz | kopecmartin: https://launchpadlibrarian.net/417456583/tempest.log.txt | 08:27 |
| arxcruz | check this | 08:27 |
| *** ykarel|away is now known as ykarel|lunch | 08:30 | |
| marios_|ruck | zbr: o/ | 08:30 |
| marios_|ruck | join? | 08:30 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039 @ https://review.openstack.org/602248, master: tripleo-ci-centos-7 (2 more messages) | 08:31 |
| kopecmartin | arxcruz, that's weird, but i see a lot of retries and Broken pipe, it might be related to the TypeError | 08:34 |
| *** ykarel|lunch has quit IRC | 08:35 | |
| sshnaidm|rover | quiquell, I don't think we install lvm2 explicitly | 08:38 |
| quiquell | sshnaidm|rover: nope we don't | 08:39 |
| quiquell | humm could be that we transfer the .repo files from host to image | 08:39 |
| quiquell | and they are different at rdocloud ? | 08:39 |
| quiquell | sshnaidm|rover: are you sure running at non CI is exactly with the same tripleo version ? | 08:40 |
| quiquell | sshnaidm|rover: same DLRN hash and all maybe there is new stuff there | 08:40 |
| *** kopecmartin is now known as kopecmartin|lunc | 08:40 | |
| sshnaidm|rover | quiquell, but lvm installation shouldn't depend on repo | 08:40 |
| sshnaidm|rover | quiquell, it's just not there | 08:40 |
| quiquell | sshnaidm|rover: it has to be installed at non-ci since the image is exactly the same | 08:41 |
| quiquell | using vgdisplay is new stuff ? | 08:41 |
| sshnaidm|rover | quiquell, I don't think it's new stuff, and it works fine in usual reproducer | 08:43 |
| sshnaidm|rover | quiquell, are you sure we use centos cloud image? | 08:43 |
| *** dsneddon has joined #oooq | 08:43 | |
| quiquell | 4 months | 08:44 |
| quiquell | sshnaidm|rover: I see fetch-images using https://cloud.centos.org/centos/7/images/CentOS-7-x86_64-GenericCloud-1802.qcow2 | 08:44 |
| quiquell | http://logs.rdoproject.org/59/19859/21/check/tripleo-ci-reproducer-fedora-28-libvirt-standalone/885275c/job-output.txt.gz | 08:44 |
| quiquell | sshnaidm|rover: do we update those images ? | 08:45 |
| quiquell | sshnaidm|rover: maybe that's step is missing at CI (Don't know why) | 08:46 |
| sshnaidm|rover | quiquell, yeah, could be.. | 08:46 |
| quiquell | sshnaidm|rover: do you see images get updated at non CI ? | 08:46 |
| *** derekh has joined #oooq | 08:46 | |
| quiquell | TASK [Update packages] | 08:48 |
| quiquell | Yep at the end | 08:48 |
| chandankumar | zbr: looks like ssh timed out I am not sure 2019-04-03 03:15:46,307 95 INFO [tempest.scenario.manager] FloatingIP: {u'router_id': u'5ff04eeb-6ad3-404f-8f2b-07fafc53363d', u'status': u'DOWN', u'description': u'', u'tags': [], u'tenant_id': u'c3460cc945234945acd827eb4333ae8a', u'created_at': u'2019-04-03T03:15:40Z', u'updated_at': u'2019-04-03T03:15:40Z', u'floating_network_id': | 08:48 |
| chandankumar | u'cf57a9b9-0154-469f-87f4-c1ea65827cc5', u'fixed_ip_address': u'10.100.0.12', u'floating_ip_address': u'192.168.24.109', u'revision_number': 0, u'project_id': u'c3460cc945234945acd827eb4333ae8a', u'port_id': u'8a4e58b1-9ac7-44df-b3bb-881d2bec2181', u'id': u'ccf5104f-255f-4e6b-9be8-4458443ec95c'} is at status: ACTIVE | 08:48 |
| quiquell | after subnodes get started | 08:49 |
| chandankumar | zbr: is it failing multiple times? | 08:50 |
| quiquell | sshnaidm|rover: I don't see lvm2 here http://logs.rdoproject.org/59/19859/21/check/tripleo-ci-reproducer-fedora-28-libvirt-standalone/885275c/tripleo-ci-reproducer/logs/01/1001/1/check/tripleo-ci-centos-7-standalone-dlrn-hash-tag/458ee46/logs/undercloud/var/log/extra/rpm-list.txt.gz | 08:52 |
| *** dsneddon has quit IRC | 08:52 | |
| quiquell | sshnaidm|rover: do you see it at your local run ? | 08:52 |
| quiquell | sshnaidm|rover: we can add it to libvirt role though | 08:52 |
| sshnaidm|rover | quiquell, I'll run now locally, let's see | 08:53 |
| sshnaidm|rover | quiquell, just curious where is the diff.. | 08:53 |
| zbr | chandankumar: i did a recheck but because this runs only at gate it will take a lot of time to get confirmed. that is the second time this change fails at the gate, but first time it was for a different reason. I suspect it was ~random. | 08:53 |
| quiquell | sshnaidm|rover: well run it here too | 08:53 |
| quiquell | wait | 08:54 |
| quiquell | nah I am going to test it too | 08:56 |
| chandankumar | zbr: sshnaidm|rover https://review.openstack.org/#/c/648678/ I have updated the commit message it will be easier to understand now | 08:58 |
| quiquell | sshnaidm|rover: there is two ansible hosts "undercloude" and "standalone" | 08:59 |
| quiquell | sshnaidm|rover: this error is at "standalone" ansible host | 08:59 |
| *** ykarel|lunch has joined #oooq | 09:00 | |
| *** chem has quit IRC | 09:00 | |
| sshnaidm|rover | quiquell, not sure I understand | 09:01 |
| quiquell | sshnaidm|rover: http://logs.rdoproject.org/59/19859/21/check/tripleo-ci-reproducer-fedora-28-libvirt-standalone/885275c/tripleo-ci-reproducer/logs/01/1001/1/check/tripleo-ci-centos-7-standalone-dlrn-hash-tag/458ee46/logs/undercloud/home/zuul/undercloud-ansible-AYeNbs/inventory.yaml.txt.gz | 09:01 |
| sshnaidm|rover | chandankumar, I still don't understand why not to put all there "allowed" in one var and in defaults? | 09:01 |
| chandankumar | sshnaidm|rover: in defaults we cannot keep it as these values are widely used in OSA | 09:02 |
| sshnaidm|rover | quiquell, and? is it a problem? | 09:02 |
| sshnaidm|rover | quiquell, it's ooo internal ansible thing | 09:02 |
| sshnaidm|rover | chandankumar, can we import them from OSA? or what is value to have them in playbook..? | 09:03 |
| quiquell | sshnaidm|rover: they point to the same machine I suppose | 09:03 |
| *** chem has joined #oooq | 09:04 | |
| chandankumar | sshnaidm|rover: we cannot import them from OSA, we need to set the value like https://review.openstack.org/#/c/648678/10/playbooks/tempest.yml@31 | 09:04 |
| chandankumar | sshnaidm|rover: tempest_service_available_{{ item }}: true | 09:04 |
| chandankumar | sshnaidm|rover: there are the list of vars needs https://github.com/openstack/openstack-ansible-os_tempest/blob/master/defaults/main.yml#L159..#L188 | 09:05 |
| sshnaidm|rover | chandankumar, yeah, that I understand, I'm talking about list of "allowed_services" only | 09:05 |
| chandankumar | sshnaidm|rover: sorry you mean just to put allowed_services var and remove the service list | 09:06 |
| sshnaidm|rover | quiquell, the same is in every other standalone job, i.e. http://logs.openstack.org/78/648678/9/check/tripleo-ci-centos-7-standalone/187ab7f/logs/undercloud/home/zuul/undercloud-ansible-8SdZpF/inventory.yaml.txt.gz | 09:06 |
| chandankumar | and if allowd_services is called it's value is set to true | 09:06 |
| quiquell | sshnaidm|rover: lvm2 is there http://logs.openstack.org/78/648678/9/check/tripleo-ci-centos-7-standalone/187ab7f/logs/undercloud/var/log/extra/rpm-list.txt.gz | 09:07 |
| chandankumar | by putting list I wanted to restrict the user on not to install random services plugins which is not needed | 09:07 |
| quiquell | weird... | 09:07 |
| sshnaidm|rover | chandankumar, something like that http://paste.openstack.org/show/748775/ | 09:08 |
| sshnaidm|rover | quiquell, yep.. | 09:08 |
| sshnaidm|rover | quiquell, and it's on the image when running locally | 09:09 |
| sshnaidm|rover | quiquell, just updated, but it was there before | 09:09 |
| sshnaidm|rover | quiquell, oh, no.. I did it with openstack, not libvirt, ignore it | 09:10 |
| quiquell | jaja | 09:10 |
| quiquell | sshnaidm|rover: but this was working 4 months ago | 09:10 |
| quiquell | have we run libvirt with standalone during this 4 months ? | 09:11 |
| chandankumar | sshnaidm|rover: regarding defaults/main.yaml change, I am not sure OSA will like it , let me propose a patch there | 09:11 |
| sshnaidm|rover | quiquell, we didn't run standalone in ci ever | 09:11 |
| zbr | chandankumar: re your patch, i see that tripleo-ci-centos-7-standalone-os-tempest failed twice in a row. | 09:11 |
| sshnaidm|rover | quiquell, I think I ran it and it worked recently, but need to recheck.. | 09:11 |
| quiquell | humm is this libvirt suppose to work with fedora28 host ? | 09:12 |
| chandankumar | zbr: it is due to tempest_services var undefined | 09:12 |
| quiquell | I am possitive it was working :-/ | 09:12 |
| chandankumar | I have fixed in new patchset | 09:12 |
| sshnaidm|rover | quiquell, idk, never tried it | 09:12 |
| quiquell | well Ci did | 09:14 |
| sshnaidm|rover | quiquell, anyway, dry run succeeded http://logs.rdoproject.org/59/19859/22/check/tripleo-ci-reproducer-fedora-28-libvirt-standalone/445de28/tripleo-ci-reproducer/logs/01/1001/1/check/tripleo-ci-centos-7-standalone-dlrn-hash-tag/75f7903/job-output.txt.gz | 09:14 |
| sshnaidm|rover | quiquell, I think it's a good start | 09:14 |
| quiquell | humm | 09:15 |
| quiquell | I get python2-libvirt | 09:15 |
| quiquell | libvirt-python no existe paquete coincidente | 09:15 |
| quiquell | at my laptop | 09:15 |
| quiquell | with fedora28 | 09:15 |
| quiquell | sshnaidm|rover: do we want to do more than dryrun for CI ? | 09:15 |
| sshnaidm|rover | quiquell, maybe need el-libvirto-pythono | 09:16 |
| sshnaidm|rover | quiquell, kidding | 09:16 |
| sshnaidm|rover | quiquell, well, I'd like to run the whole job in best case | 09:16 |
| quiquell | wtf | 09:17 |
| quiquell | fedora from zuul nodepool have libvirt-python | 09:17 |
| quiquell | mayeb is related to repos and all ? | 09:18 |
| *** dtantsur has joined #oooq | 09:18 | |
| quiquell | I am going to update my fedora | 09:18 |
| quiquell | wtf docker fedora:28 can install it too :-/ | 09:20 |
| quiquell | my fedora is so broken | 09:20 |
| quiquell | ok messed up repos | 09:21 |
| quiquell | now is fixed | 09:21 |
| quiquell | sshnaidm|rover: I think standalone have to make sure lvm2 is installed | 09:25 |
| quiquell | since it depends on it | 09:25 |
| quiquell | is that ok ? | 09:25 |
| *** dsneddon has joined #oooq | 09:25 | |
| sshnaidm|rover | quiquell, well, that's right, but still need to figure out why it works locally | 09:26 |
| quiquell | sshnaidm|rover: I think it's not going to work | 09:26 |
| quiquell | sshnaidm|rover: mayeb we haven't run standalone at libvirt since 4 months ago | 09:26 |
| sshnaidm|rover | why? | 09:26 |
| sshnaidm|rover | quiquell, well, this I'll try to check.. but my hosts are shutdown in lab :( | 09:27 |
| quiquell | sshnaidm|rover: I am starting it at my laptop | 09:27 |
| quiquell | sshnaidm|rover: let's see | 09:27 |
| sshnaidm|rover | quiquell, you're brave man | 09:27 |
| zbr | sshnaidm|rover: does https://review.openstack.org/#/c/648110/4 look ok to wf? | 09:28 |
| quiquell | marios_|ruck: want to release new version of tripleo-common | 09:28 |
| quiquell | marios_|ruck: I need to tag the project first | 09:28 |
| quiquell | marios_|ruck: Do you know what's the standar procedure to tag projects at openstack ? | 09:29 |
| quiquell | sshnaidm|rover: ^ Â? | 09:29 |
| marios_|ruck | quiquell: yeah havent done it in a loong time lemme find some notes/pointers | 09:29 |
| quiquell | marios_|ruck: cool thanks | 09:29 |
| marios_|ruck | quiquell: https://wiki.openstack.org/wiki/TripleO/ReleaseManagement | 09:30 |
| marios_|ruck | quiquell: did you see that already? | 09:30 |
| marios_|ruck | quiquell: semver | 09:30 |
| sshnaidm|rover | zbr, arxcruz if we have one featureset for all standalone jobs, why not to put it there? | 09:30 |
| quiquell | marios_|ruck: first time, thanks marios | 09:30 |
| sshnaidm|rover | quiquell, no, I think PTL should know | 09:30 |
| marios_|ruck | quiquell: ack should be all there | 09:30 |
| quiquell | marios_|ruck: do we have permission to do so ? | 09:31 |
| quiquell | marios_|ruck: or only PTL can ? | 09:31 |
| marios_|ruck | quiquell: well in the past i've done it but why not ask the ptl | 09:31 |
| marios_|ruck | at least make them aware | 09:31 |
| marios_|ruck | but it was long time ago so | 09:31 |
| marios_|ruck | just ping jaosorior | 09:31 |
| quiquell | jaosorior: can we delivery tripleo-common ? | 09:31 |
| sshnaidm|rover | zbr, arxcruz oh, it's only for tripleo-ci-centos-7-standalone-os-tempest, that's fine | 09:32 |
| *** dsneddon has quit IRC | 09:32 | |
| sshnaidm|rover | arxcruz, chandankumar with all these featureset overrides, don't you want just to create a separate featureset file? | 09:33 |
| quiquell | holy sh we didn't have release tripleo-common from 15 Feb ? | 09:35 |
| jaosorior | quiquell: anybody can propose a delivery in the releases repo | 09:47 |
| quiquell | jaosorior: So I do what's it's here ? https://wiki.openstack.org/wiki/TripleO/ReleaseManagement | 09:49 |
| quiquell | jaosorior: also looks like last release is from FEB could it be ? | 09:49 |
| jaosorior | quiquell: oh, before cutting a release, I would rather wait for this to merge https://review.openstack.org/#/c/638323/ It's blocking some folks | 09:50 |
| quiquell | jaosorior: looks like there is a lot of stuff from Feb... | 09:51 |
| arxcruz | sshnaidm|rover: no problem for me tbh | 09:52 |
| chandankumar | sshnaidm|rover: I think I can directly use tempest_plugin_distro_packages to set install tempest plugin packages | 09:56 |
| chandankumar | https://github.com/openstack/openstack-ansible-os_tempest/blob/master/vars/distro_install.yml#L28 | 09:56 |
| quiquell | jaosorior: we need to workflow this https://review.openstack.org/#/c/649460/ to have the other | 09:57 |
| *** chem has quit IRC | 10:00 | |
| *** dsneddon has joined #oooq | 10:06 | |
| quiquell | sshnaidm|rover: my laptop has die :-) | 10:08 |
| sshnaidm|rover | quiquell, seems like doesn't prevent you to send patches :) | 10:09 |
| sshnaidm|rover | quiquell, finally: https://review.rdoproject.org/r/#/c/19859/ | 10:11 |
| *** dalvarez has quit IRC | 10:12 | |
| *** dsneddon has quit IRC | 10:12 | |
| quiquell | sshnaidm|rover: I would replace the other jobs with standalone | 10:13 |
| quiquell | sshnaidm|rover: we just run standalone dryrun | 10:13 |
| quiquell | instead of hello-reproducer | 10:13 |
| sshnaidm|rover | quiquell, yeah, but let's try it first | 10:14 |
| arxcruz | chandankumar: please, add taiga on all the commits | 10:14 |
| quiquell | ack | 10:14 |
| *** radez has quit IRC | 10:15 | |
| quiquell | sshnaidm|rover: also I am adding branches to test1 https://review.rdoproject.org/r/#/c/19915/ | 10:15 |
| *** amoralej has quit IRC | 10:16 | |
| *** jpena has quit IRC | 10:16 | |
| chandankumar | arxcruz: yes, sure, https://review.rdoproject.org/r/#/c/18749/ waiting for result of the above patches | 10:16 |
| arxcruz | ok | 10:17 |
| sshnaidm|rover | quiquell, cool! | 10:18 |
| *** ykarel|lunch is now known as ykarel | 10:20 | |
| *** sshnaidm|rover is now known as sshnaidm|bbl | 10:27 | |
| *** jpena has joined #oooq | 10:28 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039, tripleo-ci-centos-7-ovb-1ctlr_1comp_1ceph-featureset024 @ (2 more messages) | 10:31 |
| chandankumar | sshnaidm|bbl: you mean a single fs only for os_tempest but still we need to override the values based on scenario jobs na/ | 10:37 |
| chandankumar | / | 10:37 |
| chandankumar | ? | 10:37 |
| *** dsneddon has joined #oooq | 10:40 | |
| *** holser_ is now known as holser|lunch | 10:40 | |
| quiquell | marios_|ruck, sshnaidm|bbl: branches working now at reproducer https://review.rdoproject.org/r/#/c/19915/ | 10:41 |
| marios_|ruck | quiquell: nice | 10:42 |
| quiquell | marios_|ruck: added documentation | 10:43 |
| quiquell | marios_|ruck: let's merge this so we can adapt reproducer script | 10:43 |
| quiquell | marios_|ruck: the story https://tree.taiga.io/project/tripleo-ci-board/task/933?kanban-status=1447275 | 10:44 |
| *** dsneddon has quit IRC | 10:46 | |
| *** quiquell is now known as quiquell|lunch | 10:48 | |
| marios_|ruck | quiquell|lunch: ack | 10:48 |
| *** bogdando has joined #oooq | 10:58 | |
| *** jpena is now known as jpena|lunch | 11:02 | |
| *** dsneddon has joined #oooq | 11:13 | |
| *** kopecmartin|lunc is now known as kopecmartin | 11:15 | |
| *** dsneddon has quit IRC | 11:22 | |
| *** quiquell|lunch is now known as quiquell | 11:24 | |
| *** udesale has quit IRC | 11:26 | |
| *** ratailor has quit IRC | 11:36 | |
| *** apetrich has quit IRC | 11:48 | |
| *** apetrich has joined #oooq | 11:49 | |
| *** dsneddon has joined #oooq | 11:51 | |
| weshay | marios_|ruck thanks for the update in the doc.. I removed queens status | 11:51 |
| marios_|ruck | weshay: ack | 11:52 |
| marios_|ruck | weshay: i did hilight the osp13 bit but ok np | 11:52 |
| marios_|ruck | weshay: i still see it | 11:52 |
| weshay | if you speak to it.. they'll call you out | 11:52 |
| marios_|ruck | k | 11:52 |
| weshay | for wasting time :) | 11:52 |
| marios_|ruck | weshay: honestly, it was a strategy to avoid folks dwelling on the fact that we had promotions last 3 days running | 11:52 |
| marios_|ruck | weshay: "look queens baad" | 11:53 |
| marios_|ruck | etc | 11:53 |
| marios_|ruck | oh well | 11:53 |
| weshay | chandankumar kopecmartin https://url.corp.redhat.com/triage-not-started | 11:53 |
| weshay | we have 9 untriaged tempest bugs | 11:53 |
| ykarel | weshay, are the directories on images.rdoproject.org created manually? | 11:54 |
| ykarel | currently centos7/stein/rdo_trunk is missing | 11:54 |
| weshay | ykarel no it's not https://images.rdoproject.org/stein/ | 11:55 |
| ykarel | weshay, https://images.rdoproject.org/centos7/ | 11:55 |
| *** rlandy has joined #oooq | 11:55 | |
| ykarel | stein ^^ missing, so fs002-upload failing | 11:55 |
| weshay | ah /me looks | 11:55 |
| weshay | ykarel any chance we can get the index visiable on that server? | 11:56 |
| ykarel | weshay, /me no idea may be jpena|lunch knows | 11:56 |
| weshay | ykarel that's really odd.. I tested a stein deployment locally now twice | 11:57 |
| weshay | I need to check what the uploader is doing | 11:57 |
| *** dsneddon has quit IRC | 11:57 | |
| ykarel | weshay, because images links still points to images.rdoproject.org/stein | 11:57 |
| ykarel | in release files ^^ | 11:57 |
| ykarel | those needs to be adjusted once we have proper links in place | 11:57 |
| weshay | chandankumar can I speak to you for a minute? | 11:58 |
| chandankumar | weshay: sure | 11:58 |
| weshay | https://bluejeans.com/u/whayutin/ | 11:58 |
| *** holser|lunch is now known as holser_ | 11:59 | |
| chandankumar | weshay: https://review.openstack.org/#/c/647696/ | 12:01 |
| chandankumar | weshay: https://review.openstack.org/#/c/648121/ | 12:02 |
| arxcruz | sshnaidm|bbl: https://review.openstack.org/#/c/648111/ can you +w? | 12:02 |
| jpena|lunch | weshay: the base /stein repo is created automatically by the uploader job, iirc. I have manually created centos7/stein as a symlink, which was not created | 12:02 |
| *** jpena|lunch is now known as jpena | 12:02 | |
| weshay | jpena k thanks.. /stein was not automatic fyi but you just did exactly what we needed.. so thank you :) | 12:03 |
| ykarel | jpena, Thanks | 12:03 |
| weshay | the jobs that consume do not use the distro path | 12:04 |
| ykarel | so possibly we will have promotion in next run | 12:04 |
| weshay | ah perfect | 12:04 |
| weshay | I noticed the stein containers were removed from rdo registry /me checks at least for f28 | 12:04 |
| quiquell | weshay: we checked esterday with skope and they were there | 12:05 |
| *** apetrich has quit IRC | 12:05 | |
| weshay | ok. | 12:05 |
| weshay | thanks quiquell | 12:05 |
| weshay | ah.. I see them now again | 12:05 |
| weshay | ok.. we're all good.. thanks jpena! | 12:05 |
| *** trown|outtypewww is now known as trown | 12:07 | |
| *** dsneddon has joined #oooq | 12:07 | |
| *** dsneddon has quit IRC | 12:13 | |
| *** quiquell is now known as quiquell|brb | 12:13 | |
| *** aakarsh has quit IRC | 12:14 | |
| weshay | marios_|ruck we are not red | 12:18 |
| weshay | marios_|ruck we go by the board | 12:18 |
| weshay | if a dfg is blocking a promotion | 12:18 |
| marios_|ruck | weshay: ack ;) i saw you deleting | 12:20 |
| marios_|ruck | weshay: glad we had that distinction/chat even though it derailed a bit | 12:21 |
| weshay | marios_|ruck so we're not blocking builds getting to QE | 12:21 |
| weshay | or to tripleo | 12:21 |
| weshay | marios_|ruck jaun does a less than ideal job of explaining the WHY here... | 12:21 |
| weshay | so I'm going to write something up w/ him | 12:21 |
| weshay | and post it in the doc | 12:22 |
| weshay | program call doc | 12:22 |
| weshay | marios_|ruck we stick to what http://rhos-release.virt.bos.redhat.com:3030/rhosp says | 12:22 |
| *** apetrich has joined #oooq | 12:23 | |
| weshay | marios_|ruck what Jan is talking about is for the non prod chain dfg's | 12:23 |
| marios_|ruck | weshay: ack | 12:25 |
| *** jfrancoa has quit IRC | 12:29 | |
| weshay | chandankumar kopecmartin so the only reason I see why we're showing up on "triage_not_started" list is priority/severity is not set https://mojo.redhat.com/docs/DOC-1145054 | 12:30 |
| weshay | so it's just a nit.. I see you guys working it :) | 12:30 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-containerized-undercloud-upgrades, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci- (3 more messages) | 12:31 |
| kopecmartin | weshay, also I didn't put some bugs from NEW->ASSIGNED because i don't have time to work on them at the moment .. so my question here is, is it better triage and don't work on them or wait with triaging until i really have time to look at it closer? | 12:32 |
| weshay | ya... according to the process it's better to triage and not start work | 12:33 |
| kopecmartin | ok, good to know | 12:34 |
| weshay | kopecmartin chandankumar FYI.. pavel is hitting http://pastebin.test.redhat.com/748951 no bug yet .. just FYI there may be incoming on osp-15 | 12:34 |
| weshay | kopecmartin++ | 12:34 |
| hubbot1 | weshay: kopecmartin's karma is now 7 | 12:34 |
| *** dtantsur is now known as dtantsur|brb | 12:36 | |
| *** dtantsur|brb is now known as dtantsur | 12:36 | |
| chandankumar | weshay: bookmarking the mojo page | 12:38 |
| *** panda|almost has joined #oooq | 12:39 | |
| *** jfrancoa has joined #oooq | 12:41 | |
| weshay | chandankumar I have all the info in https://docs.google.com/document/d/1fsd1bJ7r5CS9zaizu9I_92Uxo6z6Vtq8CWdJRsDC_Z4/edit# | 12:41 |
| *** jfrancoa has quit IRC | 12:42 | |
| *** aakarsh has joined #oooq | 12:48 | |
| *** dsneddon has joined #oooq | 12:50 | |
| chandankumar | sshnaidm|bbl: when a periodic job in check fails http://logs.rdoproject.org/49/18749/16/check/periodic-tripleo-ci-centos-7-scenario001-standalone-master/1540006/ it is not generating ara report | 12:51 |
| *** quiquell|brb is now known as quiquell | 12:54 | |
| quiquell | 908525 | 12:55 |
| quiquell | ccccccjgkgvcebjhrrvkdievvvjjlbhhucejntjurrdc | 12:55 |
| weshay | kopecmartin chandankumar pavel is waiving the tempestconf bug on osp15.. it's a keystone issue | 12:56 |
| * weshay hacks quiquell | 12:56 | |
| *** dsneddon has quit IRC | 12:56 | |
| kopecmartin | weshay, yeah, he told me, he sits next to me :D | 12:56 |
| quiquell | ups | 12:57 |
| quiquell | It's a honeykey :-) | 12:57 |
| chandankumar | weshay: thanks :-) | 12:57 |
| chandankumar | kopecmartin: :-) | 12:57 |
| weshay | kopecmartin ah good to know.. high five him for me :) | 12:58 |
| marios_|ruck | sshnaidm|bbl: https://www.timeanddate.com/countdown/generic?iso=20190403T18&p0=680&font=cursive&csz=1$ | 12:58 |
| *** dalvarez has joined #oooq | 12:58 | |
| rfolco | marios, quiquell, sshnaidm, weshay, panda, rlandy, arxcruz, mwhahaha, rfolco, chkumar, ssbarnea, kopecmartin | 12:59 |
| rfolco | retro mtg starts in one minute | 12:59 |
| *** jfrancoa has joined #oooq | 13:02 | |
| rfolco | sshnaidm|bbl, ping retro mtg | 13:02 |
| *** sshnaidm|bbl is now known as sshnaidm | 13:03 | |
| *** panda|almost is now known as panda | 13:04 | |
| sshnaidm | rfolco, can you provide a link to retro board please? | 13:06 |
| rfolco | https://trello.com/b/0VFswmht/rdo-infra-retrospective?menu=filter&filter=label:UniSprint7 | 13:06 |
| rfolco | sshnaidm, ^ | 13:06 |
| rfolco | https://etherpad.openstack.org/p/tripleo-ci-squad-meeting | 13:06 |
| sshnaidm | rfolco, thanks | 13:06 |
| chandankumar | sshnaidm: I think this will take care of tempest_service stuff nothing needed on tempest playbook | 13:10 |
| chandankumar | sshnaidm: https://review.openstack.org/#/c/649577/ | 13:10 |
| *** dsneddon has joined #oooq | 13:28 | |
| *** mjturek has joined #oooq | 13:31 | |
| *** vinaykns has joined #oooq | 13:33 | |
| *** dsneddon has quit IRC | 13:34 | |
| panda | quiquell: so the unsigned unfinished SMS I received was yours ? :) thanks. | 13:34 |
| *** radez has joined #oooq | 13:37 | |
| weshay | arxcruz https://dev.launchpad.net/LaunchpadBugTags | 13:38 |
| weshay | so we just need propose a new tag | 13:38 |
| weshay | arxcruz or we can pick one | 13:39 |
| arxcruz | weshay: i think this is launchpad side, not tripleo side | 13:39 |
| *** Vorrtex has joined #oooq | 13:40 | |
| *** ykarel has quit IRC | 13:41 | |
| weshay | arxcruz ah.. https://specs.openstack.org/openstack/tripleo-specs/specs/policy/bug-tagging.html | 13:44 |
| *** amoralej has joined #oooq | 13:44 | |
| arxcruz | :) | 13:45 |
| weshay | arxcruz zbr let's put up a spec change and add tempest | 13:45 |
| *** yolanda has quit IRC | 13:55 | |
| *** skramaja has quit IRC | 13:56 | |
| *** udesale has joined #oooq | 13:56 | |
| jfrancoa | quiquell: hey, about the error we saw in the standalone-upgrade job, do you know if there is any lp bug opened for it? | 14:04 |
| jfrancoa | quiquell: I'm seeing some shocking error in the logs of that job http://logs.openstack.org/90/625890/20/check/tripleo-ci-centos-7-standalone-upgrade/ac3600c/logs/undercloud/var/log/journal.txt.gz#_Apr_02_16_47_32 What does Internet Explorer have to do with Docker?? | 14:05 |
| *** dsneddon has joined #oooq | 14:05 | |
| quiquell | jfrancoa: looks like a bad joke | 14:09 |
| quiquell | jfrancoa: no clue about it, we are not verifying too much non voting upgrade jobs | 14:09 |
| jfrancoa | quiquell: ack, just wanted to know that. Not to open a bug which is already opened | 14:09 |
| jfrancoa | quiquell: thanks | 14:09 |
| *** yolanda has joined #oooq | 14:12 | |
| weshay | rfolco I have to drop for 15 | 14:14 |
| rfolco | weshay, ok | 14:14 |
| weshay | rfolco please get to the problem def on this if possible / time | 14:14 |
| weshay | if time goes too long.. schedule it's own mtg | 14:14 |
| *** dsneddon has quit IRC | 14:21 | |
| marios_|ruck | weshay: rfolco sshnaidm going to go to escalation callin 5 and will rejoin | 14:24 |
| rfolco | ack | 14:24 |
| *** dsneddon has joined #oooq | 14:27 | |
| *** ykarel has joined #oooq | 14:28 | |
| sshnaidm | quiquell, don't you want to start from dry-runs? https://review.rdoproject.org/r/#/c/19859/24/zuul.yaml | 14:29 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-containerized-undercloud-upgrades, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci- (3 more messages) | 14:31 |
| sshnaidm | quiquell, maybe let's switch f28 host to run dry-run, but leave centos7 hosts with hello-job | 14:33 |
| sshnaidm | quiquell, because of these tcg etc centos7 host just doesn't handle this | 14:33 |
| quiquell | sshnaidm: we can use standalone at host(c7, f28),libvirt(f28) | 14:36 |
| quiquell | sshnaidm: And we keep libvirt(c7) with hello-reproducer | 14:36 |
| sshnaidm | quiquell, yeah, that's what I suggest | 14:37 |
| sshnaidm | quiquell, although I would let it run a little, because passing once is not yet success.. | 14:37 |
| quiquell | sshnaidm: ack | 14:37 |
| sshnaidm | something bad happens when creating a swap | 14:38 |
| sshnaidm | centos dies after it, f28 just died too..\ | 14:38 |
| *** dsneddon has quit IRC | 14:40 | |
| * chandankumar needs to drop now | 14:41 | |
| sshnaidm | rfolco, let's add PTO section | 14:45 |
| *** yolanda has quit IRC | 14:50 | |
| marios_|ruck | sshnaidm: https://www.timeanddate.com/countdown/generic?iso=20190403T18&p0=680&font=cursive&csz=1$ | 14:56 |
| marios_|ruck | \o/ | 14:56 |
| marios | marios_|ruck: you doing down! | 14:56 |
| sshnaidm | marios, \\o// | 14:56 |
| marios_|ruck | marios: what did i ever do to you man | 14:56 |
| sshnaidm | marios, hit him! | 14:57 |
| * marios smack marios_|ruck | 14:57 | |
| *** yolanda has joined #oooq | 14:57 | |
| *** dsneddon has joined #oooq | 15:01 | |
| *** quiquell is now known as quiquell|off | 15:38 | |
| *** panda is now known as panda|afk | 15:38 | |
| rlandy | marios_|ruck: you can renick | 15:39 |
| marios_|ruck | rlandy: quiquell|off current status in https://etherpad.openstack.org/p/ruckroversprint7 | 15:39 |
| marios | marios_|ruck: go away nobody likes you | 15:39 |
| * marios_|ruck :( | 15:40 | |
| *** marios_|ruck has quit IRC | 15:40 | |
| rlandy | thanks | 15:40 |
| *** rlandy is now known as rlandy|ruck | 15:40 | |
| marios | rlandy|ruck: master/pike/rocky good shape and promoting ... queens blocked on the issues at the etherpad... in particular https://review.openstack.org/#/c/649084/ but is blocked on https://bugs.launchpad.net/tripleo/+bug/1822120 | 15:41 |
| openstack | Launchpad bug 1822120 in tripleo "[CI][Queens] undercloud-upgrade task fails due to rabbitmq" [Critical,Triaged] - Assigned to Cédric Jeanneret (cjeanner) | 15:41 |
| rlandy|ruck | ok | 15:41 |
| marios | rlandy|ruck: all the known things are in https://etherpad.openstack.org/p/ruckroversprint7 | 15:41 |
| rlandy|ruck | I'll read it | 15:41 |
| marios | rlandy|ruck: ack, there's not too much there | 15:41 |
| * sshnaidm folds rovers uniform | 15:44 | |
| marios | sshnaidm: its been an honour | 15:46 |
| * marios alute | 15:46 | |
| marios | salute fail | 15:46 |
| weshay | quiquell|off rlandy|ruck current know issues are on the cockpit promotion_blocker and alert bugs | 15:47 |
| rlandy|ruck | ack - reading through that is a few | 15:48 |
| rlandy|ruck | just fixing something for pidone | 15:48 |
| weshay | rlandy|ruck first order of business is to update the "ruck rover notes" section to point to a new etherpad | 15:48 |
| rlandy|ruck | k | 15:48 |
| weshay | which I think can be done via the ui if you login | 15:48 |
| sshnaidm | marios, need a decompression, like for divers | 15:48 |
| weshay | creds are in infra doc | 15:48 |
| rlandy|ruck | weshay: k - new etherpad https://etherpad.openstack.org/p/ruckroversprint8 | 15:52 |
| weshay | k.. check that you can log into the cockpit and update it | 15:52 |
| weshay | thanks | 15:52 |
| weshay | rfolco we should prep the board for tomorrow | 15:52 |
| weshay | when you have time | 15:53 |
| rlandy|ruck | checking | 15:53 |
| * marios initiate shutdown sequence | 15:53 | |
| *** panda|afk is now known as panda | 15:54 | |
| rfolco | weshay, let me finish my lunch and will ping you back | 15:55 |
| rlandy|ruck | weshay: idiotic question - log into where? I can get onto the infra instance | 16:00 |
| rlandy|ruck | I see the creds | 16:00 |
| rlandy|ruck | oh nvm | 16:01 |
| rlandy|ruck | I see it | 16:01 |
| rlandy|ruck | logged in | 16:02 |
| *** marios has quit IRC | 16:02 | |
| *** ykarel is now known as ykarel|away | 16:02 | |
| rlandy|ruck | weshay: k - edit done | 16:03 |
| weshay | perfect | 16:04 |
| rlandy|ruck | weshay: left comment - marking this fixed released https://bugs.launchpad.net/tripleo/+bug/1822888, ok? | 16:06 |
| openstack | Launchpad bug 1822888 in tripleo "f28 container build broken by tripleo-repos InvalidArguments('Branches only suppported with centos7')" [Critical,Triaged] | 16:06 |
| *** bogdando has quit IRC | 16:08 | |
| *** panda is now known as panda|off | 16:08 | |
| *** trown is now known as trown|lunch | 16:10 | |
| *** dsneddon has quit IRC | 16:10 | |
| *** jfrancoa has quit IRC | 16:11 | |
| *** jpena is now known as jpena|off | 16:14 | |
| chandankumar | kopecmartin: this will unblock https://review.openstack.org/#/c/649653/ docs failure on os_tempest | 16:15 |
| rfolco | weshay, I am ready | 16:16 |
| weshay | k.. 1 min | 16:16 |
| weshay | rfolco k.. join my blue | 16:17 |
| *** udesale has quit IRC | 16:19 | |
| rlandy|ruck | sshnaidm: question on https://etherpad.openstack.org/p/ruckroversprint7 line 13 | 16:20 |
| rlandy|ruck | 'no bug yet'? | 16:20 |
| rlandy|ruck | is that because a bug was just not created before the meeting | 16:20 |
| rlandy|ruck | or you guys expected this is a one-off? | 16:21 |
| sshnaidm | rlandy|ruck, I think it happened once only | 16:22 |
| rlandy|ruck | fine - will watch it | 16:22 |
| sshnaidm | rlandy|ruck, so nothing was opened, last job passed well | 16:22 |
| *** panda|off has quit IRC | 16:24 | |
| *** dtantsur is now known as dtantsur|afk | 16:24 | |
| *** kopecmartin is now known as kopecmartin|off | 16:25 | |
| *** ykarel|away has quit IRC | 16:30 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-containerized-undercloud-upgrades, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci- (3 more messages) | 16:31 |
| *** dsneddon has joined #oooq | 16:33 | |
| *** dsneddon has quit IRC | 16:38 | |
| rlandy|ruck | node failures | 16:39 |
| *** gkadam has quit IRC | 16:42 | |
| *** derekh has quit IRC | 16:56 | |
| *** trown|lunch is now known as trown | 17:00 | |
| weshay | sshnaidm have you seen the reproducer get through tempest? | 17:02 |
| * weshay tried a standalone-upgrade job and tempest fails on it | 17:03 | |
| weshay | so wondering if other jobs work | 17:03 |
| sshnaidm | weshay, yes, for standalone | 17:04 |
| weshay | ah k | 17:04 |
| weshay | good to know.. arxcruz ^ | 17:04 |
| weshay | arxcruz the tempest error may be directly related to upgrade | 17:04 |
| sshnaidm | weshay, but I used rdo cloud, not libvirt | 17:04 |
| weshay | sshnaidm same | 17:04 |
| weshay | maybe I'll try libvirt so I can get on the bo | 17:05 |
| weshay | x | 17:05 |
| weshay | sshnaidm rlandy|ruck so with regards to holding the node.. we do not have ability to do that bit still correct? | 17:05 |
| weshay | unless it's libvirt and the job fails | 17:05 |
| rlandy|ruck | not in the reproducer as written | 17:06 |
| rlandy|ruck | you can do it with manual execution | 17:06 |
| rlandy|ruck | we have an option | 17:06 |
| rlandy|ruck | but it never worked | 17:06 |
| sshnaidm | weshay, docker-compose exec scheduler zuul autohold --tenant tripleo-ci-reproducer --job tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001 --reason debug --project test1 | 17:06 |
| weshay | so that works or no? | 17:06 |
| sshnaidm | weshay, it works if your job failed | 17:07 |
| weshay | ah k.. /me tries that | 17:07 |
| * weshay adds to the doc for now, so others know as well | 17:07 | |
| *** dsneddon has joined #oooq | 17:08 | |
| weshay | sshnaidm so looking at my current run... I have tripleo-ci-centos-7-standalone-upgrade-dlrn-hash-tag | 17:17 |
| weshay | so for --job would I specify tripleo-ci-centos-7-standalone-upgrade-dlrn-hash-tag or tripleo-ci-centos-7-standalone-upgrade | 17:17 |
| sshnaidm | weshay, the name of job you run - tripleo-ci-centos-7-standalone-upgrade-dlrn-hash-tag | 17:17 |
| weshay | for instructions is including dlrn-hash-tag appropriate? | 17:18 |
| weshay | I'll put a review and let you see | 17:18 |
| *** Vorrtex has quit IRC | 17:23 | |
| *** amoralej is now known as amoralej|off | 17:25 | |
| weshay | sshnaidm sorry, somethings up w/ my gerrit connection.. very slow https://review.openstack.org/649677 | 17:25 |
| rlandy|ruck | weshay: 1-1? | 17:30 |
| weshay | aye | 17:31 |
| *** Vorrtex has joined #oooq | 17:37 | |
| weshay | sshnaidm ping.. we had the openstack-infra tenant cleaned up via a script on the te-broker | 17:42 |
| weshay | sshnaidm did that script move to another node? | 17:42 |
| weshay | sshnaidm nvermind | 17:43 |
| weshay | found it | 17:43 |
| *** apetrich has quit IRC | 17:46 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-containerized-undercloud-upgrades, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035, (3 more messages) | 18:31 |
| rlandy|ruck | weshay: fedora images are pushed 20 hours ago... they just take a while to render in the | 18:37 |
| rlandy|ruck | master and stein | 18:37 |
| *** holser_ has quit IRC | 18:54 | |
| *** sshnaidm is now known as sshnaidm|off | 19:00 | |
| rlandy|ruck | zbr: rfolco: ping ... config/release/tripleo-ci/Fedora-28/promotion-testing-hash-stein.yml is missing | 19:05 |
| rlandy|ruck | known? | 19:05 |
| rlandy|ruck | https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/periodic-tripleo-ci-fedora-28-standalone-stein/0e5c8be/job-output.txt.gz#_2019-04-03_15_15_50_371652 | 19:05 |
| rlandy|ruck | can't run fedora28 stein promotion w/o it | 19:06 |
| rlandy|ruck | weshay: ^^ fyi | 19:06 |
| weshay | ah.. rlandy|ruck easy enough | 19:07 |
| rlandy|ruck | weshay; yep - creating bug and putting in review | 19:07 |
| rlandy|ruck | just seems strange we missed this | 19:07 |
| rlandy|ruck | that is why asking zbr | 19:07 |
| rlandy|ruck | and rfolco | 19:08 |
| weshay | rlandy|ruck that could have been my fault in zbr and rfolco's defense I took over the copying of the release files because they were driving zbr bonkers | 19:21 |
| rlandy|ruck | weshay: for your sins, you can review my patch coming up in a minute | 19:22 |
| weshay | k.. and rlandy|ruck and gerrit is normal for you? | 19:22 |
| weshay | I think I have a networking issue introduced w/ a new router | 19:22 |
| weshay | but I only see it w/ gerrit | 19:22 |
| rlandy|ruck | seems to be | 19:22 |
| rfolco | rlandy|ruck, pls add me too, I was not in that original change, but can help fixing | 19:22 |
| rlandy|ruck | weshay: rfolco: pls review ... https://review.openstack.org/#/c/649706/ | 19:26 |
| rlandy|ruck | hastily put together so expecting comments :) | 19:26 |
| *** sshnaidm_ has joined #oooq | 19:36 | |
| *** sshnaidm|off has quit IRC | 19:36 | |
| weshay | rlandy|ruck done | 19:39 |
| * rlandy|ruck looks | 19:42 | |
| rlandy|ruck | lol - no -1 comments ... shocked | 19:42 |
| *** tosky has quit IRC | 19:44 | |
| weshay | ? | 19:44 |
| weshay | eh | 19:44 |
| weshay | heh | 19:44 |
| weshay | rlandy|ruck back at cha https://review.openstack.org/#/c/649705/ | 19:44 |
| rlandy|ruck | weshay: how often do we expect alert bug s to update on cockpit | 19:51 |
| rlandy|ruck | list is old | 19:51 |
| weshay | really? | 19:51 |
| * weshay looks | 19:51 | |
| weshay | rlandy|ruck what makes you think it's old? | 19:51 |
| rlandy|ruck | 1822888 | 19:51 |
| rlandy|ruck | link to bug | 19:51 |
| rlandy|ruck | Triaged"f28 container build broken by tripleo-repos InvalidArguments('Branches only suppported with centos7')" | 19:51 |
| rlandy|ruck | ^^ changed that to fix-released | 19:51 |
| rlandy|ruck | added a new alert bug | 19:52 |
| rlandy|ruck | not on the list | 19:52 |
| weshay | rlandy|ruck it's there | 19:52 |
| weshay | oh wait | 19:53 |
| rlandy|ruck | where? | 19:53 |
| * rlandy|ruck is blind | 19:53 | |
| weshay | rlandy|ruck what's the bug # ur looking for | 19:54 |
| weshay | I don't see it in launchpad | 19:54 |
| rlandy|ruck | https://bugs.launchpad.net/tripleo/+bug/1823066 | 19:54 |
| openstack | Launchpad bug 1823066 in tripleo "periodic-tripleo-ci-fedora-28-standalone-stein job is failing due to missing config/release/tripleo-ci/Fedora-28/promotion-testing-hash-stein.yml file" [Critical,In progress] - Assigned to Ronelle Landy (rlandy) | 19:54 |
| weshay | that one is there | 19:54 |
| weshay | under alert | 19:54 |
| weshay | you may need to refresh | 19:55 |
| weshay | 1823066 | 19:55 |
| weshay | link to bug | 19:55 |
| weshay | In Progress"periodic-tripleo-ci-fedora-28-standalone-stein job is failing due to missing config/release/tripleo-ci/Fedora-28/promotion-testing-hash-stein.yml file" | 19:55 |
| * rlandy|ruck sees only the following .... window too small? | 19:56 | |
| rlandy|ruck | 1823030 | 19:56 |
| rlandy|ruck | link to bug | 19:56 |
| rlandy|ruck | 19:56 | |
| rlandy|ruck | Triaged | 19:56 |
| rlandy|ruck | "improve debug output of [validate-services : Fails if we find failed systemd units]" | 19:56 |
| rlandy|ruck | 1822916 Triaged"tempest config error in standalone upgrade in reproducer only: TypeError: must be string or buffer | 19:56 |
| rlandy|ruck | 1822888 Triaged"f28 container build broken by tripleo-repos InvalidArguments('Branches only suppported with centos7')" | 19:56 |
| rlandy|ruck | 1819805 Triaged"zuul reproducer unable to pip install rdopkg" | 19:56 |
| rlandy|ruck | 1822867 In Progress"rmtree failed: [Errno 16] Device or resource busy: '/var/lib/docker/overlay2" | 19:56 |
| rlandy|ruck | 1815744 In Progress"build-test-package does not fail on partial builds" | 19:56 |
| weshay | rlandy|ruck even after a rfresh? | 19:56 |
| rlandy|ruck | weshay: ok - now I see it - I had to leave page and come back | 19:57 |
| weshay | k.. there is a refresh button on the page too, upper right | 19:57 |
| weshay | not sure why a regular refresh would not have worked | 19:58 |
| weshay | rlandy|ruck oddly I still see current ruck rover etherpad https://etherpad.openstack.org/p/ruckroversprint7 | 19:58 |
| weshay | which means we may need to update the txt in gerrit | 19:59 |
| weshay | rlandy|ruck any help needed for master f28? | 19:59 |
| weshay | or we're just going to watch it tomorrow | 19:59 |
| weshay | which is fine | 19:59 |
| rlandy|ruck | weshay: weird let me try edit again - hold on | 19:59 |
| rlandy|ruck | weshay: k - click edit on rock rover notes - make the change and then? return to dashboard. Is there a 'save' I am missing? | 20:01 |
| rlandy|ruck | weshay: k - found the save | 20:02 |
| rlandy|ruck | should be updated now | 20:02 |
| rlandy|ruck | weshay: wrt master f28 - going to watch it again tonight | 20:02 |
| rlandy|ruck | the older hashes pushed fine - 4 hours ago | 20:03 |
| rlandy|ruck | weshay: (sorry for all the questions) https://ci.centos.org/view/rdo/view/tripleo-gate/ pretty broken - do we still care? | 20:07 |
| weshay | k | 20:07 |
| rlandy|ruck | if yes, will fix | 20:08 |
| * rlandy|ruck looks | 20:08 | |
| rlandy|ruck | Install the undercloud - error | 20:09 |
| rlandy|ruck | 2019-04-03 19:54:11 | Error message: Options that has been deprecated and removed/replaced detected. Invalid options: ['masquerade_network'] | 20:09 |
| rlandy|ruck | tripleo-quickstart-gate-master-delorean-quick-basic - weird - release not picked up | 20:12 |
| weshay | talking to mike 1 sec | 20:13 |
| rlandy|ruck | {% if release in ['mitaka', 'newton', 'ocata', 'pike'] %} | 20:13 |
| rlandy|ruck | --release centosci/master | 20:13 |
| * rlandy|ruck just talking to myself - no worries | 20:13 | |
| weshay | rlandy|ruck I'll handle ci.centos | 20:26 |
| rlandy|ruck | weshay: half way there ... | 20:27 |
| rlandy|ruck | problem is here | 20:27 |
| rlandy|ruck | https://github.com/openstack/tripleo-quickstart-extras/blob/master/roles/undercloud-deploy/templates/undercloud.conf.j2#L152 | 20:27 |
| rlandy|ruck | masquerade_network should not be included with master | 20:27 |
| rlandy|ruck | https://github.com/openstack/tripleo-quickstart/blob/master/config/release/trunk/master.yml | 20:27 |
| rlandy|ruck | ^^ release file passed | 20:27 |
| rlandy|ruck | release: master | 20:28 |
| rlandy|ruck | https://ci.centos.org/artifacts/rdo/jenkins-tripleo-quickstart-gate-master-delorean-quick-basic-8711/undercloud/home/stack/undercloud.conf.gz | 20:29 |
| rlandy|ruck | ^^ has values that should be excluded with master | 20:29 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-containerized-undercloud-upgrades, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035, (3 more messages) | 20:31 |
| vinaykns | hello channel I couldn't run quickstart hitting this issue http://pastebin.test.redhat.com/749400 | 20:41 |
| vinaykns | couldn't do a curl request | 20:41 |
| vinaykns | weshay..^^ | 20:41 |
| *** Vorrtex has quit IRC | 20:47 | |
| *** trown is now known as trown|outtypewww | 20:52 | |
| *** mjturek has quit IRC | 20:57 | |
| rlandy|ruck | https://images.rdoproject.org/master/rdo_trunk/current-tripleo/overcloud-full.tar.md5 ... https://images.rdoproject.org/master/rdo_trunk/ ... no current_tripleo | 20:57 |
| rlandy|ruck | current-tripleo-rdp | 20:57 |
| rlandy|ruck | rdo | 20:57 |
| vinaykns | aah okay.. | 21:01 |
| vinaykns | lemme try with that again | 21:01 |
| vinaykns | rlandy|ruck: how should I change that in the oooq | 21:04 |
| rlandy|ruck | depends on the release file you are passing | 21:05 |
| vinaykns | currently I'm passing master-tripleo-ci | 21:05 |
| vinaykns | do you want me to try with rocky..? | 21:07 |
| rlandy|ruck | rocky should work atm https://images.rdoproject.org/rocky/rdo_trunk/current-tripleo/ | 21:09 |
| rlandy|ruck | current-tripleo images are not at master atm | 21:10 |
| vinaykns | okay..will try that | 21:10 |
| vinaykns | rlandy|ruck: what should I pass to capture current-tripleo-rdo | 21:14 |
| vinaykns | in the oooq | 21:14 |
| rlandy|ruck | you shouldn't need to - in general those images should be there | 21:26 |
| rlandy|ruck | https://github.com/openstack/tripleo-quickstart/blob/master/config/release/trunk/master.yml | 21:27 |
| rlandy|ruck | ^^ will get current-tripleo-rd | 21:27 |
| rlandy|ruck | rdo for the moment | 21:27 |
| *** aakarsh has quit IRC | 21:37 | |
| rlandy|ruck | 2019-04-03 04:22:25,340 6846 INFO promoter Promoting the container images for dlrn hash 06f30fb4ebbc998898dbb6db7b1b6db644b0215c on master to current-tripleo | 21:37 |
| rlandy|ruck | promoter sftp> ln -s /var/www/html/images/master/rdo_trunk/06f30fb4ebbc998898dbb6db7b1b6db644b0215c_3457cc9b /var/www/html/images/master/rdo_trunk/current-tripleo | 21:44 |
| rlandy|ruck | weshay" ^^ should we manually promote new hash with link | 21:45 |
| rlandy|ruck | or wait for next promotion? | 21:45 |
| weshay | we need to fix it now | 21:46 |
| weshay | I just need to get off the phone | 21:46 |
| weshay | rlandy|ruck ok finally | 21:49 |
| weshay | want to tmate? still have time | 21:49 |
| rlandy|ruck | weshay: k - on the prmoter server | 21:49 |
| rlandy|ruck | yes | 21:49 |
| weshay | ah perfect | 21:49 |
| weshay | rlandy|ruck https://bluejeans.com/u/whayutin/ | 21:50 |
| rlandy|ruck | let me get out and tmate | 21:50 |
| rlandy|ruck | sec | 21:50 |
| rlandy|ruck | vinaykns: https://images.rdoproject.org/master/rdo_trunk/current-tripleo/ is back | 21:55 |
| rlandy|ruck | you can go back to original file | 21:56 |
| vinaykns | cool, will try again | 21:56 |
| vinaykns | thx..! | 21:56 |
| *** rlandy|ruck is now known as rlandy | 22:08 | |
| *** weshay is now known as weshay|ruck | 22:08 | |
| *** rlandy is now known as rlandy|bbl | 22:09 | |
| *** vinaykns has quit IRC | 22:17 | |
| *** aakarsh has joined #oooq | 22:27 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-containerized-undercloud-upgrades, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035, (3 more messages) | 22:31 |
| *** dsneddon has quit IRC | 22:43 | |
| *** dsneddon has joined #oooq | 22:50 | |
| *** vinaykns has joined #oooq | 23:52 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!