| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039, tripleo-ci-centos-7-scenario001-multinode-oooq-container @ (2 more messages) | 00:29 |
|---|---|---|
| *** dsneddon has quit IRC | 02:14 | |
| *** irclogbot_2 has quit IRC | 02:19 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039, tripleo-ci-centos-7-scenario001-multinode-oooq-container @ (2 more messages) | 02:29 |
| *** apetrich has quit IRC | 02:58 | |
| *** dsneddon has joined #oooq | 03:17 | |
| *** dsneddon has quit IRC | 03:21 | |
| *** dsneddon has joined #oooq | 03:36 | |
| *** dsneddon has quit IRC | 03:41 | |
| *** dsneddon has joined #oooq | 03:52 | |
| *** dsneddon has quit IRC | 03:59 | |
| *** dsneddon has joined #oooq | 04:00 | |
| *** udesale has joined #oooq | 04:02 | |
| *** dsneddon has quit IRC | 04:05 | |
| *** ykarel has joined #oooq | 04:08 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039, tripleo-ci-centos-7-scenario004-multinode-oooq-container @ (2 more messages) | 04:30 |
| *** jaosorior has joined #oooq | 04:39 | |
| *** ykarel has quit IRC | 04:52 | |
| *** ykarel has joined #oooq | 05:15 | |
| *** ratailor has joined #oooq | 05:32 | |
| *** chandankumar has joined #oooq | 05:33 | |
| *** marios has joined #oooq | 05:42 | |
| *** quiquell has joined #oooq | 05:42 | |
| *** marios_|ruck has joined #oooq | 05:43 | |
| *** jaosorior has quit IRC | 05:46 | |
| *** jaosorior has joined #oooq | 05:47 | |
| quiquell | sshnaidm|off: ping | 05:47 |
| *** dsneddon has joined #oooq | 05:56 | |
| *** jfrancoa has joined #oooq | 05:57 | |
| *** dsneddon has quit IRC | 06:01 | |
| *** dsneddon has joined #oooq | 06:06 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039, tripleo-ci-centos-7-scenario004-multinode-oooq-container @ (2 more messages) | 06:30 |
| *** kopecmartin|off is now known as kopecmartin | 06:30 | |
| *** ratailor_ has joined #oooq | 06:38 | |
| *** ratailor has quit IRC | 06:40 | |
| *** holser_ has joined #oooq | 06:48 | |
| *** ccamacho has quit IRC | 06:53 | |
| *** tosky has joined #oooq | 07:13 | |
| *** quiquell is now known as quiquell|brb | 07:22 | |
| *** ccamacho has joined #oooq | 07:22 | |
| *** dsneddon has quit IRC | 07:28 | |
| *** apetrich has joined #oooq | 07:32 | |
| *** amoralej|off is now known as amoralej | 07:34 | |
| *** ykarel is now known as ykarel|lunch | 07:35 | |
| *** quiquell|brb is now known as quiquell | 07:46 | |
| marios_|ruck | arxcruz: chandankumar kopecmartin sshnaidm|off o/ folks can we sync about that queens issue today? please https://bugs.launchpad.net/tripleo/+bug/1822080 | 07:47 |
| openstack | Launchpad bug 1822080 in tripleo "periodic centos-7-ovb-1ctlr_1comp-featureset020-queens failing tempest "Endpoint not found"" [Critical,Triaged] - Assigned to Martin Kopec (mkopec) | 07:47 |
| marios_|ruck | arxcruz: chandankumar kopecmartin sshnaidm|off if i send a call for 1 hour from now does it work for you? maybe quick call even 15 mins | 07:47 |
| kopecmartin | sure | 07:47 |
| *** sshnaidm|off is now known as sshnaidm|rover | 07:48 | |
| sshnaidm|rover | quiquell, pong | 07:48 |
| sshnaidm|rover | marios_|ruck, works for me | 07:49 |
| marios_|ruck | thnks kopecmartin will wait few moremins and send an invite ack sshnaidm|rover | 07:49 |
| quiquell | sshnaidm|rover: stats /dev/log issue with reproducer is gone updateing images | 07:49 |
| sshnaidm|rover | quiquell, great, as I thought.. | 07:49 |
| quiquell | sshnaidm|rover: I am preparing a playbook to update them | 07:49 |
| sshnaidm|rover | quiquell, cool, thanks | 07:49 |
| *** dsneddon has joined #oooq | 07:52 | |
| chandankumar | marios_|ruck: I need to go for 1:1 in another 30 mins | 07:53 |
| marios_|ruck | chandankumar: ack thanks join if/when you can np | 07:54 |
| *** jbadiapa has joined #oooq | 07:54 | |
| *** jpena|off is now known as jpena | 07:57 | |
| *** gkadam has joined #oooq | 08:02 | |
| arxcruz | marios_|ruck: sure | 08:05 |
| arxcruz | chandankumar: around? let's start the plugins release | 08:05 |
| *** zbr is now known as zbr|pto | 08:08 | |
| zbr|pto | hi!. i will be on pto today, no reason to worry, just had to finish few errands. | 08:08 |
| quiquell | marios_|ruck: what was the issue with repro ? | 08:08 |
| marios_|ruck | quiquell: i hit two things, both hit before. first was 'no test equalto' jinja related (rerun cleared it friday, so wtf) and then 500 error waiting for zuul | 08:10 |
| marios_|ruck | quiquell: centos7 vm yum update first & reboot - but still hitting equalto issue http://paste.openstack.org/show/748594/ & rerun clears (wtf) brief irc weshay, finally fails again like http://paste.openstack.org/show/748596/ like last time 500 error | 08:10 |
| quiquell | marios_|ruck: pufff :-/ | 08:12 |
| quiquell | marios_|ruck: 500 issue is more like a symptom | 08:15 |
| *** bogdando has joined #oooq | 08:18 | |
| *** ykarel|lunch is now known as ykarel | 08:19 | |
| chandankumar | arxcruz: yes | 08:20 |
| chandankumar | arxcruz: https://tree.taiga.io/project/tripleo-ci-board/task/898 | 08:24 |
| arxcruz | chandankumar: so, only need to change rdo/config ? | 08:25 |
| chandankumar | arxcruz: nope | 08:26 |
| arxcruz | you mentioned 3 patches for each plugin | 08:26 |
| chandankumar | arxcruz: I am making a doc and sharing with you | 08:26 |
| arxcruz | ok | 08:26 |
| sshnaidm|rover | chandankumar, arxcruz hi, folks, need your help in tempest novajoin: http://logs.rdoproject.org/24/639324/12/openstack-check/tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039/656e761/logs/undercloud/home/zuul/tempest.log.gz | 08:28 |
| sshnaidm|rover | arxcruz, chandankumar it's new job | 08:28 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039, tripleo-ci-centos-7-scenario004-multinode-oooq-container @ (2 more messages) | 08:30 |
| *** gkadam has quit IRC | 08:32 | |
| chandankumar | arxcruz: https://gitlab.cee.redhat.com/snippets/830 | 08:33 |
| *** dtantsur|afk is now known as dtantsur | 08:35 | |
| chandankumar | arxcruz: let's create a section with your name so that we can know on which we are working on | 08:35 |
| *** derekh has joined #oooq | 08:37 | |
| arxcruz | chandankumar: just a sec, need to finish some stuff here at home | 08:44 |
| marios_|ruck | arxcruz: chandankumar kopecmartin sshnaidm|rover o/ folks reminder call in 5 join if you can bluejeans in calendar | 08:55 |
| chandankumar | marios_|ruck: in meeting | 08:56 |
| marios_|ruck | chandankumar: ack thx | 09:00 |
| quiquell | sshnaidm|rover: what was the command to upload the images ? | 09:09 |
| *** chem has joined #oooq | 09:09 | |
| sshnaidm|rover | quiquell, "image create" I suppose | 09:10 |
| sshnaidm|rover | quiquell, but do we want to delete previous before? | 09:10 |
| quiquell | I know I mean about public flags and all | 09:10 |
| quiquell | eyp I will just overwrite | 09:10 |
| sshnaidm|rover | quiquell, openstack image set --property visibility=shared $ID | 09:11 |
| quiquell | ok two steps | 09:11 |
| sshnaidm|rover | quiquell, glance member-create ${IMAGE_ID} ${TENANT_ID} | 09:11 |
| quiquell | thanks | 09:11 |
| sshnaidm|rover | quiquell, I left necessary steps in taiga task, did you see them? | 09:12 |
| sshnaidm|rover | quiquell, like cloud init install, etc | 09:12 |
| quiquell | pass me the link | 09:12 |
| quiquell | member-create is necessary ? | 09:13 |
| quiquell | They are public now | 09:13 |
| quiquell | sshnaidm|rover: I am going to use new names | 09:14 |
| quiquell | sshnaidm|rover: upstream-tripleo-ci-reproducer-centos-7 and centos-28 | 09:14 |
| quiquell | sshnaidm|rover: to be clear | 09:15 |
| quiquell | sshnaidm|rover: ack ? | 09:15 |
| sshnaidm|rover | quiquell, did you check them? | 09:15 |
| *** jrist has quit IRC | 09:15 | |
| quiquell | not yet | 09:15 |
| quiquell | I am going to upload new ones | 09:15 |
| quiquell | a damn the public one is the one with cloudinit :-/ | 09:15 |
| quiquell | ok step by step | 09:16 |
| quiquell | upload new ones | 09:16 |
| quiquell | creatd with playbook | 09:16 |
| quiquell | test it | 09:16 |
| sshnaidm|rover | quiquell, taiga doesn't have a good search, will take time to find it.. | 09:16 |
| *** jrist has joined #oooq | 09:16 | |
| quiquell | sshnaidm|rover: I am doing this openstack --os-cloud nodepool image create --disk-format raw --container-format bare --min-disk 8 --file $image_path {{ item.dest }} | 09:16 |
| sshnaidm|rover | quiquell, wait a sec pls | 09:17 |
| quiquell | sshnaidm|rover: ack | 09:18 |
| sshnaidm|rover | quiquell, this is the steps: https://tree.taiga.io/project/tripleo-ci-board/task/640?kanban-status=1447277 | 09:18 |
| sshnaidm|rover | quiquell, and this is the epic: https://tree.taiga.io/project/tripleo-ci-board/us/681 | 09:19 |
| quiquell | sshnaidm|rover: ack, I think is same thing by image type is raw | 09:20 |
| quiquell | sshnaidm|rover: is the type we are downloading from nodepool | 09:20 |
| quiquell | sshnaidm|rover: was debug it here https://tree.taiga.io/project/tripleo-ci-board/task/935?kanban-status=1447275 | 09:22 |
| sshnaidm|rover | quiquell, we have qcow2 https://nb01.openstack.org/images/ | 09:28 |
| *** gkadam has joined #oooq | 09:28 | |
| arxcruz | chandankumar: can we bj and you go through it with me ? | 09:29 |
| quiquell | sshnaidm|rover: Let me put some stuff to review | 09:29 |
| quiquell | sshnaidm|rover: I am just testing | 09:30 |
| arxcruz | chandankumar: ? | 09:33 |
| marios_|ruck | arxcruz: kopecmartin sshnaidm|rover fyi thanks comment #14 https://bugs.launchpad.net/tripleo/+bug/1822080 | 09:37 |
| openstack | Launchpad bug 1822080 in tripleo "periodic centos-7-ovb-1ctlr_1comp-featureset020-queens failing tempest "Endpoint not found"" [Critical,Triaged] - Assigned to Martin Kopec (mkopec) | 09:37 |
| arxcruz | chandankumar: | 09:42 |
| arxcruz | ykarel: around ? | 09:45 |
| ykarel | arxcruz, yes | 09:46 |
| chandankumar | arxcruz: yes | 09:49 |
| chandankumar | marios_|ruck: checking | 09:49 |
| arxcruz | chandankumar: walkthrough with me with one | 09:49 |
| arxcruz | your checklist doesn't explain too much | 09:50 |
| arxcruz | for example, cinder plugin | 09:50 |
| arxcruz | in taiga you put cinder-tests-tempest | 09:50 |
| arxcruz | when i tried to clone with rdopkg says this doesn't exist | 09:50 |
| arxcruz | it took me a while to realize the name is python-cinder-tests-tempest | 09:51 |
| arxcruz | and realize I can get it from https://github.com/redhat-openstack/rdoinfo/blob/master/rdo.yml | 09:51 |
| arxcruz | now, I have doubts regarding the requirements | 09:51 |
| chandankumar | arxcruz: one min | 09:51 |
| arxcruz | do i need to update anything, is there a easy way check the requirements or is it a manual thing | 09:52 |
| arxcruz | tempest >= 18, shall we move to tempest >= 20 ? | 09:52 |
| arxcruz | or 19 | 09:52 |
| arxcruz | k | 09:52 |
| arxcruz | i'll grab a coffee brb | 09:53 |
| *** d0ugal has quit IRC | 09:55 | |
| chandankumar | arxcruz: bj https://redhat.bluejeans.com/1571313919/6145 | 09:57 |
| *** holser_ is now known as holser|lunch | 09:58 | |
| *** d0ugal has joined #oooq | 10:05 | |
| chandankumar | arxcruz: let me know when you are back! | 10:18 |
| chandankumar | sshnaidm|rover: I will take a look at nova join job keeping high in todo | 10:19 |
| sshnaidm|rover | chandankumar, thanks | 10:21 |
| *** sshnaidm|rover is now known as sshnaidm|rover|e | 10:22 | |
| chandankumar | arxcruz: rdopkg reqcheck | 10:25 |
| arxcruz | chandankumar: http://paste.openstack.org/show/748650/ | 10:26 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039, tripleo-ci-centos-7-scenario004-multinode-oooq-container @ (2 more messages) | 10:30 |
| arxcruz | chandankumar: https://review.rdoproject.org/r/#/c/19862/ | 10:31 |
| chandankumar | arxcruz: stein-branching | 10:32 |
| chandankumar | arxcruz: https://review.rdoproject.org/r/#/c/19804/2 | 10:33 |
| chandankumar | arxcruz: https://github.com/rdo-infra/review.rdoproject.org-config | 10:37 |
| chandankumar | arxcruz: https://github.com/rdo-packages/cinder-tempest-plugin-distgit/commits/rpm-master | 10:38 |
| chandankumar | arxcruz: http://paste.openstack.org/show/748652/ | 10:40 |
| arxcruz | chandankumar: https://review.rdoproject.org/r/#/c/19863/ | 10:40 |
| chandankumar | arxcruz: https://review.openstack.org/#/q/project:openstack/releases | 10:41 |
| chandankumar | arxcruz: https://review.openstack.org/#/q/project:openstack/releases+cinder-tempest-plugin | 10:41 |
| chandankumar | arxcruz: https://review.openstack.org/#/c/648265/2/deliverables/stein/cinder-tempest-plugin.yaml | 10:42 |
| chandankumar | arxcruz: http://paste.openstack.org/show/748653/ | 10:44 |
| arxcruz | chandankumar: https://review.rdoproject.org/r/#/c/19864/ | 10:46 |
| chandankumar | ykarel: nice info | 10:49 |
| ykarel | chandankumar, ? heat plugin? | 10:50 |
| chandankumar | ykarel: yes | 10:50 |
| ykarel | ack | 10:51 |
| *** ykarel is now known as ykarel|afk | 10:53 | |
| *** ykarel|afk is now known as ykarel | 11:12 | |
| *** udesale has quit IRC | 11:22 | |
| *** holser|lunch is now known as holser_ | 11:25 | |
| *** jpena is now known as jpena|lunch | 11:35 | |
| *** ratailor_ has quit IRC | 11:46 | |
| *** sshnaidm|rover|e is now known as sshnaidm|rover | 11:46 | |
| quiquell | sshnaidm|rover: updated reproducer image with upstream images and proper virt-customize https://review.rdoproject.org/r/#/c/19861/ | 11:47 |
| sshnaidm|rover | quiquell, what's about names? | 11:49 |
| sshnaidm|rover | quiquell, and old images | 11:49 |
| sshnaidm|rover | quiquell, should we delete old one before/after? | 11:50 |
| quiquell | sshnaidm|rover: yep, will add i t | 11:50 |
| quiquell | sshnaidm|rover: going to check that standalone works now with this ones | 11:50 |
| quiquell | sshnaidm|rover: rdo ones has same failure of stat /dev/log :-/ | 11:51 |
| quiquell | sshnaidm|rover: Can you add the delete thing at the review so I don't forget ? | 11:51 |
| sshnaidm|rover | quiquell, yeah, will review the whole patch | 11:52 |
| quiquell | ack | 11:52 |
| *** weshay|rover is now known as weshay | 12:03 | |
| weshay | morning | 12:03 |
| marios_|ruck | o/ afternoon | 12:03 |
| quiquell | weshay: hello there | 12:04 |
| *** quiquell is now known as quiquell|lunch | 12:09 | |
| *** trown|outtypewww is now known as trown | 12:12 | |
| *** rlandy has joined #oooq | 12:14 | |
| *** apetrich has quit IRC | 12:16 | |
| *** rfolco has joined #oooq | 12:16 | |
| *** apetrich has joined #oooq | 12:17 | |
| *** amoralej is now known as amoralej|lunch | 12:18 | |
| weshay | marios_|ruck you are working the cherry picks for https://bugs.launchpad.net/tripleo/+bug/1822080 ? | 12:21 |
| openstack | Launchpad bug 1822080 in tripleo "periodic centos-7-ovb-1ctlr_1comp-featureset020-queens failing tempest "Endpoint not found"" [Critical,Triaged] - Assigned to Martin Kopec (mkopec) | 12:21 |
| marios_|ruck | weshay: well, no. want to catch up with abishop again. we had a call with the tempest folks earlier today see comment #14 https://bugs.launchpad.net/tripleo/+bug/1822080 | 12:22 |
| openstack | Launchpad bug 1822080 in tripleo "periodic centos-7-ovb-1ctlr_1comp-featureset020-queens failing tempest "Endpoint not found"" [Critical,Triaged] - Assigned to Martin Kopec (mkopec) | 12:22 |
| weshay | k | 12:23 |
| weshay | thanks | 12:23 |
| marios_|ruck | weshay: escalation call in few mins, latest stuff is at https://etherpad.openstack.org/p/ruckroversprint7 otherwise we can catchyup after if you like | 12:23 |
| *** jpena|lunch is now known as jpena | 12:29 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039 @ https://review.openstack.org/602248, master: tripleo-ci-centos-7 (2 more messages) | 12:30 |
| weshay | marios_|ruck yes go through all | 12:35 |
| *** apetrich has quit IRC | 12:36 | |
| weshay | marios_|ruck NO | 12:41 |
| weshay | don't move to preventative | 12:41 |
| weshay | NO | 12:41 |
| weshay | NO | 12:41 |
| weshay | marios_|ruck leave it | 12:41 |
| weshay | where it is | 12:42 |
| *** quiquell|lunch is now known as quiquell | 12:43 | |
| quiquell | rfolco: o/ | 12:43 |
| rfolco | quiquell, hi o/ | 12:44 |
| quiquell | rfolco: f28 stein containres ready https://review.rdoproject.org/r/#/c/19819/ | 12:44 |
| quiquell | rfolco: we need to merge some stuff though and RDO guys fixed another stuff | 12:44 |
| quiquell | rfolco: I am going to push containers to docker hub ok ? | 12:45 |
| rfolco | quiquell, ok, can you please update task with what has been changed/fixed ? | 12:45 |
| quiquell | rfolco: have added a review there | 12:45 |
| quiquell | rfolco: and a comment with RDO work | 12:46 |
| rfolco | quiquell, oh fedora-stable nunances, I remeber that | 12:48 |
| rfolco | quiquell, thanks | 12:48 |
| quiquell | Let's review and merge those reviews | 12:48 |
| marios_|ruck | weshay: haha | 12:49 |
| marios_|ruck | weshay: just saw irc | 12:49 |
| marios_|ruck | :D | 12:49 |
| quiquell | sshnaidm|rover: bad news new images still fail with stat /dev/log, I am possible I saw it passing last week, but I just injected my pub key | 12:53 |
| sshnaidm|rover | quiquell, hmm | 12:53 |
| *** apetrich has joined #oooq | 12:53 | |
| quiquell | sshnaidm|rover: Going to re-run with last week image I used | 12:54 |
| sshnaidm|rover | quiquell, did you try the latest? | 12:54 |
| quiquell | sshnaidm|rover: yep | 12:55 |
| weshay | sshnaidm|rover quiquell did that fix container download/ | 12:55 |
| sshnaidm|rover | weshay, ? | 12:56 |
| quiquell | sshnaidm|rover: centos-7-0000034434 | 12:56 |
| weshay | sshnaidm|rover the standalone deployment was failing in container downloads because of /dev/log | 12:57 |
| quiquell | weshay: looks update image does not work, I am verify the manual update I did last week (I am possitive it was working) | 12:57 |
| weshay | new image may fix it | 12:57 |
| sshnaidm|rover | weshay, seems like doesn't | 12:57 |
| quiquell | weshay: that's what we are testing | 12:57 |
| quiquell | sshnaidm|rover: could be that our cloud-init part affect it somehow ? | 12:59 |
| sshnaidm|rover | quiquell, we can remove cloud init before job starts | 13:01 |
| sshnaidm|rover | quiquell, in some pre playbook | 13:01 |
| quiquell | sshnaidm|rover: I am running now towards an image that just injected my key over upstream image, let's see if it passes | 13:01 |
| sshnaidm|rover | quiquell, I wouldn't expect it will break, but who knows.. | 13:01 |
| quiquell | sshnaidm|rover: /dev/log is there | 13:03 |
| quiquell | sshnaidm|rover: in the image that I just inject pub key | 13:03 |
| weshay | cool.. thanks guys | 13:03 |
| sshnaidm|rover | quiquell, amazing.. | 13:03 |
| quiquell | sshnaidm|rover: does it make sense ? | 13:03 |
| sshnaidm|rover | quiquell, well, seems like cloud init breaks it.. should we create /dev/log in cloud init script? :) | 13:04 |
| quiquell | weshay: btw should we start looking at zuul-runner from zuul guys ? | 13:04 |
| quiquell | sshnaidm|rover: no clue... | 13:04 |
| *** ykarel is now known as ykarel|afk | 13:05 | |
| weshay | quiquell I want to watch it, think about it.. yes | 13:05 |
| quiquell | weshay: ack | 13:05 |
| weshay | however I am very skeptical it's designed for what we need | 13:05 |
| weshay | would like to hear others thoughts | 13:06 |
| quiquell | weshay: let's see how it works | 13:06 |
| sshnaidm|rover | quiquell, where is zuul-runner docs? | 13:06 |
| weshay | did it merge? | 13:06 |
| quiquell | sshnaidm|rover: there is only a WIP topic at gerrit freeze_jobs I think | 13:06 |
| quiquell | sshnaidm|rover: TristanC told me we can start to play around with it | 13:07 |
| sshnaidm|rover | quiquell, but what is it? | 13:07 |
| quiquell | sshnaidm|rover: https://tree.taiga.io/project/morucci-software-factory/epic/2188 | 13:07 |
| quiquell | sshnaidm|rover: They have put zuul executor upside down so it spit out some ansible playbooks to run jobs (I think) | 13:08 |
| *** aakarsh has quit IRC | 13:12 | |
| quiquell | rfolco: pushing fedora28 stein containers to docker hub | 13:13 |
| quiquell | rfolco: Cam I unblock https://tree.taiga.io/project/tripleo-ci-board/task/927?kanban-status=1447274 ? | 13:14 |
| rfolco | quiquell, \o/ | 13:14 |
| rfolco | quiquell, indeed | 13:15 |
| *** vinaykns has joined #oooq | 13:18 | |
| *** amoralej|lunch is now known as amoralej | 13:18 | |
| *** irclogbot_0 has joined #oooq | 13:25 | |
| weshay | marios_|ruck can you please follow up to make queens scen01/02 voting/gating | 13:31 |
| weshay | on top of that patch | 13:31 |
| marios_|ruck | ack weshay | 13:31 |
| weshay | thanks | 13:31 |
| *** aakarsh has joined #oooq | 13:33 | |
| weshay | quiquell sshnaidm|rover do you guys have the gerrit change avail for the zuul_runner? | 13:39 |
| quiquell | weshay: https://review.openstack.org/#/q/topic:freeze_job | 13:40 |
| marios_|ruck | weshay: so voting only for queens? https://github.com/openstack-infra/tripleo-ci/blob/3a7a87ce71fcd03252a3cb19ad1bd374f929dd9e/zuul.d/multinode-jobs.yaml#L134 | 13:42 |
| weshay | quiquell dang.. thanks.. /me looking for a spec or blueprint | 13:42 |
| quiquell | weshay: what do we do with this ? https://tree.taiga.io/project/tripleo-ci-board/task/860?kanban-status=1447275 | 13:43 |
| marios_|ruck | weshay: in which cas will need -queens job definition . or we want it voting for rocky and pike too | 13:43 |
| quiquell | weshay: TL;DR promoter changes for fedora/stein | 13:43 |
| weshay | marios_|ruck multinode should vote on pike/queens | 13:43 |
| weshay | but rocky/stein/master have standalone | 13:44 |
| weshay | quiquell let's chat for a sec | 13:44 |
| marios_|ruck | weshay: right ack. so we'll need to likely have a new job definition np we can tlk on scrum in 14 anyway | 13:44 |
| weshay | we have a couple topics | 13:44 |
| weshay | quiquell https://bluejeans.com/u/whayutin/ | 13:44 |
| marios_|ruck | weshay: well no rocky standalone yet | 13:45 |
| marios_|ruck | weshay: so maybe multinode voting on rocky also | 13:45 |
| *** ykarel|afk is now known as ykarel | 13:45 | |
| weshay | ah tru | 13:45 |
| marios_|ruck | stein/master for standalone. | 13:45 |
| marios_|ruck | which makes life easier wrt https://github.com/openstack-infra/tripleo-ci/blob/master/zuul.d/multinode-jobs.yaml#L134 | 13:45 |
| marios_|ruck | weshay: k posting in a sec thanks (which ones? all the scenarios? 1/2/3/4?) | 13:45 |
| marios_|ruck | weshay: nm you're on a call ttyl | 13:46 |
| weshay | quiquell http://lists.zuul-ci.org/pipermail/zuul-discuss/2019-March/thread.html | 13:52 |
| quiquell | weshay: http://lists.zuul-ci.org/pipermail/zuul-discuss/2019-March/000817.html | 13:52 |
| weshay | quiquell https://k3s.io/ | 13:53 |
| marios_|ruck | weshay: like that? https://review.openstack.org/649035 ? | 13:59 |
| rfolco | pinglist: marios, quiquell, sshnaidm, weshay, panda, rlandy, arxcruz, mwhahaha, rfolco, chkumar, ssbarnea, kopecmartin | 14:00 |
| rfolco | ping scrum | 14:01 |
| marios_|ruck | * run reproducer standalone centos rdo cloud: centos7 vm yum update first & reboot - but still hitting equalto issue http://paste.openstack.org/show/748594/ & rerun clears (wtf) brief irc weshay, finally fails again like http://paste.openstack.org/show/748596/ like last time 500 error ... revisit next week | 14:03 |
| *** mjturek has joined #oooq | 14:03 | |
| rfolco | weshay, scrum? | 14:04 |
| weshay | sorry.. joining now | 14:04 |
| weshay | had to chat w/ toure | 14:04 |
| kopecmartin | arxcruz, chandankumar could you, please, have a look? https://review.openstack.org/#/c/647415/ https://review.openstack.org/#/c/647696/ | 14:22 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039 @ https://review.openstack.org/602248, master: tripleo-ci-centos-7 (2 more messages) | 14:30 |
| marios_|ruck | rfolco && * anyone else interested this is what i was referring to by 'defaults to master if it can't find the override' : https://zuul-ci.org/docs/zuul/user/config.html#attr-project.default-branch https://zuul-ci.org/docs/zuul/user/config.html#attr-job.override-checkout | 14:36 |
| marios_|ruck | quiquell: ^ | 14:36 |
| *** ykarel is now known as ykarel|away | 14:37 | |
| *** derekh has quit IRC | 14:40 | |
| *** derekh has joined #oooq | 14:40 | |
| quiquell | sshnaidm|rover, weshay: standalone at reproducer working with an image just adding pub key | 14:41 |
| quiquell | sshnaidm|rover, weshay: So it has to be related to cloud-init | 14:41 |
| *** ykarel|away has quit IRC | 14:42 | |
| kopecmartin | arxcruz, thank you | 14:43 |
| arxcruz | kopecmartin: it's 1 beer per review :P | 14:51 |
| *** mjturek has quit IRC | 14:51 | |
| kopecmartin | arxcruz, ok, I'll increase your beer counter :) | 14:51 |
| *** mjturek has joined #oooq | 14:54 | |
| sshnaidm|rover | quiquell, fyi https://review.rdoproject.org/r/#/c/19352/ | 14:54 |
| quiquell | sshnaidm|rover \o/ !! | 14:55 |
| quiquell | sshnaidm|rover: btw how do we reset cloud-init or the like ? | 14:55 |
| quiquell | sshnaidm|rover: just remove package ? | 14:55 |
| sshnaidm|rover | quiquell, I'll test it.. maybe need to remove, or maybe just to restart syslogs | 14:55 |
| sshnaidm|rover | *syslogd | 14:55 |
| quiquell | sshnaidm|rover: maybe at "base" job | 14:55 |
| quiquell | sshnaidm|rover: in a pre so it get run the first | 14:55 |
| sshnaidm|rover | quiquell, maybe we have a way to restart am image before the job? would be cool.. | 14:56 |
| quiquell | sshnaidm|rover: what do you mean with restart ? | 14:56 |
| sshnaidm|rover | quiquell, reboot | 14:56 |
| sshnaidm|rover | quiquell, but zuul won't like it obviously.. | 14:57 |
| quiquell | sshnaidm|rover: maybe just restart syslog is enough or rsyslog or whatever is mounting /dev/log/ | 14:58 |
| sshnaidm|rover | quiquell, yeah, restart syslog helps, tried it | 14:58 |
| quiquell | Going to do it manually in the failing image to check it out | 14:58 |
| sshnaidm|rover | but who knows what else could be broken, how is it related at all.. | 14:58 |
| quiquell | maybe is not bad thing to remove cloud-init after all | 14:59 |
| weshay | sshnaidm|rover I think there is a bug w/ collect logs ovb | 15:04 |
| weshay | afaict.. we're deleting the stack right after tempest | 15:05 |
| weshay | we need to run tempest, collect logs, then delete the stack | 15:05 |
| weshay | or we won't get the overcloud nodes | 15:05 |
| weshay | sshnaidm|rover hrm.. although that's what you have here https://review.rdoproject.org/r/#/c/19812/4/playbooks/tripleo-rdo-base/post-ovb-delete.yaml | 15:06 |
| rlandy | rfolco: pls list reviews/card you want me to QE | 15:06 |
| weshay | marios_|ruck I'm in | 15:06 |
| weshay | sshnaidm|rover oh sorry.. two hours old | 15:07 |
| weshay | sshnaidm|rover have a test job on https://review.rdoproject.org/r/#/c/19812/ ? | 15:07 |
| quiquell | weshay: for fedora/stein "#887Create images and push to stein " is done ? | 15:08 |
| weshay | quiquell yes | 15:08 |
| quiquell | ack | 15:08 |
| *** Vorrtex has joined #oooq | 15:09 | |
| quiquell | sshnaidm|rover: /dev/log is there even with the failing image :-/ | 15:13 |
| quiquell | sshnaidm|rover: now I don't know | 15:13 |
| quiquell | sshnaidm|rover: At least we now that just pushing public key at the image make it works | 15:13 |
| quiquell | more tomorrow | 15:13 |
| quiquell | read you tomorrow guys | 15:13 |
| sshnaidm|rover | weshay, new jobs should be fine | 15:14 |
| weshay | sshnaidm|rover k | 15:14 |
| *** ykarel|away has joined #oooq | 15:15 | |
| *** ykarel|away is now known as ykarel | 15:15 | |
| *** quiquell is now known as quiquell|off | 15:20 | |
| *** ykarel is now known as ykarel|away | 15:40 | |
| *** dtantsur is now known as dtantsur|afk | 16:07 | |
| sshnaidm|rover | rlandy, just reviewed, we don't need anything there: https://review.rdoproject.org/r/#/c/19580/ | 16:13 |
| sshnaidm|rover | rlandy, we need only tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-stein-branch and it's already there | 16:14 |
| rlandy | rfolco: ^^ do you agree with that?? | 16:15 |
| marios_|ruck | sshnaidm|rover: /me almost out but weshay lets merge that https://review.openstack.org/#/c/648749/ | 16:15 |
| rlandy | + vexxhost | 16:16 |
| rfolco | rlandy, looking | 16:16 |
| sshnaidm|rover | marios_|ruck, there is some alternative in comments | 16:16 |
| sshnaidm|rover | rfolco, rlandy we can change rocky by stein in vexxhost, but only after stein is passing fine | 16:17 |
| marios_|ruck | sshnaidm|rover: yeah see ' | 16:17 |
| marios_|ruck | sshnaidm|rover: tripleo | 16:17 |
| marios_|ruck | and comment 17 in https://bugs.launchpad.net/tripleo/+bug/1822080 weshay sshnaidm|rover | 16:17 |
| openstack | Launchpad bug 1822080 in tripleo "periodic centos-7-ovb-1ctlr_1comp-featureset020-queens failing tempest "Endpoint not found"" [Critical,Triaged] - Assigned to Martin Kopec (mkopec) | 16:17 |
| *** bogdando has quit IRC | 16:19 | |
| rlandy | sshnaidm|rover: it's just a job definition | 16:20 |
| sshnaidm|rover | rlandy, ok, then fine | 16:20 |
| rlandy | sshnaidm|rover: secrets would not be available from playbooks kicked fro, toci, right? | 16:22 |
| rlandy | from toci | 16:22 |
| marios_|ruck | weshay: this is what you asked for before? make scenarios vote in pike/queens/master posted https://review.openstack.org/649035 | 16:22 |
| sshnaidm|rover | rlandy, no, secrets are available only in roles that are defined in config/ | 16:23 |
| *** dsneddon has quit IRC | 16:24 | |
| sshnaidm|rover | rlandy, what is the issue? | 16:24 |
| rlandy | sshnaidm|rover: I am helping the pidone team move their hardware tests to internal zuul | 16:24 |
| * weshay looks | 16:24 | |
| rlandy | they have an access key I would like to place in secrets | 16:24 |
| rlandy | but really the virthost has to be added within toci | 16:25 |
| rlandy | we can get around it though | 16:25 |
| sshnaidm|rover | rlandy, with internal it's easier.. | 16:25 |
| sshnaidm|rover | rlandy, you can expose creds there, nobody sees this anyway | 16:25 |
| rlandy | sshnaidm|rover: yeah - I can just ask them to add it to tripleo-environments | 16:25 |
| rfolco | rlandy, tasks 869 871, added you as QE | 16:27 |
| rlandy | thanks | 16:27 |
| marios_|ruck | ttyl folks | 16:27 |
| weshay | marios_|ruck /me takging notes | 16:27 |
| weshay | https://etherpad.openstack.org/p/scenario001-004_stable_branch_voting | 16:27 |
| weshay | ttyl | 16:27 |
| marios_|ruck | weshay: looking | 16:27 |
| marios_|ruck | weshay: k will check tomorrow morning | 16:28 |
| rfolco | rlandy, I don't know why we needed those branchful jobs, I'll take sagi's review as reference | 16:28 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039 @ https://review.openstack.org/602248, master: tripleo-ci-centos-7 (2 more messages) | 16:30 |
| *** kopecmartin is now known as kopecmartin|off | 16:30 | |
| *** marios has quit IRC | 16:30 | |
| *** marios_|ruck has quit IRC | 16:30 | |
| rlandy | thanks | 16:37 |
| *** dsneddon has joined #oooq | 16:49 | |
| *** trown is now known as trown|lunch | 16:49 | |
| *** holser_ has quit IRC | 16:53 | |
| *** holser_ has joined #oooq | 16:54 | |
| *** derekh has quit IRC | 17:02 | |
| *** jpena is now known as jpena|off | 17:04 | |
| *** holser_ has quit IRC | 17:05 | |
| *** gkadam has quit IRC | 17:05 | |
| *** holser_ has joined #oooq | 17:05 | |
| *** dsneddon has quit IRC | 17:06 | |
| *** holser_ has quit IRC | 17:12 | |
| *** ykarel|away has quit IRC | 17:20 | |
| *** holser_ has joined #oooq | 17:29 | |
| *** holser_ has quit IRC | 17:31 | |
| *** dsneddon has joined #oooq | 17:47 | |
| *** trown|lunch is now known as trown | 18:04 | |
| rlandy | rfolco: so we ditch this entire review: https://review.rdoproject.org/r/#/c/19580/? | 18:08 |
| rfolco | rlandy, maybe we need to ask sshnaidm|rover if there is job in 3rd party that we DO NOT want to run on stein. If not, 3rd party stein branching is no-op change, ready. | 18:09 |
| rfolco | if the branches regex do not block stable/stein, we're good. | 18:10 |
| rlandy | rfolco: k - let me know | 18:12 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039 @ https://review.openstack.org/602248, master: tripleo-ci-centos-7 (2 more messages) | 18:30 |
| *** jfrancoa has quit IRC | 18:42 | |
| *** matbu has quit IRC | 18:51 | |
| *** matbu has joined #oooq | 18:53 | |
| *** rascasoft has quit IRC | 19:32 | |
| *** rascasoft has joined #oooq | 19:34 | |
| *** rlandy is now known as rlandy|brb | 19:36 | |
| *** rlandy|brb is now known as rlandy | 19:54 | |
| rlandy | weshay: we've started ... we have zuul access to pidone hw ... https://sf.hosted.upshift.rdu2.redhat.com/logs/13/166613/18/check/test-virthost/2a044ee/job-output.txt.gz | 19:55 |
| weshay | ah nice | 19:56 |
| rlandy | parented off upstream | 19:56 |
| * rlandy switches to qe mode | 19:57 | |
| *** matbu has quit IRC | 19:57 | |
| *** dsneddon has quit IRC | 20:03 | |
| *** dsneddon has joined #oooq | 20:21 | |
| *** dsneddon has quit IRC | 20:27 | |
| *** dsneddon has joined #oooq | 20:29 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039 @ (2 more messages) | 20:30 |
| *** dsneddon has quit IRC | 20:34 | |
| rlandy | weshay: should we be joining this card https://trello.com/c/NTQUJeYC/25-improve-test-effectiveness-using-top-customer-scenarios? | 20:42 |
| *** trown is now known as trown|outtypewww | 20:42 | |
| rlandy | due to apevec's comment - linking bm | 20:42 |
| *** holser_ has joined #oooq | 20:43 | |
| weshay | not sure tbh.. I'm now on the watch list | 20:43 |
| rlandy | weshay: well then watch it for me pls | 20:43 |
| rfolco | rlandy, bm jobs stein, replicate the rocky ones or all bm master ones ? | 20:43 |
| rlandy | rfolco; neither | 20:44 |
| rfolco | :( | 20:44 |
| rlandy | add a stein job - use env B | 20:44 |
| rlandy | leave job in review until we have hw to test | 20:44 |
| rfolco | ah like envE has rocky job ? | 20:44 |
| rlandy | correct | 20:44 |
| rfolco | rlandy, ^ | 20:44 |
| rfolco | k | 20:44 |
| rfolco | thx | 20:44 |
| rlandy | whatever atm - we won't know until we get our hw back | 20:45 |
| *** mjturek has quit IRC | 20:45 | |
| *** matbu has joined #oooq | 20:46 | |
| *** dsneddon has joined #oooq | 20:48 | |
| rlandy | weshay: I am not on the board so I can't join the card but I'd like to so we can follow what is planned | 20:48 |
| rlandy | nvm I see the join board button | 20:49 |
| rlandy | rfolco:enough QE info to move https://tree.taiga.io/project/tripleo-ci-board/task/869?kanban-status=1538166 to done? | 21:25 |
| rlandy | rfolco:see last comment | 21:25 |
| rfolco | rlandy, if you verify that all jobs added are running in periodic, yes, done | 21:27 |
| rlandy | rfolco; lost link to container registry rdo - can you paste | 21:37 |
| *** ccamacho has quit IRC | 21:43 | |
| rfolco | rlandy, http://logs.rdoproject.org/19/19819/9/check/periodic-tripleo-fedora-28-stein-containers-build-push/cbeb6ad/conf/kolla-build.conf.txt.gz | 21:47 |
| rfolco | registry=trunk.registry.rdoproject.org | 21:48 |
| rlandy | yeah - know that piece | 21:49 |
| rlandy | rfolco: no stein promoter logs yet? http://38.145.34.55/ | 21:49 |
| *** holser_ has quit IRC | 21:50 | |
| rfolco | rlandy, isn't this the work quiquell|off is doing now? | 21:50 |
| rlandy | rfolco: just QE'ing https://tree.taiga.io/project/tripleo-ci-board/task/871?kanban-status=1538166 | 21:51 |
| rlandy | hard to tell if promoter stuff added is working w/o logs | 21:51 |
| rfolco | rlandy, good point | 21:52 |
| rlandy | rfolco; can confirm review is merged but not much more | 21:53 |
| rlandy | will leave notes and check back tomorrow | 21:53 |
| rfolco | ok, will take a look at this first thing tomorrow | 21:53 |
| rlandy | weshay; if you have a few minutes to look at the rhos job failure with me ... https://sf.hosted.upshift.rdu2.redhat.com/logs/74/166374/3/check/tripleo-ci-rhel-7-standalone-rhos-14/1433418/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz | 21:59 |
| weshay | sure.. need 1 min | 21:59 |
| weshay | rlandy bluejeans? | 22:03 |
| weshay | I think I have an idea | 22:03 |
| rlandy | yep - joining | 22:03 |
| *** holser_ has joined #oooq | 22:03 | |
| *** aakarsh has quit IRC | 22:14 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039 @ (2 more messages) | 22:30 |
| *** holser_ has quit IRC | 22:38 | |
| *** vinaykns has quit IRC | 22:57 | |
| *** aakarsh has joined #oooq | 23:01 | |
| *** tosky has quit IRC | 23:28 | |
| *** Vorrtex has quit IRC | 23:46 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!