| *** sshnaidm|afk has quit IRC | 00:04 | |
| *** sshnaidm has joined #oooq | 00:06 | |
| *** panda|rover has quit IRC | 00:10 | |
| *** panda has joined #oooq | 00:11 | |
| *** Vorrtex has quit IRC | 00:33 | |
| *** ykarel|away has joined #oooq | 02:29 | |
| *** ykarel|away is now known as ykarel | 02:44 | |
| *** saneax has quit IRC | 03:09 | |
| *** udesale has joined #oooq | 03:47 | |
| *** yolanda_ has quit IRC | 03:49 | |
| *** ratailor has joined #oooq | 04:46 | |
| *** ykarel is now known as ykarel|away | 05:03 | |
| *** raukadah is now known as chandankumar | 05:06 | |
| *** ykarel|away has quit IRC | 05:08 | |
| *** marios_|ruck has joined #oooq | 05:16 | |
| *** marios has joined #oooq | 05:16 | |
| *** marios_|ruck has quit IRC | 05:17 | |
| *** marios_|ruck has joined #oooq | 05:18 | |
| *** ykarel|away has joined #oooq | 05:20 | |
| chandankumar | marios_|ruck: Good morning | 05:21 |
|---|---|---|
| chandankumar | marios_|ruck: for rhel8 standalone tempest-> cirros image cache https://code.engineering.redhat.com/gerrit/#/c/170843/ and https://review.opendev.org/#/c/660246/ -> tempest image fs overrirde | 05:22 |
| marios_|ruck | chandankumar: morning thanks will check in bit | 05:25 |
| *** ykarel|away is now known as ykarel | 05:28 | |
| ykarel | marios_|ruck, can you also check comment in https://review.opendev.org/#/c/660150/ | 05:28 |
| ykarel | and i also reverted https://review.rdoproject.org/r/#/c/20859/ | 05:29 |
| ykarel | for more context, just forwarded mail to you | 05:30 |
| marios_|ruck | ykarel: thanks | 05:30 |
| ykarel | and testing blocking ansible 2.8, and seems my guess is correct | 05:31 |
| chandankumar | marios_|ruck: holy shit https://sf.hosted.upshift.rdu2.redhat.com/logs/40/167540/34/check/tripleo-ci-rhel-8-standalone-rhos-15/e101979/logs/tempest.html.gz | 05:32 |
| chandankumar | marios_|ruck: it passed | 05:32 |
| chandankumar | now time to make cirros image cache nodepool part | 05:32 |
| chandankumar | sorry wrong language | 05:33 |
| marios_|ruck | chandankumar: cool :D | 05:33 |
| chandankumar | is it possible to test sf-config changes with tripleo-ci internal jobs? | 05:33 |
| chandankumar | marios_|ruck: I have one more idea, we can create two standalone jobs | 05:34 |
| chandankumar | one which will test cirros image and another rhel 8 image during tempest run | 05:34 |
| chandankumar | as customers will be spinning RHEL 8 instances so | 05:34 |
| chandankumar | in the cloud | 05:35 |
| *** quiquell has joined #oooq | 05:38 | |
| marios_|ruck | chandankumar: no if you mean cos its a trusted project? | 05:38 |
| *** quiquell has quit IRC | 05:38 | |
| marios_|ruck | chandankumar: you have to wait for it to merge before you can run it (sf-config... i *think* is trusted)( | 05:38 |
| chandankumar | marios_|ruck: no idea, I donot like the idea of trusted project, | 05:39 |
| marios_|ruck | chandankumar: :D | 05:39 |
| chandankumar | nothing can be trusted everything needs to be tested in order to make it trusted | 05:39 |
| *** quiquell has joined #oooq | 05:39 | |
| *** ykarel_ has joined #oooq | 05:42 | |
| *** udesale has quit IRC | 05:44 | |
| *** ykarel has quit IRC | 05:44 | |
| *** udesale has joined #oooq | 05:44 | |
| *** saneax has joined #oooq | 05:51 | |
| *** ykarel__ has joined #oooq | 06:01 | |
| *** ykarel_ has quit IRC | 06:03 | |
| *** ykarel__ is now known as ykarel | 06:04 | |
| *** zbr_ has quit IRC | 06:05 | |
| *** skramaja has joined #oooq | 06:25 | |
| *** holser_ has joined #oooq | 06:33 | |
| *** jfrancoa has joined #oooq | 06:45 | |
| *** ccamacho has joined #oooq | 06:56 | |
| *** tesseract has joined #oooq | 07:06 | |
| marios_|ruck | chandankumar: any idea why wes set that to fix-released? which patches i mean? https://bugs.launchpad.net/tripleo/+bug/1828860 | 07:11 |
| openstack | Launchpad bug 1828860 in tripleo "queens, tempest.api.volume multiple failures" [Critical,Fix released] - Assigned to chandan kumar (chkumar246) | 07:11 |
| chandankumar | marios_|ruck: the jobs started passing from yesterday | 07:12 |
| chandankumar | no idea what changes entered into the system | 07:12 |
| *** kopecmartin|off is now known as kopecmartin | 07:12 | |
| marios_|ruck | chandankumar: k thx | 07:12 |
| chandankumar | marios_|ruck: we need to fix the logserver logging | 07:13 |
| chandankumar | https://review.rdoproject.org/r/#/c/20840/ needs to be merged | 07:14 |
| marios_|ruck | chandankumar: looks like apevec just did | 07:17 |
| chandankumar | marios_|ruck: yes thanks :-) | 07:21 |
| *** apetrich has joined #oooq | 07:28 | |
| chandankumar | panda: Hello, needs +2 and +w on https://review.opendev.org/#/c/660246/ for rhel-8 tempest proof: https://sf.hosted.upshift.rdu2.redhat.com/logs/40/167540/34/check/tripleo-ci-rhel-8-standalone-rhos-15/e101979/logs/tempest.html.gz | 07:29 |
| chandankumar | panda: thanks :-) | 07:29 |
| *** amoralej|off is now known as amoralej | 07:35 | |
| *** ykarel is now known as ykarel|lunch | 07:35 | |
| *** udesale has quit IRC | 07:35 | |
| *** udesale has joined #oooq | 07:36 | |
| *** tosky has joined #oooq | 07:41 | |
| *** apahim_ has joined #oooq | 07:53 | |
| *** jpena|off is now known as jpena | 07:56 | |
| *** bogdando has joined #oooq | 08:03 | |
| *** skramaja_ has joined #oooq | 08:05 | |
| *** skramaja has quit IRC | 08:07 | |
| Tengu | hello there! quick question: would the CI consider adding a job for molecule tests? I'm thinking about adding molecule unit testing to the tripleo-validations (https://molecule.readthedocs.io/en/stable/) | 08:13 |
| Tengu | I'm pretty sure this will add value to the validations, and make possible to actually CI-test the whole work. | 08:13 |
| marios_|ruck | Tengu: you should talk to soren zbr he is away still but has some wip on that | 08:18 |
| * marios_|ruck going for food biab | 08:18 | |
| marios_|ruck | Tengu: i can find you pointers when i'm back if you dont find in meantime | 08:18 |
| *** dtantsur|afk is now known as dtantsur | 08:20 | |
| Tengu | marios_|ruck: great, thanks! I'm currently reading molecule doc. | 08:20 |
| *** d0ugal has quit IRC | 08:21 | |
| *** ykarel|lunch is now known as ykarel | 08:30 | |
| *** derekh has joined #oooq | 08:43 | |
| *** d0ugal has joined #oooq | 08:56 | |
| panda | ykarel: sorry for the ignorance what does the buildsys-tag do ? | 09:05 |
| ykarel | panda, it pushes the build to delorean deps repo | 09:05 |
| *** apahim_ has quit IRC | 09:05 | |
| ykarel | package build | 09:05 |
| *** dtantsur is now known as dtantsur|brb | 09:06 | |
| panda | ykarel: thanks | 09:06 |
| chandankumar | marios_|ruck: Hello | 09:06 |
| chandankumar | marios_|ruck: since rhel-8 standalone base is working fine, do we want to move other standalone scenario jobs or we wait for weshay to come online for RHEL8 and RHOS15? | 09:07 |
| panda | ykarel: updating to ansible 2.8 is a definitive no-no for us. It removes support for the roles: keyword, that we have still pretty much everywhere. | 09:07 |
| ykarel | panda, no no for quickstart or tripleo? | 09:08 |
| panda | ykarel: quickstart for sure, for tripleo I'd check the playbooks first | 09:10 |
| panda | if I can find where they are ... | 09:10 |
| *** marios has quit IRC | 09:11 | |
| *** marios_|ruck has quit IRC | 09:11 | |
| ykarel | panda, but quickstart uses venv for ansible | 09:11 |
| ykarel | and there it must be what it's defined in requirements.txt | 09:11 |
| panda | ykarel: yes, do you know where the tripleo playbooks are ? | 09:11 |
| *** marios_|ruck has joined #oooq | 09:11 | |
| marios_|ruck | chandankumar: ack except it isn't really 'fine' | 09:12 |
| marios_|ruck | chandankumar: the depends on are hacks | 09:12 |
| marios_|ruck | chandankumar: which we need to tidy up today work out how to carry the rest | 09:12 |
| *** marios has joined #oooq | 09:12 | |
| ykarel | panda, they are distributed, one u can find in tht/common/deploy-tasks.yml | 09:12 |
| marios_|ruck | chandankumar: fantastic we got the green run but we can't merge stuff as is | 09:13 |
| ykarel | panda, for this particual issue(container build job which mixes ansible package and ansible zuul executor) we just reverted ansible 2.8 bump in Train, atleast some TripleO standalone/multinode jobs passed with ansible 2.8 | 09:13 |
| ykarel | marios_|ruck, ^^ | 09:13 |
| ykarel | https://review.rdoproject.org/r/#/q/topic:ansible28+(status:open+OR+status:merged) | 09:13 |
| chandankumar | marios_|ruck: let's hope cirros image nodepool also gets merged by evening | 09:13 |
| *** openstackstatus has quit IRC | 09:13 | |
| *** openstackstatus has joined #oooq | 09:14 | |
| *** ChanServ sets mode: +v openstackstatus | 09:14 | |
| marios_|ruck | chandankumar: ack# | 09:14 |
| ykarel | panda, mainly the playbooks are build on the fly | 09:15 |
| ykarel | you can see those in jobs logs | 09:15 |
| panda | ykarel: ... I see some roles: still | 09:16 |
| ykarel | panda, so only two possibilities: those roles executed and passed or not executed at all | 09:17 |
| ykarel | https://review.rdoproject.org/r/#/c/20755/ | 09:18 |
| ykarel | ceph-ansible also passed | 09:19 |
| panda | ykarel: mmhh, re reading documentation, I think I have outdated information. There is no mention anymore on the removal of roles: . Maybe they changed their plans. | 09:22 |
| ykarel | hmm may be, /me not aware of any such plans though | 09:24 |
| panda | marios_|ruck: what are you working on ? | 09:29 |
| marios_|ruck | panda: looking at https://bugs.launchpad.net/tripleo/+bug/1829736 but about to switch to the rhel8 | 09:30 |
| openstack | Launchpad bug 1829736 in tripleo "periodic-tripleo--master-containers-build is failing with "Failed to import docker-py - cannot import name __version__"" [Critical,In progress] | 09:30 |
| marios_|ruck | panda: what do you need | 09:30 |
| marios_|ruck | panda: things look 'calm' in the sense i haven't filed new bugs today | 09:31 |
| marios_|ruck | panda: up to date on https://etherpad.openstack.org/p/ruckroversprint10 and less on fire with the warlock thing gone | 09:31 |
| marios_|ruck | panda: whatsup | 09:31 |
| *** skramaja has joined #oooq | 09:42 | |
| marios_|ruck | chandankumar: can you check comment #5 when you can please https://bugs.launchpad.net/tripleo/+bug/1828859 still seeing the same in the periodic run today | 09:42 |
| openstack | Launchpad bug 1828859 in tripleo "[queens] Unhandled error: NoOptionError: No option 'tempest_roles' in section: 'auth'" [Critical,Fix released] - Assigned to chandan kumar (chkumar246) | 09:42 |
| *** skramaja_ has quit IRC | 09:43 | |
| marios_|ruck | ykarel: so that should fix us for now https://review.rdoproject.org/r/#/c/20860/ periodics containers build i mean | 09:48 |
| marios_|ruck | ykarel: the ansible 2.8 revert | 09:48 |
| ykarel | marios_|ruck, looking | 09:48 |
| ykarel | the repo | 09:48 |
| *** holser_ is now known as holser|lunch | 09:49 | |
| ykarel | we also need to remove that from repo | 09:49 |
| ykarel | amoralej, can you clear it ^^ | 09:49 |
| amoralej | yes | 09:49 |
| amoralej | give me a while | 09:49 |
| marios_|ruck | ykarel: you mean ansible 2.8 from the repo | 09:49 |
| marios_|ruck | ykarel: ok thanks amoralej let us know if we need to do something to help | 09:50 |
| ykarel | marios_|ruck, yes, | 09:50 |
| marios_|ruck | ok thanks ykarel amoralej | 09:50 |
| amoralej | marios_|ruck, ykarel done | 10:01 |
| ykarel | ack | 10:01 |
| amoralej | not it should fetch 2.7 | 10:01 |
| marios_|ruck | amoralej: thanks | 10:01 |
| ykarel | yes, trired locally, ansible-2.7 is pulled | 10:02 |
| amoralej | ykarel, it only works with ansible-2.8.0 in the venv | 10:08 |
| amoralej | fails with 2.5, 2.6 and 2.7 | 10:09 |
| amoralej | maybe we could uninstall ansible before running the docker_login ? | 10:09 |
| amoralej | i guess we don't need it? | 10:10 |
| amoralej | an that stage | 10:10 |
| ykarel | amoralej, ack, /me also noticed the same venv and package part | 10:10 |
| ykarel | and yes ansible package should not be needed before docker_login | 10:11 |
| amoralej | i think the problem is related to PYTHONPATH and versions mess | 10:12 |
| amoralej | when tries to import docker | 10:12 |
| amoralej | instead of using /usr/lib/python2.7/site-packages/docker | 10:12 |
| amoralej | it loads /usr/lib/python2.7/site-packages/ansible/module_utils/docker/ | 10:12 |
| chandankumar | marios_|ruck: How can I know here https://logs.rdoproject.org/openstack-periodic-24hr/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset017-queens/d657319/job-output.txt.gz at which commit tqe master is clonned? | 10:20 |
| ykarel | amoralej, hmm, with earlier version i see docker_common and with 2.8 it's docker under module_utils | 10:23 |
| amoralej | yeah, i think that's the issue | 10:24 |
| amoralej | together with some change in load order or something in 2.8 | 10:24 |
| amoralej | that's why it works with 2.8 i guess | 10:24 |
| marios_|ruck | chandankumar: can see some info there ? https://logs.rdoproject.org/openstack-periodic-24hr/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset017-queens/d657319/logs/zuul-variables.yaml | 10:24 |
| ykarel | amoralej, https://github.com/ansible/ansible/commit/0c2bb3da043e38e779f91b6037d0999c1e60b862#diff-5f3137ef16fa6f5a89b619f82d4e6c57 | 10:25 |
| marios_|ruck | chandankumar: for the commit not sure | 10:25 |
| marios_|ruck | chandankumar: so you think it need another run maybe? | 10:25 |
| marios_|ruck | chandankumar: can you add a comment at least on the bug for now? | 10:25 |
| chandankumar | marios_|ruck: yes | 10:25 |
| chandankumar | done! | 10:28 |
| marios_|ruck | panda: rfolco can you folks check out this https://code.engineering.redhat.com/gerrit/#/c/167407/ when rlandy and weshay come online i think we can try merge that one | 10:30 |
| ykarel | chandankumar, marios i see tempest_roles in fs017, shouldn't that be cleaned instead of the patch that merged? | 10:30 |
| *** dtantsur|brb is now known as dtantsur | 10:30 | |
| ykarel | chandankumar, marios_|ruck i mean https://opendev.org/openstack/tripleo-quickstart/src/branch/master/config/general_config/featureset017.yml#L171 | 10:31 |
| sshnaidm | ykarel, do you know if centos7 nodepool image comes with ansible installed? | 10:32 |
| marios_|ruck | ykarel: not sure question for chandankumar but i don't see any changes in the fs from reviews we can post one though unless chandankumar disagrees | 10:32 |
| amoralej | ykarel, re-tested, so we need removing ansible from the primary host before running the docker_login or get ansible updated to 2.8 in zuul executors | 10:33 |
| amoralej | or running the login as a nested playbook with local ansible | 10:33 |
| ykarel | amoralej, getting ansible 2.8 in zuul executor will take time, so 1st option seems valid for now | 10:33 |
| amoralej | as done in the build-containers | 10:33 |
| amoralej | but i'd say that's uggly | 10:33 |
| amoralej | yes, i think removing ansible from there is better | 10:33 |
| chandankumar | ykarel: https://logs.rdoproject.org/32/20832/3/check/periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset017-queens/a8164a6/logs/undercloud/home/zuul/tempest.log.txt.gz#_2019-05-20_10_58_06 | 10:34 |
| ykarel | sshnaidm, /me doubt on ansible package installed, but can check nodepool log to confirm | 10:34 |
| marios_|ruck | ykarel: wondering if that is used elsewhere other jobs i mean fs17 | 10:34 |
| marios_|ruck | ykarel: other than queens... only thing i could think of but i posted your comments on the bug just now anyway | 10:34 |
| chandankumar | ykarel: We need to figoure out which commit is tqe is pointing whne clonned in fs017 job | 10:34 |
| ykarel | auth.tempest_roles 'Member, creator' | 10:34 |
| ykarel | i see ^^ in fs017 file | 10:34 |
| chandankumar | ykarel: it is needed | 10:35 |
| marios_|ruck | ykarel: chandankumar there is also fs17 rocky/queens/pike | 10:35 |
| *** udesale has quit IRC | 10:46 | |
| *** holser|lunch is now known as holser_ | 10:55 | |
| *** amoralej is now known as amoralej|lunch | 11:01 | |
| ykarel | marios_|ruck, yes that job runs in other releases as well | 11:08 |
| ykarel | marios_|ruck, in short: more work is needed apart from the merged patch to resolve fs017-queens issue | 11:09 |
| sshnaidm | panda, marios_|ruck please review https://review.opendev.org/#/c/660138/ it's required for ovb internal jobs | 11:10 |
| marios_|ruck | sshnaidm: ack | 11:12 |
| marios_|ruck | ykarel: thanks | 11:12 |
| *** panda is now known as panda|rover|eat | 11:13 | |
| sshnaidm | marios_|ruck, for context: https://code.engineering.redhat.com/gerrit/#/c/170796/ | 11:27 |
| sshnaidm | marios_|ruck, it's in internal gerrit, I can't post it in upstream | 11:27 |
| weshay | marios_|ruck good afternoon | 11:31 |
| marios_|ruck | weshay: o/ | 11:32 |
| marios_|ruck | thanks sshnaidm | 11:32 |
| marios_|ruck | weshay: did you see the green job chandankumar++ (and obv rlandy++ )@ https://code.engineering.redhat.com/gerrit/#/c/167540/34 | 11:33 |
| marios_|ruck | weshay: but we need to tidy up the hacks | 11:33 |
| marios_|ruck | weshay: or work out where to carry them | 11:33 |
| marios_|ruck | weshay: been trying to do that a bit but also waiting to sync with rlandy and you (waiting to see if python_interpreter works in the config at https://code.engineering.redhat.com/gerrit/#/c/167407/9 now with recheck in the job | 11:34 |
| marios_|ruck | weshay: and updated rlandy's review at v9 @ https://review.opendev.org/660151 | 11:34 |
| marios_|ruck | currently looking at yours https://review.opendev.org/#/c/659879/5 | 11:34 |
| marios_|ruck | weshay: been commenting on them as i go | 11:34 |
| Tengu | yay! we get the common_function back in tripleo-ci \o/ | 11:35 |
| chandankumar | weshay: https://code.engineering.redhat.com/gerrit/#/c/170843/ needs +w on this for rhel 8 cirros imahe | 11:38 |
| chandankumar | once merge we can test it directly there | 11:38 |
| chandankumar | *image | 11:38 |
| weshay | chandankumar marios_|ruck ++ I sure did :) | 11:38 |
| weshay | marios_|ruck chandankumar now we just have to remove hacks | 11:38 |
| weshay | marios_|ruck you added python_interp to the release.. so we can remove https://review.opendev.org/#/c/660189/5 from the depends on and retry? | 11:39 |
| marios_|ruck | weshay: but it fails because i removed the become /nodes https://review.opendev.org/#/c/660151/9/playbooks/openstack-zuul-jobs/legacy/pre.yaml | 11:39 |
| weshay | chandankumar can you ask the packaging team what we should do re: the mock rpm for rhel8? | 11:41 |
| weshay | I don't see any builds | 11:41 |
| weshay | chandankumar perhaps we can build it and add it to dlrn-deps so we don't need epel | 11:41 |
| weshay | chandankumar context... we install epel -> install mock -> remove epel immediately | 11:42 |
| weshay | atm | 11:42 |
| weshay | marios_|ruck so we have to set the interp on the playbook w/ a when clause I guess? | 11:42 |
| weshay | marios_|ruck should we get on blue? | 11:43 |
| weshay | this is too slow for me | 11:43 |
| marios_|ruck | weshay: yah i think so | 11:43 |
| weshay | chandankumar join w/ us | 11:43 |
| weshay | https://bluejeans.com/u/whayutin/ | 11:44 |
| chandankumar | weshay: https://brewweb.engineering.redhat.com/brew/buildinfo?buildID=871801 | 11:44 |
| chandankumar | weshay: we can do something like this dnf -y install http://download.eng.bos.redhat.com/brewroot/vol/rhel-8/packages/mock/1.4.14/2.el8eng/noarch/mock-1.4.14-2.el8eng.noarch.rpm | 11:47 |
| *** marios_|ruck is now known as marios_|ruck|cal | 12:00 | |
| *** jpena is now known as jpena|lunch | 12:00 | |
| *** marios_|ruck|cal is now known as marios_|ruckcall | 12:00 | |
| *** marios_|ruckcall is now known as marios_|ruck | 12:01 | |
| marios_|ruck | weshay: chandankumar done v10 https://code.engineering.redhat.com/gerrit/#/c/167407/ | 12:02 |
| *** ratailor has quit IRC | 12:09 | |
| chandankumar | weshay: marios_|ruck we can get the mock from rcm-tools repo https://mojo.redhat.com/docs/DOC-1058262#jive_content_id_Distribution_and_availability | 12:11 |
| *** udesale has joined #oooq | 12:16 | |
| chandankumar | weshay: marios_|ruck https://code.engineering.redhat.com/gerrit/#/c/170843/ | 12:19 |
| marios_|ruck | chandankumar: https://tree.taiga.io/project/tripleo-ci-board/task/988?kanban-status=1447275 | 12:19 |
| weshay | marios_|ruck going to get you the starlink spaceX internet service to avoid the 1.5 second delay :) | 12:21 |
| marios_|ruck | weshay: ;) | 12:22 |
| weshay | radez is in our channel.. love that guy.. go punch toure for me | 12:22 |
| weshay | just cause | 12:22 |
| *** amoralej|lunch is now known as amoralej | 12:25 | |
| *** panda|rover|eat is now known as panda|rover | 12:27 | |
| chandankumar | marios_|ruck: please rebase this https://code.engineering.redhat.com/gerrit/#/c/167540/ | 12:29 |
| chandankumar | weshay: marios_|ruck updated the patch https://code.engineering.redhat.com/gerrit/167407 for mock | 12:29 |
| weshay | panda|rover would I be interupting you for a small request? http://logs.rdoproject.org/openstack-periodic-latest-released/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-7-standalone-upgrade-stein/aa9ce85/logs/undercloud/home/zuul/standalone_upgrade.log.txt.gz | 12:29 |
| rfolco | does anybody have a demo for today's community call ? weshay sshnaidm marios_|ruck arxcruz ? | 12:29 |
| weshay | panda|rover the tripleo-upgrade job in periodic is trying to pull containers from docker.io vs.. rdo reg | 12:30 |
| marios_|ruck | chandankumar: ack gimme few | 12:30 |
| weshay | should be a easy fix | 12:30 |
| arxcruz | rfolco: I can | 12:30 |
| weshay | 2019-05-21 10:52:50 | 2019-05-21 10:52:50.687 110948 ERROR tripleoclient.v1.tripleo_upgrade.Upgrade [-] Exception: Not found image: docker://docker.io/tripleostein/centos-binary-cinder-api:tripleo-ci-testing: ImageNotFoundException: Not found image: docker://docker.io/tripleostein/centos-binary-cinder-api:tripleo-ci-testing[00m | 12:30 |
| rfolco | arxcruz, nice! thank you | 12:30 |
| panda|rover | weshay: ok | 12:31 |
| weshay | panda|rover thanks man | 12:32 |
| sshnaidm | rfolco, mm.. not yet, I think firstly this patch should be merged https://review.rdoproject.org/r/#/c/20548/ | 12:32 |
| rfolco | sshnaidm, ack | 12:32 |
| rfolco | sshnaidm, but you don't necessarily need to talk about merged stuff. You can even go through your patch in the demo :) | 12:33 |
| rfolco | sshnaidm, arxcruz will make it today, please consider next week. | 12:33 |
| sshnaidm | rfolco, ack | 12:34 |
| chandankumar | marios_|ruck: card updated | 12:35 |
| *** rlandy has joined #oooq | 12:37 | |
| rlandy | weshay: hi ... saw https://code.engineering.redhat.com/gerrit/#/c/170843/ just merged. Was an image build job kicked as well? | 12:39 |
| rlandy | otherwise that only happens periodically | 12:39 |
| weshay | rlandy there is one more change to make.. I think marios_|ruck is updating the image so we don't have to create and chmod things like /etc/nodepool | 12:40 |
| rlandy | oh cool - thanks | 12:40 |
| weshay | rlandy if you can trying to work out install-deps for rhel-8 that would be pushing it forward | 12:40 |
| weshay | rlandy https://sf.hosted.upshift.rdu2.redhat.com/logs/40/167540/34/check/tripleo-ci-rhel-8-standalone-rhos-15/e101979/ | 12:40 |
| *** aakarsh has quit IRC | 12:40 | |
| weshay | check it out | 12:41 |
| marios_|ruck | weshay: rlandy here ... not sure if we want to create the /private and the rest of the files or just /etc nodepool https://code.engineering.redhat.com/gerrit/170920 | 12:42 |
| marios_|ruck | weshay: rlandy maybe worth adding before request image build | 12:42 |
| weshay | agree | 12:43 |
| *** Goneri has joined #oooq | 12:43 | |
| chandankumar | marios_|ruck: it needs rebase | 12:43 |
| weshay | marios_|ruck you sure that will have the right perm? | 12:43 |
| chandankumar | nodepool patch | 12:43 |
| marios_|ruck | chandankumar: ack that was quick :D | 12:43 |
| marios_|ruck | chandankumar: but gonna update now anyway thanks | 12:43 |
| marios_|ruck | weshay: no but i don't think we can set 777 :D | 12:44 |
| marios_|ruck | weshay: like https://review.opendev.org/#/c/660151/8..9/playbooks/openstack-zuul-jobs/legacy/pre.yaml | 12:44 |
| chandankumar | rlandy: I tried the rdo log upload from sf hosted https://code.engineering.redhat.com/gerrit/#/c/170519/7/zuul.d/standalone-jobs.yaml does nto worked | 12:44 |
| rlandy | chandankumar: k - I'll look at it later today | 12:46 |
| chandankumar | sure, thanks! | 12:46 |
| rlandy | marios_|ruck, really we do run the role to create all those dir | 12:46 |
| rlandy | the issue is the order we ran the roles in | 12:46 |
| marios_|ruck | rlandy: shouldn't hurt to have few mkdirs in https://code.engineering.redhat.com/gerrit/#/c/170920/1/nodepool/elements/virt-customize/rhel8-cloud.yaml if it means we can iterate quickly | 12:47 |
| marios_|ruck | rlandy: ? | 12:47 |
| marios_|ruck | rlandy: or you think not necessary | 12:47 |
| rlandy | marios_|ruck: well, imho, it should match the upstream-rhel-7 image as closely as possible | 12:47 |
| rlandy | but really I am not sure what is in that image :) | 12:48 |
| rlandy | the less rhel-8 specific changes the better | 12:48 |
| rlandy | either way the job inherits from upstream so the role will run | 12:48 |
| rlandy | marios_|ruck: do you know what was build in upstream-rhel-7? | 12:48 |
| rlandy | https://review.opendev.org/#/c/660151/9/zuul.d/base.yaml | 12:49 |
| rlandy | ^^ we are switching the order there | 12:49 |
| panda|rover | weshay: is there a bug to close for that standalone upgrade error ? | 12:49 |
| rlandy | because https://github.com/openstack/tripleo-ci/blob/master/playbooks/nodepool-provider/pre.yaml#L12 didn't work | 12:50 |
| rlandy | marios_|ruck: so as you think - as long as the upstream playbook can still run w/o error | 12:50 |
| weshay | panda|rover not atm no.. just saw it | 12:50 |
| *** jpena|lunch is now known as jpena | 12:52 | |
| *** saneax has quit IRC | 12:53 | |
| panda|rover | weshay: ok, trying with https://review.opendev.org/660379 | 12:53 |
| panda|rover | weshay: I'll try to launch a test on this in rdo | 12:53 |
| ykarel | weshay, panda|rover also notice tripleo-ci-testing tag is used instead of hash | 12:53 |
| ykarel | which is not pushed to registry with new container build job | 12:54 |
| ykarel | so need to consider that as well | 12:54 |
| weshay | panda++ thanks | 12:55 |
| ykarel | weshay, u checked my comment in https://review.opendev.org/#/c/660150/? | 12:56 |
| ykarel | if not can you check please | 12:56 |
| marios_|ruck | rlandy: ok lets see i'll update for now ... wrt minimal changes this is the 'fake' rhel8 element we're talking about here so we should just make it what we need until we switch to the 'real' one | 12:56 |
| marios_|ruck | rlandy: assuming it works always... | 12:57 |
| rlandy | marios_|ruck: removed setting python interp from https://review.opendev.org/#/c/660189/5..7/playbooks/multinode-standalone.yml | 12:57 |
| rlandy | ^^ is it set elsewhere | 12:57 |
| rlandy | looks like weshay did that | 12:58 |
| marios_|ruck | rlandy: i added it to the config | 12:58 |
| rlandy | the job config? | 12:58 |
| marios_|ruck | rlandy: here https://code.engineering.redhat.com/gerrit/167407 | 12:58 |
| marios_|ruck | rlandy: we don't know if that works yet | 12:58 |
| panda|rover | ykarel: mmhh, I'll try to replace dlrn_hash_tag with container_build_id, but I'm not sure if it's valued in this job | 12:59 |
| ykarel | panda|rover, can you check my comment in https://review.opendev.org/#/c/660150/, and then think about the solutin to fix it | 12:59 |
| ykarel | i think u need to go some steps back to fix the job completely | 13:00 |
| marios_|ruck | rlandy: wrt https://review.opendev.org/#/c/660151/8..9/playbooks/openstack-zuul-jobs/legacy/pre.yaml ... i can't set 777 on the directory do you think we need to override default permissions? | 13:00 |
| marios_|ruck | rlandy: i mean in creating on the image for /etc/nodepool/sub_nodes etc | 13:00 |
| marios_|ruck | rlandy: those have 766 but we won't do that... should we try default see what happens | 13:00 |
| weshay | arxcruz not sure if you'll be able to do anything w/ this.. seeing basicops fails in fs001 tests as well in check.. so could be real http://logs.rdoproject.org/openstack-periodic-latest-released/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-7-standalone-stein/58a70d7/logs/tempest.html.gz | 13:00 |
| marios_|ruck | rlandy: means iterating image builds though... :/ | 13:01 |
| arxcruz | weshay: it's network problem imho, but i'll take a look | 13:01 |
| rlandy | marios_|ruck: yep - hard to tell - we just hacked until it worked | 13:01 |
| rlandy | marios_|ruck: my best guess is to ask admins what is in rhel-7 | 13:01 |
| rfolco | weshay, ci community mtg needs to be updated in rhos mtg calendar, can you please update it ? It shows as canceled for me. | 13:01 |
| rlandy | and do the same | 13:01 |
| rlandy | save the build again issue | 13:02 |
| rlandy | marios_|ruck: also - if they can tell us how to kick the build job, that would be helpful | 13:02 |
| rfolco | weshay, I don't have write access to rhos mtg calendar. | 13:02 |
| weshay | arxcruz fs001 example fyi http://logs.rdoproject.org/33/614633/12/openstack-check/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001/a2a6539/logs/tempest.html.gz | 13:02 |
| rlandy | then we don;t have to bug them | 13:02 |
| sshnaidm | rfolco, when is community meeting? | 13:02 |
| rfolco | in 30 min from now | 13:02 |
| arxcruz | weshay:this is different | 13:03 |
| rfolco | 30 min before tripleo irc mtg | 13:03 |
| radez | weshay: home today, license to for tomorrow when I see him :) | 13:03 |
| arxcruz | weshay: same test, different errors, i'll check both | 13:03 |
| weshay | arxcruz thank you sir | 13:04 |
| weshay | rfolco I can't help you.. ask our admin eliska | 13:05 |
| marios_|ruck | rlandy: gona chown zuul:zuul on /etc/nodepool there should work | 13:05 |
| marios_|ruck | lets see | 13:05 |
| rfolco | weshay, arxcruz chandankumar panda|rover marios_|ruck rlandy sshnaidm: ci community mtg starts in 25 min from now. arxcruz will give a 10 min demo on zuul-runner today. | 13:05 |
| marios_|ruck | thanks rfolco | 13:05 |
| rfolco | so ignore what calendar says, will send email to eliska to fix it. | 13:06 |
| *** skramaja has quit IRC | 13:06 | |
| marios_|ruck | rfolco: in my calendar the time is correct 'rhos team meetings' cal | 13:06 |
| rfolco | hmm, weird | 13:07 |
| rfolco | ok then | 13:07 |
| rlandy | marios_|ruck: you can remove the changes in https://review.opendev.org/#/c/660151/9/zuul.d/base.yaml | 13:07 |
| rlandy | and | 13:08 |
| rlandy | https://review.opendev.org/#/c/660189/7/playbooks/multinode-standalone.yml | 13:08 |
| marios_|ruck | rlandy: k wdyt https://code.engineering.redhat.com/gerrit/#/c/170920/2/nodepool/elements/virt-customize/rhel8-cloud.yaml did i miss anything otherwise will reach out for rebuild | 13:09 |
| marios_|ruck | rlandy: looking | 13:09 |
| marios_|ruck | rlandy: k can you comment in gerrit otherwise it is impossible to track | 13:10 |
| rlandy | marios_|ruck: ha ... https://github.com/openstack/project-config/tree/master/nodepool/elements/zuul-worker | 13:10 |
| rlandy | https://github.com/openstack/project-config/blob/master/nodepool/elements/zuul-worker/install.d/60-zuul-worker | 13:10 |
| marios_|ruck | rlandy: nice that might be worth looking into if it fails here | 13:10 |
| marios_|ruck | rlandy: looking (creates the user etc?) | 13:10 |
| rlandy | marios_|ruck: see the install and the zuul_user piece | 13:11 |
| marios_|ruck | rlandy: ack it does keys but not directories | 13:11 |
| marios_|ruck | or i missed it | 13:11 |
| rlandy | marios_|ruck: ttps://github.com/openstack/project-config/blob/master/nodepool/elements/zuul-worker/extra-data.d/60-zuul-user | 13:11 |
| rlandy | ^^ keys | 13:11 |
| marios_|ruck | rlandy: 16:12 < marios_|ruck> rlandy: ack it does keys but not directories | 13:12 |
| *** aakarsh has joined #oooq | 13:12 | |
| marios_|ruck | rlandy: like /etc/nodepool stuff i added in virt-customize just now | 13:12 |
| rlandy | marios_|ruck: ack - I that was some legacy stuff we still carry around with us | 13:13 |
| rlandy | marios_|ruck: wrt https://code.engineering.redhat.com/gerrit/#/c/170920/2/nodepool/elements/virt-customize/rhel8-cloud.yaml | 13:14 |
| rlandy | looks ok - not sure if we need lines 159-162 but we will find out | 13:15 |
| rlandy | if we can run out own image builds, we can just rerun and take it out if no good | 13:15 |
| marios_|ruck | rlandy: ack... talking of which | 13:15 |
| marios_|ruck | jpena: o/ hello we please need new image build for https://code.engineering.redhat.com/gerrit/#/c/170920/ rhel8-cloud. is it something we can do instead of harrassing you every day? | 13:15 |
| marios_|ruck | jpena: we know there is periodic but we are trying to iterate as fast as we can | 13:16 |
| marios_|ruck | jpena: please thanks sorry | 13:16 |
| marios_|ruck | :D | 13:16 |
| jpena | marios_|ruck: besides periodic, you need an SF admin to do it, so feel free to ping me | 13:16 |
| jpena | it's just a simple command, no big deal | 13:16 |
| marios_|ruck | jpena: k :/ sorry about that thanks whenever you next can then we need to test that https://code.engineering.redhat.com/gerrit/#/c/170920/ | 13:16 |
| marios_|ruck | jpena: thank you very much | 13:16 |
| marios_|ruck | chandankumar: weshay rlandy fyi ^^^ | 13:17 |
| rlandy | marios_|ruck: k- let's try it | 13:17 |
| rlandy | if it's no good, we'll know before jpena EOD | 13:17 |
| rlandy | and we can revert | 13:17 |
| jpena | rlandy: in the US afternoon, you can count on nhicher | 13:18 |
| marios_|ruck | weshay: can't find your username to add you to this review https://code.engineering.redhat.com/gerrit/#/c/170920/Add creation of /etc/nodepool into the fake rhel-8 image for tripleo | 13:18 |
| marios_|ruck | https://code.engineering.redhat.com/gerrit/#/c/170920/ | 13:19 |
| marios_|ruck | jpena: you're right updating thanks | 13:19 |
| rlandy | jpena: thank you | 13:19 |
| jpena | np | 13:20 |
| marios_|ruck | jpena: well | 13:20 |
| marios_|ruck | jpena: i think we're gonna try with owner zuul | 13:20 |
| marios_|ruck | jpena: and default perms for now | 13:21 |
| marios_|ruck | jpena: i already have chown zuul:zuul | 13:21 |
| marios_|ruck | jpena: should be OK? | 13:21 |
| marios_|ruck | jpena: problem is i don't know what the 'correct' permissions are there but hoping that making zuul the owner will overcome the issue and defaults should be OK | 13:22 |
| marios_|ruck | jpena: replying on the review i think its ok | 13:22 |
| rlandy | marios_|ruck: I have some concerns about adding ansible_python_interpreter to the settings file. I shoudl work for standalone but may impact bm. If it's a problem when I get to bm, I'll do something about it then | 13:23 |
| rlandy | the virthost and the undercloud are not always the same distro | 13:24 |
| marios_|ruck | jpena: replied https://code.engineering.redhat.com/gerrit/#/c/170920/3 thanks please have a look too rlandy weshay chandankumar wdyt | 13:24 |
| marios_|ruck | rlandy: reading now | 13:25 |
| marios_|ruck | rlandy: ack | 13:25 |
| chandankumar | it is already workflowed | 13:28 |
| chandankumar | marios_|ruck: let's rebase the main patch | 13:28 |
| marios_|ruck | chandankumar: ack, getting to it now | 13:29 |
| marios_|ruck | chandankumar: but we have a call to join | 13:29 |
| marios_|ruck | brb | 13:29 |
| rlandy | marios_|ruck: ack - lets try this! | 13:29 |
| chandankumar | marios_|ruck: weshay rlandy do we want to merge https://code.engineering.redhat.com/gerrit/#/c/167407/ this | 13:29 |
| chandankumar | mock changes stuff | 13:29 |
| chandankumar | feel free to cleanup the depends on which got merged | 13:29 |
| rlandy | chandankumar: not yet - I'd like to see if the python interp change is correct | 13:29 |
| chandankumar | ok | 13:30 |
| rlandy | if it works this run, yeah | 13:30 |
| rlandy | chandankumar: thanks for fixing the missing tempest image | 13:30 |
| rlandy | rfolco: bj for meeting? | 13:31 |
| chandankumar | rlandy: we also enabled gerritbot for dowstream reviews on sf-ops | 13:31 |
| weshay | chandankumar ack.. I see https://download.devel.redhat.com/rel-eng/RCMTOOLS/latest-RCMTOOLS-2-RHEL-8/compose/BaseOS/x86_64/os/Packages/mock-1.4.14-2.el8eng.noarch.rpm perhaps we only want to install the one rpm | 13:32 |
| weshay | not sure if the rest would polute the build? | 13:32 |
| weshay | chandankumar we can create the repo properly and do an includes | 13:32 |
| weshay | if you look at other release file examples | 13:32 |
| chandankumar | weshay: nope it will install only mock | 13:32 |
| chandankumar | and it's dependencies | 13:32 |
| rfolco | weshay, ci community call | 13:33 |
| weshay | oh ya | 13:33 |
| weshay | you remove it | 13:33 |
| rfolco | https://bluejeans.com/4113567798 | 13:33 |
| weshay | k | 13:33 |
| weshay | rlandy based on ^ we can remove the exception on the build-test-packages ( maybe ) | 13:33 |
| chandankumar | weshay: https://code.engineering.redhat.com/gerrit/#/c/167407/10..11/config/release/rhos-15.yml just removing rcm repo after installing mock | 13:33 |
| weshay | chandankumar aye.. looks good | 13:34 |
| chandankumar | downstream standalone jobs does not have ara also | 13:35 |
| rlandy | weshay: ack - definitely | 13:35 |
| chandankumar | ykarel: ping | 13:37 |
| chandankumar | ykarel: https://logs.rdoproject.org/32/20832/4/check/periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset017-queens/a6f38bb/logs/undercloud/home/zuul/tempest.log.txt.gz#_2019-05-21_12_24_01 | 13:39 |
| chandankumar | rfolco: what is the bj link for community call? | 13:39 |
| rfolco | chandankumar, https://bluejeans.com/4113567798 | 13:39 |
| rfolco | wes room | 13:39 |
| panda|rover | jfrancoa: ping | 13:40 |
| jfrancoa | panda|rover: pong, hello o/ | 13:40 |
| panda|rover | jfrancoa: I have question regarding standalone upgrade | 13:41 |
| jfrancoa | panda|rover: sure, shoot. | 13:41 |
| chandankumar | rfolco: please add me to the community calender invite | 13:43 |
| chandankumar | ykarel: https://logs.rdoproject.org/32/20832/4/check/periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset017-queens/a6f38bb/logs/undercloud/var/log/extra/rpm-list.txt.gz | 13:43 |
| jpena | rlandy, marios_|ruck the updated image is now ready | 13:44 |
| rlandy | jpena: thanks | 13:44 |
| marios_|ruck | jpena: ty | 13:45 |
| marios_|ruck | jpena++ | 13:45 |
| ykarel | chandankumar, yes tempestconf 2.2.0 picked | 13:52 |
| ykarel | but job passed , how, /me looking ur patch now | 13:53 |
| chandankumar | ykarel: ok | 13:54 |
| chandankumar | ykarel: will polish the commit message then | 13:54 |
| ykarel | chandankumar, ack | 13:55 |
| ykarel | but strange removing Member role fixed it | 13:55 |
| *** holser__ has joined #oooq | 13:56 | |
| *** rf0lc0 has joined #oooq | 13:57 | |
| *** Vorrtex has joined #oooq | 13:58 | |
| weshay | sshnaidm sorry | 13:59 |
| marios_|ruck | chandankumar: actually going to try the rebase now... | 14:02 |
| marios_|ruck | chandankumar: fyi | 14:02 |
| marios_|ruck | on the 15 job i mean | 14:03 |
| chandankumar | marios_|ruck: I am fixing queens now | 14:03 |
| marios_|ruck | chandankumar: cool | 14:04 |
| marios_|ruck | chandankumar: thx | 14:04 |
| *** tosky__ has joined #oooq | 14:05 | |
| marios_|ruck | panda|rover: isn't the standup in half hour? | 14:05 |
| panda|rover | ... | 14:06 |
| *** derekh has quit IRC | 14:06 | |
| *** tosky has quit IRC | 14:06 | |
| *** holser_ has quit IRC | 14:06 | |
| *** rfolco has quit IRC | 14:06 | |
| *** dalvarez has quit IRC | 14:06 | |
| *** amoralej has quit IRC | 14:06 | |
| panda|rover | marios_|ruck: yyes, my font is very little | 14:06 |
| marios_|ruck | panda|rover: or should i join | 14:06 |
| *** udesale has quit IRC | 14:06 | |
| *** sshnaidm has quit IRC | 14:06 | |
| *** ksambor has quit IRC | 14:06 | |
| *** mjturek has quit IRC | 14:06 | |
| marios_|ruck | panda|rover: ack thx | 14:06 |
| *** udesale has joined #oooq | 14:07 | |
| *** amoralej` has joined #oooq | 14:08 | |
| *** rf0lc0 is now known as rfolco | 14:12 | |
| marios_|ruck | rebased chandankumar rlandy https://code.engineering.redhat.com/gerrit/#/c/167540/ | 14:12 |
| rlandy | thanks | 14:12 |
| rlandy | run again? | 14:12 |
| *** sshnaidm has joined #oooq | 14:12 | |
| chandankumar | ykarel: marios_|ruck: panda|rover here is the queens fix https://review.opendev.org/#/c/660351/2 | 14:14 |
| ykarel | ack | 14:15 |
| panda|rover | god save the queens. | 14:15 |
| rfolco | I want to break free | 14:15 |
| rfolco | God knows | 14:15 |
| chandankumar | panda|rover: call the King, | 14:16 |
| rfolco | fix the code coz I want to break free | 14:16 |
| *** vinaykns has joined #oooq | 14:17 | |
| rfolco | (molecule is needed for testing :D) | 14:25 |
| rfolco | marios_|ruck, sorin from desert smiles | 14:25 |
| rlandy | marios_|ruck: if this run works, I'd like to remove some hack from the reviews so we are only left with what we really need | 14:27 |
| *** ykarel is now known as ykarel|afk | 14:27 | |
| marios | panda|rover: reminder call in 1 min | 14:28 |
| marios | panda|rover: ;) | 14:28 |
| marios | rlandy: yes well thats what i've been trying to do | 14:28 |
| marios | rlandy: joining call | 14:28 |
| marios | rlandy: ack | 14:29 |
| chandankumar | marios: rlandy regarding nodepool changes http://paste.openstack.org/show/751876/ | 14:31 |
| rlandy | yeah - we have a lot of legacy stiff to ditch | 14:32 |
| rlandy | but that is a much larger project than this | 14:32 |
| *** marios_|ruck is now known as marios|ruck|call | 14:33 | |
| *** amoralej` is now known as amoralej | 14:38 | |
| chandankumar | weshay: rlandy marios I am logging out, feel free to mail us for tomorrow rhos-15 debugging, thanks, See ya tomorrow :-) | 14:46 |
| rlandy | sure | 14:46 |
| *** chandankumar is now known as raukadah | 14:46 | |
| marios|ruck|call | raukadah: thanks | 14:46 |
| weshay | raukadah I'm trying to get an OSP phase 2 build up now | 14:46 |
| *** tosky__ is now known as tosky | 14:59 | |
| *** hamzy has quit IRC | 15:07 | |
| *** dalvarez has joined #oooq | 15:12 | |
| rlandy | marios|ruck|call: marios++ looking good - on tempest | 15:12 |
| marios|ruck|call | rlandy: ack i didn't want to say but it didn't fail immediately so... | 15:13 |
| marios|ruck|call | rlandy: waiting | 15:13 |
| marios|ruck|call | rlandy: on the dfg:all call | 15:13 |
| marios|ruck|call | hopign this is green cos going to drop after the call | 15:13 |
| marios|ruck|call | rlandy: i will continue tomorrow to try and remove the hacks depending on where you leave off in your day today | 15:13 |
| rlandy | marios|ruck|call: ack - please ping before your EOD so we can confirm what still needs to be done | 15:13 |
| marios|ruck|call | rlandy: sorry didn't make huge progess was ruck half of day | 15:13 |
| marios|ruck|call | rlandy: ack | 15:13 |
| marios|ruck|call | rlandy: but compared to 2 days ago a LOT of patches and hacks there and there has actually been a green job so | 15:14 |
| marios|ruck|call | rlandy++ | 15:14 |
| rlandy | team effort here | 15:14 |
| *** ksambor_ has joined #oooq | 15:18 | |
| rlandy | 2019-05-21 11:16:43.862423 | primary | Traceback (most recent call last): | 15:22 |
| rlandy | 2019-05-21 11:16:43.862427 | primary | File "./tempestmail.py", line 22, in <module> | 15:22 |
| rlandy | 2019-05-21 11:16:43.862435 | primary | import requests | 15:22 |
| rlandy | 2019-05-21 11:16:43.862440 | primary | ImportError: No module named requests | 15:22 |
| rlandy | 2019-05-21 11:16:43.862444 | primary | | 15:22 |
| rlandy | ^^ real issue? | 15:23 |
| marios|ruck|call | https://sf.hosted.upshift.rdu2.redhat.com/logs/40/167540/35/check/tripleo-ci-rhel-8-standalone-rhos-15/a323169/ | 15:24 |
| marios|ruck|call | green ? | 15:25 |
| marios|ruck|call | rlandy: so lets merge that weshay https://code.engineering.redhat.com/gerrit/#/c/167407/ rlandy | 15:29 |
| marios|ruck|call | and this https://review.opendev.org/#/c/660151/ | 15:29 |
| weshay | marios|ruck|call agree, rlandy need review on https://review.opendev.org/#/c/660151/ | 15:32 |
| rlandy | weshay: marios|ruck|call: I am not sure if we need https://review.opendev.org/#/c/660151/9/zuul.d/base.y | 15:33 |
| marios|ruck|call | rlandy: ack you wana comment we can flip and recheck | 15:34 |
| *** hamzy has joined #oooq | 15:35 | |
| weshay | ya.. worth checking w/o the zuul change w/ the new image.. agree | 15:37 |
| rlandy | weshay: yeah - would like to make fewer changes that impact upstream | 15:39 |
| rlandy | one more run w/o change | 15:39 |
| rlandy | then merge | 15:39 |
| rlandy | ok? | 15:39 |
| rlandy | weshay: marios: let's also enable build ... https://review.opendev.org/#/c/660189/7/playbooks/multinode-standalone.yml | 15:41 |
| weshay | rlandy go 4 it | 15:44 |
| rlandy | in progress | 15:44 |
| *** ykarel|afk is now known as ykarel|away | 15:45 | |
| marios|ruck|call | https://review.opendev.org/#/c/660151/ rlandy just hit the update button but you beat me to it | 15:46 |
| rlandy | marios: updating the job file | 15:46 |
| weshay | rlandy started anything w/ install-deps.sh? Just curious because if not I'll fire up a vm | 15:50 |
| *** ykarel|away has quit IRC | 15:51 | |
| rlandy | weshay: no - that still needs works and clean up | 15:51 |
| rlandy | weshay: I have the rhel8 box from yesterday if you want | 15:52 |
| rlandy | marios|ruck|call: weshay: kicked a jobs with reduced depends-ons https://code.engineering.redhat.com/gerrit/#/c/167540/36 | 15:52 |
| marios|ruck|call | rlandy: nice thanks | 15:52 |
| rlandy | if that works, we have just the install-deps stuff to hash out | 15:52 |
| *** marios|ruck|call is now known as marios|ruck | 15:52 | |
| weshay | nice.. down to two | 15:53 |
| rlandy | yep - https://review.opendev.org/660151 is pretty harmless | 15:53 |
| rlandy | so really one major | 15:53 |
| rlandy | weshay: want to work on install-deps together or you have that sorted? if so, will move on to osp-15 on bm | 15:54 |
| marios|ruck | weshay: rlandy /me begins shutdown sequence need something else? | 15:55 |
| marios|ruck | rlandy: thanks for picking up looks like you're well in | 15:55 |
| rlandy | marios|ruck: not really - will just move the card on taiga out of blocked | 15:55 |
| rlandy | as I think we are close | 15:55 |
| *** ccamacho has quit IRC | 15:57 | |
| * marios|ruck shutdown sequence | 15:57 | |
| weshay | marios|ruck \0 thanks man | 15:58 |
| raukadah | weshay: will take a look once jobs finishes | 15:58 |
| marios|ruck | weshay: thanks you raukadah rlandy team ++ | 15:58 |
| raukadah | marios|ruck: you are welcome :-) | 15:59 |
| marios|ruck | rlandy: update taiga/email/fax whatever you prefer for me to pickup tomorrow | 16:01 |
| * rlandy finds fax | 16:02 | |
| rlandy | to update marios|ruck | 16:02 |
| marios|ruck | :D | 16:02 |
| rlandy | carrier pigeon? | 16:02 |
| rlandy | weshay: :( ... still timing out on bm in periodic ... https://sf.hosted.upshift.rdu2.redhat.com/logs/periodic-hourly/code.engineering.redhat.com/openstack/tripleo-ci-internal-jobs/master/periodic-tripleo-ci-centos-7-bm_envB-3ctlr_1comp-featureset001-stein/6e9666a/job-output.txt.gz | 16:03 |
| rlandy | ok if increase job timeout on our tenant? | 16:03 |
| rlandy | weshay:^^ | 16:03 |
| raukadah | better go for telegraphy | 16:03 |
| rlandy | overcloud deploy is not stuck - it just gets only an hour to go ... https://sf.hosted.upshift.rdu2.redhat.com/logs/periodic-hourly/code.engineering.redhat.com/openstack/tripleo-ci-internal-jobs/master/periodic-tripleo-ci-centos-7-bm_envB-3ctlr_1comp-featureset001-stein/6e9666a/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz | 16:04 |
| *** marios|ruck has quit IRC | 16:09 | |
| *** marios has quit IRC | 16:09 | |
| *** panda|rover has quit IRC | 16:09 | |
| *** holser__ has quit IRC | 16:09 | |
| *** bogdando has quit IRC | 16:10 | |
| *** panda has joined #oooq | 16:10 | |
| *** holser_ has joined #oooq | 16:11 | |
| *** jfrancoa has quit IRC | 16:11 | |
| weshay | rlandy you using the very latest centos? | 16:13 |
| rlandy | weshay:on the virthost? | 16:14 |
| rlandy | the image for sure | 16:14 |
| rlandy | I still need to virt-customize the centos image | 16:15 |
| rlandy | and build overcloud images | 16:15 |
| rlandy | takes time | 16:15 |
| weshay | rlandy ovb is down on fs001 master.. would be cool if I could see a bm run on it | 16:20 |
| weshay | rlandy http://logs.rdoproject.org/79/660379/1/openstack-check/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001/62b25d7/logs/tempest.html.gz | 16:20 |
| weshay | so I can determine if it's the infra.. arxcruz fyi ^ | 16:21 |
| rlandy | checking last master run | 16:22 |
| rlandy | weshay: timing out again ... https://sf.hosted.upshift.rdu2.redhat.com/logs/periodic-hourly/code.engineering.redhat.com/openstack/tripleo-ci-internal-jobs/master/periodic-tripleo-ci-centos-7-bm_envA-3ctlr_1comp-featureset001-master/a0966d1/job-output.txt.gz | 16:23 |
| rlandy | I asked nhicher before if he had an issue with my increasing the job timeout, he said no problem from admins | 16:24 |
| rlandy | so we can do thta | 16:24 |
| rlandy | that | 16:24 |
| rlandy | otherwise no way we will finish these jobs | 16:24 |
| * rlandy puts up review | 16:24 | |
| *** holser__ has joined #oooq | 16:29 | |
| *** holser_ has quit IRC | 16:30 | |
| *** holser__ has quit IRC | 16:30 | |
| *** holser_ has joined #oooq | 16:31 | |
| weshay | rlandy if you can have bm fs001 by EOD that would be awesome | 16:34 |
| weshay | baremetal | 16:34 |
| weshay | context http://dashboard-ci.tripleo.org/d/cEEjGFFmz/cockpit?orgId=1&fullscreen&panelId=207 | 16:34 |
| rlandy | weshay: pls vote on these two before I merge: https://code.engineering.redhat.com/gerrit/170968 Increase tripleo-ci-internal job timeout | 16:35 |
| rlandy | and | 16:35 |
| rlandy | https://code.engineering.redhat.com/gerrit/170970 Increase BM job timeout | 16:36 |
| rlandy | otherwise no show | 16:36 |
| rlandy | https://code.engineering.redhat.com/gerrit/#/c/170968/ needs to go first | 16:37 |
| *** dtantsur is now known as dtantsur|afk | 16:48 | |
| rlandy | weshay: thanks merged - kicking bm master - you want tripleo-ci-testing or current-tripleo? | 16:50 |
| rlandy | check | 16:50 |
| weshay | rlandy current-tripleo | 16:50 |
| rlandy | so current-tripleo | 16:50 |
| rlandy | got it | 16:50 |
| weshay | thanks | 16:50 |
| *** udesale has quit IRC | 16:53 | |
| rlandy | https://sf.hosted.upshift.rdu2.redhat.com/zuul/t/tripleo-ci-internal/stream/9997adb7d1314687b4ce4c8503fbcbf1?logfile=console.log | 16:54 |
| rlandy | ^^ osp-15 standalone ... looks good with minimal depends-on | 16:55 |
| rlandy | weshay: ^^ | 16:55 |
| weshay | aye.. saw | 16:55 |
| rlandy | weshay: https://code.engineering.redhat.com/gerrit/165815 Test master | 17:00 |
| rlandy | just checking that gets the right release file | 17:01 |
| *** quiquell has quit IRC | 17:10 | |
| rlandy | sshnaidm: ping re: https://code.engineering.redhat.com/gerrit/#/c/170910/ | 17:22 |
| rlandy | sshnaidm: we don;t want ovb config on all rhel jobs | 17:23 |
| sshnaidm | rlandy, I'd say it doesn't hurt | 17:23 |
| rlandy | something up with bm config | 17:24 |
| *** jpena is now known as jpena|off | 17:24 | |
| * rlandy checks for unintended changes | 17:24 | |
| rlandy | sshnaidm: no need | 17:24 |
| sshnaidm | rlandy, I can extract it to different playbook | 17:24 |
| rlandy | how did baremetal.yml become baremetal-internal???? | 17:27 |
| sshnaidm | rlandy, maybe this caused? https://review.opendev.org/#/c/660138/ | 17:31 |
| rlandy | https://github.com/openstack/tripleo-ci/blob/745ee233679b8f8b08ab36f7f84fc99f24b87e17/roles/common/vars/main.yaml#L50 | 17:31 |
| *** kopecmartin is now known as kopecmartin|off | 17:31 | |
| *** saneax has joined #oooq | 17:32 | |
| rlandy | sshnaidm: ^^ that is causing it to look for a file called baremetal-internal | 17:32 |
| rlandy | https://github.com/openstack/tripleo-ci/blob/master/roles/common/vars/main.yaml#L50 | 17:32 |
| sshnaidm | rlandy, and before this rhcloud wasn't defined, right? | 17:33 |
| rlandy | does not exist | 17:33 |
| rlandy | correct | 17:33 |
| rlandy | now ... | 17:33 |
| rlandy | {%- elif nodepool_provider == 'tripleo-ci' -%} | 17:33 |
| rlandy | internal | 17:33 |
| rlandy | {%- endif -%} | 17:33 |
| sshnaidm | rlandy, yeah | 17:33 |
| rlandy | if we ran bm on rdoclud before we would have hit this | 17:33 |
| rlandy | sshnaidm: can you case that on ovb? | 17:33 |
| sshnaidm | rlandy, I can change to "elif nodepool_provider == 'tripleo-ci' and environment_type == 'ovb'" | 17:34 |
| sshnaidm | rlandy, although a little bit ugly | 17:34 |
| rlandy | sshnaidm: ok by me | 17:35 |
| rlandy | when you have the review, I'll try it with the current test run | 17:35 |
| sshnaidm | rlandy, https://review.opendev.org/660474 | 17:37 |
| *** ykarel|away has joined #oooq | 17:37 | |
| sshnaidm | rlandy, also https://code.engineering.redhat.com/gerrit/170977 | 17:37 |
| rlandy | sshnaidm: https://code.engineering.redhat.com/gerrit/170977 looks good | 17:38 |
| rlandy | trying tripleo-ci patch - thanks | 17:39 |
| *** saneax has quit IRC | 17:41 | |
| *** saneax has joined #oooq | 17:42 | |
| *** amoralej is now known as amoralej|off | 17:46 | |
| sshnaidm | rlandy, seems like introspection fails in upshift cloud we have: https://sf.hosted.upshift.rdu2.redhat.com/logs/61/168161/23/check/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-internal/b0c4cc2/logs/undercloud/home/zuul/overcloud_prep_images.log.txt.gz#_2019-05-20_22_14_35 | 17:49 |
| sshnaidm | or maybe problem with bmc | 17:50 |
| rlandy | ok - bm looks better with pacth | 18:02 |
| rlandy | looking at introspection | 18:02 |
| rlandy | sshnaidm: can you get on the undercloud and see if you have ipmi control? | 18:03 |
| rlandy | is the instackenv.json correct? | 18:03 |
| sshnaidm | rlandy, not sure I can get on undercloud.. | 18:12 |
| sshnaidm | will try tomorrow to do it manually | 18:12 |
| rlandy | sshnaidm: k - ping me tomorrow if you want to work on it together | 18:13 |
| *** panda has quit IRC | 18:14 | |
| *** panda has joined #oooq | 18:15 | |
| *** holser_ has quit IRC | 18:16 | |
| *** tcw has quit IRC | 18:19 | |
| rlandy | weshay: are you working on bindep stuff or can I edit https://review.opendev.org/#/c/659879/? | 18:51 |
| * rlandy also wants to fix containers-prepare-parameters | 18:52 | |
| weshay | rlandy happy to tmate w/ you | 18:52 |
| weshay | it was all kinds of f'd | 18:52 |
| weshay | why break a tradition | 18:52 |
| * weshay gets a soda | 18:52 | |
| rlandy | ok- give me 5 - I'll ping you with the tmate session | 18:52 |
| weshay | rlandy I have it going | 18:53 |
| rlandy | k | 18:54 |
| sshnaidm | weshay, images were built: https://sf.hosted.upshift.rdu2.redhat.com/logs/61/168161/25/check/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-internal/0c9b5ad/logs/undercloud/home/zuul/overcloud_image_build.log.txt.gz#_2019-05-21_14_17_26 | 18:56 |
| sshnaidm | weshay, just problems with ipmi on cloud, will look tomorrow.. | 18:56 |
| *** sshnaidm is now known as sshnaidm|afk | 18:56 | |
| rlandy | weshay: sudo $(package_manager) install shyaml || true # install when available | 19:14 |
| rlandy | https://review.opendev.org/#/c/660151/10/roles/run-test/templates/toci_gate_test.sh.j2 | 19:18 |
| rlandy | weshay: ^^ | 19:18 |
| *** phyrox has joined #oooq | 19:32 | |
| *** ykarel|away has quit IRC | 19:36 | |
| *** vinaykns has quit IRC | 19:38 | |
| *** vkapalav has joined #oooq | 19:38 | |
| weshay | rlandy k.. cool -y is included btw https://opendev.org/openstack/tripleo-ci/src/branch/master/roles/run-test/templates/oooq_common_functions.sh.j2#L232 | 19:43 |
| rlandy | oh - I see | 19:43 |
| rlandy | that's why it worked | 19:43 |
| *** Goneri has quit IRC | 19:48 | |
| *** saneax has quit IRC | 19:51 | |
| *** Goneri has joined #oooq | 20:02 | |
| *** Vorrtex has quit IRC | 20:02 | |
| *** dsneddon has quit IRC | 20:04 | |
| rlandy | weshay: need to run out for about 20 mins - biab | 20:05 |
| rlandy | there are two master bm jobs running | 20:06 |
| rlandy | one on tripleo-ci and and one on tripleo-ci-testing | 20:06 |
| *** rlandy is now known as rlandy|biab | 20:06 | |
| weshay | k.. thanks | 20:06 |
| *** rlandy|biab is now known as rlandy | 20:39 | |
| rlandy | weshay: pls vote - fixes change breaking bm https://review.opendev.org/#/c/660474 | 20:43 |
| weshay | rlandy fyi.. rhel-8 install-deps works.. I borked something on centos | 20:44 |
| weshay | updated the code.eng review fyi | 20:44 |
| rlandy | ha - you win some you lose some | 20:44 |
| *** hamzy has quit IRC | 20:45 | |
| phyrox | I just saw that tripleo ui is deprecated starting in stein, is there anything to replace it? | 20:53 |
| weshay | phyrox for tripleo-ui I'm afaid not.. horizon will still be avail | 20:54 |
| phyrox | i see | 20:55 |
| weshay | rlandy the latest run got through tempest, so I think we're good.. the odd thing is that the job is canceled when I update a dependency | 21:19 |
| rlandy | weshay: I rechecked it | 21:20 |
| rlandy | it's running again now | 21:20 |
| rlandy | it will get canceled if you update the dep | 21:20 |
| weshay | I saw.. but hold off as I'm updated the install-deps a bit | 21:20 |
| weshay | ya.. that is weird right | 21:21 |
| weshay | I think I added the default python to just the test instance... and not my source :) | 21:21 |
| weshay | easy | 21:21 |
| weshay | looks like it's working now | 21:21 |
| weshay | so I think I'm probably good on that patch | 21:21 |
| weshay | https://review.rdoproject.org/zuul/stream/36a5edf10bc2466f982cc3f9b6d32bde?logfile=console.log | 21:22 |
| rlandy | weshay: not sure if you saw my question on #prodchain-infra-dfg ... | 21:22 |
| * weshay rereads | 21:22 | |
| rlandy | do you want to export all your log on review.rdo ? If yes, perhaps change the post job is the best solution | 21:22 |
| rlandy | weshay: ^^ | 21:22 |
| weshay | rlandy where would that change be made, previously it was env config | 21:24 |
| weshay | not sure now that we're all zuul'd out | 21:24 |
| rlandy | weshay: so I was testing with raukadah on a test-basis | 21:25 |
| rlandy | but we'd make the change on config | 21:25 |
| rlandy | tripleo-ci-internal-config | 21:25 |
| rlandy | we'll need this https://github.com/rdo-infra/review.rdoproject.org-config/blob/master/zuul.d/secrets.yaml | 21:25 |
| rlandy | equivalent thereof | 21:25 |
| rlandy | http://git.app.eng.bos.redhat.com/git/openstack/tripleo-ci-internal-config.git/tree/playbooks/base/post.yaml#n14 | 21:26 |
| rlandy | ^^ change the values here | 21:26 |
| rlandy | on base | 21:26 |
| weshay | ah cool | 21:28 |
| weshay | thanks rlandy | 21:28 |
| weshay | ugh | 21:33 |
| weshay | what's wrong w/ me | 21:33 |
| rlandy | oh - what now? | 21:33 |
| weshay | rlandy just forgot all the changes you helped me correct | 21:33 |
| rlandy | weshay: should we tmate again? | 21:34 |
| rlandy | sorry - was busy watching th bm job | 21:34 |
| weshay | rlandy don't think so | 21:34 |
| weshay | rlandy let me resubmit and have you look | 21:34 |
| weshay | doing so now | 21:34 |
| rlandy | weshay: ack | 21:34 |
| weshay | rlandy https://review.opendev.org/#/c/660508/4/install-deps.sh | 21:35 |
| rlandy | looking | 21:35 |
| rlandy | weshay: https://review.opendev.org/#/c/660508 - two questions/comments | 21:39 |
| *** aakarsh has quit IRC | 21:42 | |
| weshay | rlandy I don't think oooq_common is used locally | 21:59 |
| weshay | or for the repro | 21:59 |
| *** tesseract has quit IRC | 22:07 | |
| *** vkapalav has quit IRC | 22:24 | |
| *** aakarsh has joined #oooq | 22:29 | |
| *** phyrox has quit IRC | 22:34 | |
| rlandy | weshay: looks good with what's running now - so ok | 22:39 |
| *** tosky has quit IRC | 22:42 | |
| rlandy | weshay: so bm finished with failing overcloud deploy on master ... https://sf.hosted.upshift.rdu2.redhat.com/logs/15/165815/52/check/periodic-tripleo-ci-centos-7-bm_envD-3ctlr_1comp-featureset001-master/cf99e1e/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz | 23:07 |
| rlandy | iiuc, OVB was only failing tempest? | 23:07 |
| rlandy | weird - passed but failed? | 23:09 |
| rlandy | weshay: wait, I lie ... we have it - the tempest failure !! | 23:09 |
| rlandy | https://sf.hosted.upshift.rdu2.redhat.com/logs/15/165815/52/check/periodic-tripleo-ci-centos-7-bm_envD-3ctlr_1comp-featureset001-master/cf99e1e/logs/undercloud/home/zuul/tempest.log.txt.gz | 23:10 |
| rlandy | ha - there we go | 23:10 |
| rlandy | machine is yours or arxcruz if you still want it | 23:10 |
| rlandy | 2019-05-21 22:44:31.382903 | primary | TASK [validate-tempest : Execute tempest] ************************************** | 23:11 |
| rlandy | 2019-05-21 22:44:31.467143 | primary | Tuesday 21 May 2019 22:44:31 +0000 (0:00:00.157) 0:01:01.572 *********** | 23:11 |
| rlandy | 2019-05-21 22:46:51.612512 | primary | fatal: [undercloud]: FAILED! => { | 23:11 |
| rlandy | perfect!! | 23:11 |
| weshay | nice | 23:24 |
| weshay | bbl have the kids on tues :) | 23:24 |
| rlandy | sure | 23:28 |
| *** Goneri has quit IRC | 23:52 | |
| *** rlandy has quit IRC | 23:58 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!