| *** aakarsh has quit IRC | 00:05 | |
| *** dsneddon has quit IRC | 00:21 | |
| *** dsneddon has joined #oooq | 00:29 | |
| *** sanjayu__ has joined #oooq | 01:26 | |
| *** dsneddon has quit IRC | 01:40 | |
| *** rlandy|bbl is now known as rlandy | 01:42 | |
| rlandy | weshay: still having ovb issues? | 01:42 |
|---|---|---|
| weshay | rlandy I'm going to reload w/ centos I tihnk | 01:42 |
| weshay | and hello :) | 01:42 |
| rlandy | hello | 01:42 |
| rlandy | reload? | 01:42 |
| rlandy | weshay: also - logs change merged ok I think | 01:43 |
| rlandy | https://sf.hosted.upshift.rdu2.redhat.com/logs/35/173835/7/check/tripleo-ci-rhel-8-standalone-rhos-15/b7da029/ | 01:43 |
| rlandy | logged alright | 01:43 |
| rlandy | will try the using rdo log server with a down sf rocky job | 01:43 |
| rlandy | I don't think that will leak any info | 01:44 |
| weshay | perfect, sounds good | 01:44 |
| rlandy | https://code.engineering.redhat.com/gerrit/174055 DNM: really DNM: testing logging to RDOCLOUD log server | 01:45 |
| rlandy | OVB jobs look ok now | 01:46 |
| weshay | the dlrn issue? | 01:49 |
| rlandy | weshay: looky here ... https://logs.rdoproject.org/55/174055/1/check/tox-linters/5536e5c/ | 01:54 |
| rlandy | ^^ from dowstream sf | 01:54 |
| weshay | k.. np | 01:55 |
| *** apetrich has quit IRC | 01:57 | |
| weshay | rlandy I think I may have deleted the wrong bmc-tempalte | 02:19 |
| weshay | template | 02:19 |
| rlandy | weshay: maybe I can send it to you? | 02:20 |
| rlandy | from my personal tenant | 02:20 |
| weshay | sshnaidm ^ check your tenant for a public and shared copy of the bmc tempalte | 02:20 |
| weshay | rlandy ya.. maybe I can import it | 02:20 |
| rlandy | which one did you delete? | 02:20 |
| weshay | where do I change which template it uses? | 02:20 |
| weshay | the shared one, I can't delete the public one | 02:21 |
| rlandy | I think you can just import the shared one again | 02:21 |
| weshay | matbu dupe cloud images causing some headaches | 02:21 |
| * weshay looks at instructions | 02:21 | |
| * rlandy gets | 02:21 | |
| weshay | I think we nuked those instructions | 02:22 |
| rlandy | https://github.com/rdo-infra/review.rdoproject.org-config/blob/master/roles/ovb-manage/tasks/ovb-create-stack.yml#L34 | 02:23 |
| rlandy | looking for args | 02:23 |
| weshay | k | 02:23 |
| rlandy | https://github.com/rdo-infra/review.rdoproject.org-config/blob/5571f385299bbd01c5d227ef805952e124f7c3a5/roles/ovb-manage/tasks/generate_template.yml#L21 | 02:24 |
| rlandy | there we go | 02:24 |
| rlandy | bmc_template_name|default("bmc-template") | 02:25 |
| weshay | rdocloud: | 02:25 |
| weshay | public_ip_net: 38.145.32.0/22 | 02:25 |
| weshay | undercloud_flavor: m1.large | 02:25 |
| weshay | baremetal_flavor: m1.large | 02:25 |
| weshay | bmc_flavor: m1.medium | 02:25 |
| weshay | extra_node_flavor: m1.small | 02:25 |
| weshay | baremetal_image: CentOS-7-x86_64-GenericCloud-1804_02 | 02:25 |
| rlandy | so that is how it picks what image to use | 02:25 |
| weshay | oh | 02:25 |
| rlandy | default("bmc-template") | 02:25 |
| rlandy | getting the shared template for you to import | 02:26 |
| weshay | rlandy if kieren pushed a public version w/ the same name.. that is not working.. that busts this whole thing up | 02:26 |
| rlandy | weshay: exactly | 02:26 |
| weshay | unless we change the name | 02:26 |
| weshay | maybe we can auto import for people | 02:27 |
| rlandy | weshay: you could set var to ID I think | 02:27 |
| weshay | ya | 02:27 |
| rlandy | bmc_template_name|default("bmc-template") | 02:27 |
| weshay | ya | 02:27 |
| rlandy | what happens if we set bmc_template_name to the ID we want? | 02:27 |
| weshay | that should work | 02:27 |
| weshay | but people need access to it | 02:27 |
| rlandy | bmc-template Image Active shared No QCOW2 1.34 GB | 02:28 |
| rlandy | bmc-template Image Active Public Yes QCOW2 1.44 GB | 02:28 |
| rlandy | I have those two | 02:28 |
| rlandy | ID | 02:29 |
| rlandy | 2d281a65-bb80-4bff-b065-3775bedd7bea | 02:29 |
| rlandy | ^^ shared one | 02:29 |
| weshay | k.. /me looks for the google doc | 02:29 |
| weshay | thanks btw :) | 02:29 |
| weshay | got it | 02:30 |
| rlandy | you got the instruction? | 02:30 |
| weshay | hrm.. ya.. but don't see anything about bmc-template | 02:30 |
| weshay | https://docs.google.com/document/d/1i1-_L2mx8_erETAVblchLnymC6-9h7A2x02begy99UQ/edit# | 02:31 |
| rlandy | the bmc temp;ate was needed way before reproducer | 02:33 |
| weshay | https://bugs.launchpad.net/tripleo/+bug/1824954 | 02:33 |
| openstack | Launchpad bug 1824954 in tripleo "zuul reproducer Error validating value 'bmc-template': No images matching {'name': u'bmc-template'}." [High,Fix released] | 02:33 |
| rlandy | status: Triaged → Fix Released | 02:34 |
| rlandy | very informative | 02:35 |
| weshay | rlandy meh.. let's call it | 02:37 |
| weshay | chat tomorrow | 02:37 |
| rlandy | still looking - if I find it, I'll paste | 02:37 |
| rlandy | gn | 02:37 |
| weshay | rlandy I'll be watching for results from https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-reproducer-centos-7-libvirt-standalone-vexxhost | 02:38 |
| rlandy | k | 02:38 |
| rlandy | openstack image set --accept <image ID> | 02:40 |
| rlandy | weshay: ^^ can you try that? | 02:40 |
| rlandy | raukadah: when you come on line, pls see rdocloud logs from internal sf example ... code.engineering.redhat.com/gerrit/#/c/174055 | 02:44 |
| *** rlandy has quit IRC | 02:44 | |
| *** Goneri has quit IRC | 03:18 | |
| *** dsneddon has joined #oooq | 03:19 | |
| *** ykarel has joined #oooq | 03:25 | |
| *** dsneddon has quit IRC | 03:27 | |
| *** udesale has joined #oooq | 03:56 | |
| *** skramaja has joined #oooq | 03:59 | |
| *** skramaja has quit IRC | 04:14 | |
| *** skramaja has joined #oooq | 04:16 | |
| *** sanjayu__ has quit IRC | 04:22 | |
| *** ykarel has quit IRC | 04:40 | |
| *** ykarel has joined #oooq | 04:55 | |
| *** ratailor has joined #oooq | 05:22 | |
| *** dsneddon has joined #oooq | 05:24 | |
| *** ratailor_ has joined #oooq | 05:24 | |
| *** ratailor has quit IRC | 05:27 | |
| *** dsneddon has quit IRC | 05:28 | |
| *** dsneddon has joined #oooq | 05:29 | |
| *** dtrainor has quit IRC | 05:31 | |
| *** dsneddon has quit IRC | 05:33 | |
| *** marios has joined #oooq | 05:37 | |
| *** raukadah is now known as chandankumar | 05:40 | |
| *** ccamacho has quit IRC | 05:48 | |
| *** udesale has quit IRC | 06:07 | |
| *** udesale has joined #oooq | 06:08 | |
| *** jfrancoa has joined #oooq | 06:37 | |
| *** ccamacho has joined #oooq | 06:42 | |
| *** saneax has joined #oooq | 06:45 | |
| *** udesale has quit IRC | 06:45 | |
| *** udesale has joined #oooq | 06:46 | |
| *** jtomasek has joined #oooq | 06:48 | |
| *** kopecmartin|off is now known as kopecmartin | 06:53 | |
| *** apetrich has joined #oooq | 06:58 | |
| *** tesseract has joined #oooq | 07:07 | |
| *** ccamacho has quit IRC | 07:09 | |
| *** amoralej|off is now known as amoralej | 07:21 | |
| *** tosky has joined #oooq | 07:26 | |
| *** udesale has quit IRC | 07:27 | |
| *** udesale has joined #oooq | 07:28 | |
| *** udesale has quit IRC | 07:33 | |
| *** udesale has joined #oooq | 07:33 | |
| *** udesale has quit IRC | 07:33 | |
| *** udesale has joined #oooq | 07:34 | |
| arxcruz | sshnaidm https://review.rdoproject.org/r/#/c/21203/ | 07:35 |
| arxcruz | sshnaidm can you take a look? | 07:35 |
| *** ykarel is now known as ykarel|lunch | 07:38 | |
| *** jpena|off is now known as jpena | 07:46 | |
| *** hamzy has quit IRC | 07:50 | |
| *** holser_ has joined #oooq | 07:51 | |
| *** hamzy has joined #oooq | 07:51 | |
| *** gkadam has joined #oooq | 07:58 | |
| *** chem has joined #oooq | 07:59 | |
| *** saneax has quit IRC | 08:06 | |
| *** sanjayu_ has joined #oooq | 08:06 | |
| *** bogdando has joined #oooq | 08:12 | |
| *** bogdando has quit IRC | 08:14 | |
| *** zbr has joined #oooq | 08:17 | |
| *** udesale has quit IRC | 08:23 | |
| *** udesale has joined #oooq | 08:23 | |
| *** ykarel|lunch is now known as ykarel | 08:36 | |
| *** ccamacho has joined #oooq | 08:45 | |
| arxcruz | sshnaidm the tox job isn't being executed | 08:46 |
| arxcruz | wondering if i am missing something | 08:47 |
| arxcruz | amoralej ^https://review.rdoproject.org/r/#/c/21203/ | 08:47 |
| amoralej | arxcruz, let me check | 08:48 |
| amoralej | i think you need to add the repo to zuul config | 08:48 |
| amoralej | arxcruz, https://review.rdoproject.org/r/#/c/20652/ did you do something similar? | 08:50 |
| arxcruz | amoralej of course not :/ | 08:51 |
| arxcruz | duh | 08:51 |
| arxcruz | amoralej doing it now | 08:51 |
| amoralej | need to add the repo around https://github.com/rdo-infra/review.rdoproject.org-config/blob/master/zuul/rdo.yaml#L151 | 08:51 |
| *** yolanda has quit IRC | 08:54 | |
| zbr | what an wonderful day... github down,... all zuul changes failed over the night for various reasons,.... | 08:55 |
| arxcruz | amoralej https://review.rdoproject.org/r/21205 | 08:55 |
| *** derekh has joined #oooq | 09:12 | |
| arxcruz | amoralej is that instantly or do i need to wait some zuul refresh ? | 09:19 |
| amoralej | should be fine, i think | 09:20 |
| *** ykarel is now known as ykarel|afk | 09:28 | |
| sshnaidm | chandankumar, arxcruz , can you please take a look? http://logs.openstack.org/37/664537/7/check/tripleo-ci-centos-7-scenario012-multinode-oooq-container/f55b3a8/job-output.txt.gz#_2019-06-19_21_56_26_780680 | 09:43 |
| sshnaidm | it's with ansible 2.7 | 09:43 |
| arxcruz | sshnaidm i'm seeing this in tempest log | 09:45 |
| arxcruz | http://logs.openstack.org/37/664537/7/check/tripleo-ci-centos-7-scenario012-multinode-oooq-container/f55b3a8/logs/undercloud/home/zuul/tempest.log.txt.gz | 09:45 |
| arxcruz | 2019-06-19 21:56:23 | Node efdcac7e-753e-4740-a3ad-45d3596335a3 failed to reach state manageable. It's in unexpected stable state enroll | 09:46 |
| arxcruz | at this point it should fail, don't know why it continue with the tempest stuff | 09:46 |
| sshnaidm | arxcruz, seems like tempest dir should be created before.. | 09:47 |
| arxcruz | sshnaidm tempest dir is created after this openstack baremetal commands | 09:48 |
| arxcruz | http://logs.openstack.org/37/664537/7/check/tripleo-ci-centos-7-scenario012-multinode-oooq-container/f55b3a8/logs/undercloud/home/zuul/tempest-setup.sh.txt.gz | 09:48 |
| *** holser_ has quit IRC | 10:39 | |
| *** blueDuckRandom has joined #oooq | 10:42 | |
| *** blueDuckRandomSt has quit IRC | 10:44 | |
| *** blueDuckRandomSt has joined #oooq | 10:44 | |
| *** blueDuckRandom has quit IRC | 10:47 | |
| arxcruz | amoralej i think it's not working yet | 10:51 |
| *** ykarel|afk is now known as ykarel | 10:52 | |
| amoralej | arxcruz, this is what we tipically do when we add a project https://review.rdoproject.org/r/#/q/topic:add-puppet-rsyslog | 10:53 |
| amoralej | although those have the project-config in the config not in the node itself | 10:54 |
| amoralej | in the project itself, i mean | 10:54 |
| amoralej | i'm not sure if you need to to something else for zuul to read auul.yaml from the project | 10:55 |
| amoralej | jpena, ^ | 10:55 |
| jpena | arxcruz: with that change, the zuul configuration from the project is included. What do you want to run next? | 10:57 |
| jpena | I mean, if you need to run a job, that job definition has to be created somewhere | 10:58 |
| *** jpena is now known as jpena|lunch | 10:59 | |
| *** amoralej is now known as amoralej|lunch | 10:59 | |
| *** skramaja has quit IRC | 11:06 | |
| *** holser_ has joined #oooq | 11:08 | |
| *** ykarel has quit IRC | 11:08 | |
| *** ykarel has joined #oooq | 11:09 | |
| *** blueDuckRandomSt has quit IRC | 11:10 | |
| *** blueDuckRandom has joined #oooq | 11:12 | |
| *** rfolco_off has joined #oooq | 11:20 | |
| *** rfolco_off is now known as rfolco | 11:21 | |
| *** holser_ has quit IRC | 11:24 | |
| *** blueDuckRandom has quit IRC | 11:24 | |
| *** blueDuckRandom has joined #oooq | 11:25 | |
| *** holser_ has joined #oooq | 11:25 | |
| arxcruz | jpena|lunch I though this would make the zuul job execute, as when we add a new job on zuul.yaml it does run on the patch you're adding it | 11:26 |
| *** rlandy has joined #oooq | 11:30 | |
| rlandy | weshay:hi | 11:31 |
| ykarel | arxcruz, can u try renaming zuul.yaml to .zuul.yaml | 11:31 |
| rlandy | I left the intstruction to add the image | 11:31 |
| * rlandy goes to dr appt back later | 11:32 | |
| rlandy | rfolco: ^^ will miss the scrum meeting | 11:32 |
| *** rlandy is now known as rlandy|afk | 11:33 | |
| *** brault has quit IRC | 11:35 | |
| *** rfolco is now known as rfolco_pto | 11:36 | |
| *** udesale has quit IRC | 11:36 | |
| *** udesale has joined #oooq | 11:37 | |
| *** udesale has quit IRC | 11:58 | |
| *** udesale has joined #oooq | 11:59 | |
| *** holser_ has quit IRC | 12:02 | |
| *** tosky__ has joined #oooq | 12:07 | |
| *** tosky is now known as Guest73721 | 12:07 | |
| *** tosky__ is now known as tosky | 12:07 | |
| arxcruz | rlandy|afk today is holiday in brazil, i don't think folco will be available :) | 12:08 |
| arxcruz | ykarel i'll try | 12:09 |
| *** Guest73721 has quit IRC | 12:11 | |
| weshay | sshnaidm arxcruz this is good news http://logs.rdoproject.org/openstack-periodic-24hr/review.rdoproject.org/rdo-infra/ansible-role-tripleo-ci-reproducer/master/periodic-tripleo-ci-reproducer-centos-7-libvirt-standalone-vexxhost/44cb5b5/job-output.txt.gz | 12:12 |
| arxcruz | weshay you are too late, i already update the card with this information | 12:12 |
| weshay | sshnaidm any advice on the bmc-template issue I'm hitting, should I be using the public or shared one? | 12:13 |
| *** holser_ has joined #oooq | 12:15 | |
| *** amoralej|lunch is now known as amoralej | 12:15 | |
| *** jpena|lunch is now known as jpena | 12:17 | |
| sshnaidm | weshay, they should be same, what is the issue? | 12:18 |
| weshay | sshnaidm see my email | 12:19 |
| weshay | zbr renick, can you check if we have a bug on http://logs.rdoproject.org/openstack-periodic-latest-released/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-stein/9484d52/logs/undercloud/var/log/tripleo-container-image-prepare.log.txt.gz | 12:19 |
| zbr | sure | 12:22 |
| arxcruz | amoralej https://review.rdoproject.org/r/#/c/21210/ | 12:26 |
| arxcruz | i think that's the last missing piece | 12:26 |
| zbr | weshay: no bug, and apparently this file is not sent to logstash, so no idea about how spread it is. other similar errors are visible but not with trunk.registry.rdoproject.org | 12:28 |
| zbr | i think we need to sort this ASAP as is the n-th time when we cannot investigate because we do not send that log. | 12:29 |
| weshay | zbr https://review.rdoproject.org/zuul/status | 12:30 |
| weshay | openstack-periodic-latest-released | 12:30 |
| weshay | queue | 12:30 |
| weshay | http://logs.rdoproject.org/openstack-periodic-latest-released/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-7-scenario001-standalone-stein/33c2408/logs/undercloud/var/log/tripleo-container-image-prepare.log.txt.gz | 12:31 |
| arxcruz | ykarel https://review.rdoproject.org/r/#/c/21210/ | 12:31 |
| weshay | zbr give me a min for 1-1 | 12:32 |
| zbr | weshay: sure. apparently: Error determining manifest MIME type for docker | 12:34 |
| ykarel | arxcruz, ack | 12:34 |
| *** rfolco_pto has quit IRC | 12:35 | |
| arxcruz | ykarel thanks | 12:35 |
| sshnaidm | weshay, I saw mail and bug, I didn't see any issue with bmc template | 12:35 |
| zbr | podman pull fails with 404 on centos-binary-mariadb | 12:36 |
| weshay | sshnaidm the public one? | 12:38 |
| weshay | maybe it's my key | 12:39 |
| weshay | tripleo-ci-team key.. | 12:39 |
| weshay | sshnaidm refresh my memory re: that.. is that just my personal key renamed? | 12:39 |
| sshnaidm | weshay, sorry, I don't follow | 12:39 |
| weshay | Name: Error Validating Value 'Tripleo-Ci-Team': The Key (Tripleo-Ci-Team) Could Not Be Found | 12:40 |
| weshay | I see that error on a stack | 12:40 |
| sshnaidm | weshay, oh, you need to create it in your tenant | 12:41 |
| sshnaidm | weshay, it's by default | 12:41 |
| weshay | k .. and that's just what ever ssh pub key I am using for the repro but named TripleO-Ci-Team? | 12:42 |
| sshnaidm | weshay, can you paste a full log? | 12:42 |
| *** rfolco has joined #oooq | 12:42 | |
| weshay | sshnaidm I don't see the ovb log in /home/weshayutin/tripleo-ci-reproducer | 12:45 |
| weshay | perhaps create the key, and retry? | 12:45 |
| sshnaidm | weshay, yeah | 12:46 |
| weshay | k k | 12:46 |
| weshay | zbr 1-1? | 12:46 |
| sshnaidm | weshay, we inject it to bmc host | 12:46 |
| sshnaidm | just found it.. | 12:46 |
| *** Goneri has joined #oooq | 12:48 | |
| sshnaidm | weshay, you can try to pass to job key_name: "my_key_name" - https://github.com/rdo-infra/review.rdoproject.org-config/blob/master/roles/ovb-manage/defaults/main.yml#L14 | 12:49 |
| sshnaidm | I think it should be in reproducer script.. | 12:49 |
| weshay | zbr http://logs.rdoproject.org/openstack-periodic-latest-released/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-stein/9484d52/logs/undercloud/var/log/tripleo-container-image-prepare.log.txt.gz | 12:50 |
| weshay | zbr HTTPSConnectionPool(host='trunk.registry.rdoproject.org', port=443): Max retries exceeded with url: /v2/tripleostein/centos-binary-cron/manifests/8b366d2a278680f2248886ca82b08c7e320bc284_45a5b26e | 12:50 |
| weshay | zbr https://console.registry.rdoproject.org/registry | 12:53 |
| zbr | weshay: raised at https://bugs.launchpad.net/tripleo/+bug/1833557 | 12:57 |
| openstack | Launchpad bug 1833557 in tripleo "infra: unable to pull images from trunk.registry.rdoproject.org" [Critical,Confirmed] - Assigned to Sorin Sbarnea (ssbarnea) | 12:57 |
| *** udesale has quit IRC | 12:58 | |
| *** ykarel is now known as ykarel|afk | 13:24 | |
| *** ratailor_ has quit IRC | 13:27 | |
| * marios blue trouble reconnecting | 13:31 | |
| arxcruz | amoralej anything else i need to do? https://review.rdoproject.org/r/#/c/21203/ still not triggering the tox job... | 13:34 |
| *** blueDuckRandomSt has joined #oooq | 13:35 | |
| *** blueDuckRandom has quit IRC | 13:36 | |
| amoralej | arxcruz, mmmm | 13:38 |
| amoralej | could you abandon and enable it again? | 13:38 |
| arxcruz | sure | 13:39 |
| amoralej | mm last ps was after last merge, right? | 13:39 |
| amoralej | so it will probably not help much | 13:39 |
| arxcruz | amoralej I just did a dummy change to send a ps | 13:39 |
| *** rlandy|afk is now known as rlandy | 13:40 | |
| rlandy | chandankumar: hi ... | 13:41 |
| marios | sshnaidm: weshay: sorry i was confusing it with this Adds validation of release config files https://review.opendev.org/649965 I36df81e2c8839f54abf8556e011c026ea505fcdb so not sure we have molecule tripleo-repos | 13:41 |
| rlandy | chandankumar: got logging to rdocloud from downstream sf ... https://logs.rdoproject.org/55/174055/1/check/tripleo-ci-centos-7-standalone-internal-rocky/ee4fd28/ | 13:42 |
| amoralej | arxcruz, i see the project exists in zuul https://review.rdoproject.org/zuul/project/rdo-infra/ansible-role-tripleo-ci-zuul-runner | 13:42 |
| marios | weshay: sshnaidm actually it merged there https://review.opendev.org/#/c/651772/ | 13:43 |
| marios | ( Adds tripleo-repos molecule test job ) | 13:43 |
| arxcruz | amoralej i'll remove the change-id, and submit again as a new review | 13:43 |
| rlandy | periodic-tripleo-ci-rhel-8-standalone-need-images-rhos-15openstack/tripleo-ci-internal-jobsmasterperiodicmaster54072019-06-20T00:02:07SUCCESS | 13:44 |
| rlandy | tripleo-ci-rhel-8-standalone-rhos-15openstack/tripleo-ci-internal-jobsmasterperiodicmaster51862019-06-20T00:01:18SUCCESS | 13:44 |
| rlandy | yay - finally | 13:44 |
| ykarel|afk | weshay, zbr, is the f28 failures known already? i see it's failing randomly and affecting gate | 13:45 |
| amoralej | arxcruz, i don't thing that's related | 13:45 |
| arxcruz | amoralej didn't work anyway | 13:45 |
| arxcruz | amoralej i don't know what else should i do... | 13:45 |
| zbr | ykarel|afk: kinda-off, i observed it yesterday, apparently 40-50% chance of happeing, not sure why. | 13:45 |
| ykarel|afk | weshay, zbr i checked a little and think https://review.opendev.org/#/c/664872/1 could cause it | 13:46 |
| ykarel|afk | zbr, okk, can u filter failures based on my comment in ^^ | 13:46 |
| ykarel|afk | that will help in confirming the issue and fixing it | 13:46 |
| zbr | because the rate of failure is so high and because we are not allowed to merge fixes related to f28 I propose to make it non-voting. | 13:46 |
| amoralej | arxcruz, i'm asking jpena, he knows much more that me about zuul | 13:47 |
| amoralej | the point is that i think something is needed for projects to read the .zuul.yaml from the repo itself | 13:47 |
| amoralej | in rdo, we usually load jobs configuration from config project, not the repo itself | 13:47 |
| zbr | ykarel|afk: i made https://review.opendev.org/#/c/666283/ yesterday, do you have a failure that contains that new debugging code? | 13:48 |
| ykarel|afk | zbr, yes i seen failures today and ^^ was merged 10 hours back | 13:48 |
| zbr | mainly the execution of yum_update.sh was too opaque, and we were not able to find any explanation (or log). | 13:49 |
| ykarel|afk | zbr, u saw my comment? | 13:49 |
| *** tosky has quit IRC | 13:50 | |
| jpena | amoralej, arxcruz: I think the zuul configuration we have is enough, but let me see | 13:50 |
| arxcruz | jpena gracias! | 13:50 |
| amoralej | arxcruz, let me try to change .zuul.yaml | 13:51 |
| jpena | we can compare that configuration with what we have at https://github.com/rdo-infra/rdo-infra-playbooks/blob/master/.zuul.yaml | 13:51 |
| jpena | that's known to work | 13:51 |
| amoralej | arxcruz, running! :) | 13:52 |
| arxcruz | amoralej++ | 13:52 |
| panda | zbr: why you started to use os-release instead of platform | 13:52 |
| amoralej | arxcruz, if you want that job to run in a different node type you'll probably need some more work | 13:56 |
| amoralej | right now it's running in rdo-centos7 but seems to be working fine | 13:57 |
| weshay | sshnaidm ping.. do you have 5 min for me? | 13:57 |
| arxcruz | amoralej yeah, it's a work in progress :) thanks! | 13:57 |
| sshnaidm | weshay, yep | 13:58 |
| weshay | sshnaidm re: reproducer | 13:58 |
| *** rfolco has quit IRC | 13:58 | |
| sshnaidm | weshay, in blue? | 13:58 |
| weshay | sshnaidm ya | 13:58 |
| rlandy | rfolco: sshnaidm: did we change the internal playbooks to use the new mirror? | 13:59 |
| *** vinaykns has joined #oooq | 13:59 | |
| *** sanjayu_ has quit IRC | 13:59 | |
| rlandy | is it working for you? | 13:59 |
| sshnaidm | rlandy, I didn't | 13:59 |
| rlandy | sshnaidm: k - will look into testing that later | 13:59 |
| zbr | panda: i miss the context of os-release question | 14:04 |
| *** marios has quit IRC | 14:04 | |
| *** marios has joined #oooq | 14:06 | |
| rlandy | weshay: rhos-15 standalone and needs_images are back to passing now :) | 14:07 |
| rlandy | zbr: everything ok in ruck-land? | 14:07 |
| panda | zbr: in https://github.com/openstack/tripleo-repos/blob/master/tripleo_repos/main.py#L71 you replaced the use of platform with a call to bash. I remember you mentioning it, but I don't remember why you did it | 14:09 |
| *** sanjayu_ has joined #oooq | 14:12 | |
| weshay | sshnaidm http://logs.rdoproject.org/70/665970/1/openstack-check/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001/f6cb07e/logs/reproducer-quickstart/ | 14:14 |
| weshay | sshnaidm http://codesearch.openstack.org/?q=Synchronize%20src%20repos%20to%20workspace%20directory&i=nope&files=&repos= | 14:15 |
| chandankumar | rlandy: I checked the logs in the morning did not find any sensitive ips | 14:18 |
| chandankumar | not looked at rpm packages stuff | 14:18 |
| ykarel|afk | zbr, /me leaving u can debug more and if u have some conclusion comment on the review itself, i can check, i also notified the author jpichon on #rhos-delivery | 14:19 |
| rlandy | chandankumar: was just showing you that the option is available- you can choose to add it to whatever jobs you are comfortable with | 14:20 |
| chandankumar | rlandy: I want to gate os_tempest project with that job | 14:21 |
| zbr | ykarel|afk: rlandy: this should fix f28 repoquery: https://review.opendev.org/#/c/666589/1 --- you were right | 14:21 |
| chandankumar | if it is ok? | 14:21 |
| zbr | weshay: should I file a bug for f28 bug or ^ fix should be enough? | 14:21 |
| weshay | zbr fix is good enough.. please renick though | 14:22 |
| ykarel|afk | zbr, seems a hack as with this some updates will be missed | 14:22 |
| ykarel|afk | from delorean-current | 14:22 |
| rlandy | chandankumar: yeah - it was juts an example | 14:23 |
| rlandy | just | 14:23 |
| weshay | arxcruz question for you when you have a sec | 14:23 |
| arxcruz | weshay sure boss | 14:23 |
| arxcruz | weshay bj or irc ? | 14:24 |
| arxcruz | sshnaidm https://review.rdoproject.org/r/#/c/21212/ | 14:24 |
| weshay | arxcruz as you have time.. can you please try to put up the same reproducer job as you just did, but have it run the ovb fs001 workflow? | 14:25 |
| arxcruz | weshay okay | 14:25 |
| weshay | arxcruz thanks.. | 14:25 |
| * ykarel|afk out | 14:25 | |
| arxcruz | weshay that's all ? | 14:26 |
| zbr | ykarel|afk: while I hate the entire script (as I would have prefered the standard wide yum update all), I think that his hack should be ok, because it only affects available_versions - which would endup empty. | 14:26 |
| ykarel|afk | wide yum update in the past resulted in timeout | 14:26 |
| weshay | zbr ykarel|afk is correct there.. we have to limit the updates | 14:26 |
| arxcruz | weshay you mean the ci-reproducer right, not the zuul-runner | 14:27 |
| arxcruz | just to make sure :) | 14:27 |
| weshay | arxcruz right.. so atm.. the ci-reproducer.. as you get the zuul-runner work in.. I would expect to see periodic jobs testing that as well | 14:27 |
| arxcruz | weshay sure | 14:28 |
| weshay | arxcruz we need daily feedback on the quality of this reproducer whether it be container or zuul-runner | 14:28 |
| arxcruz | weshay ack | 14:28 |
| weshay | thank you!! | 14:28 |
| *** ykarel|afk is now known as ykarel|away | 14:29 | |
| zbr | panda: re source os-release: that was because /etc/os-release provides more info than python os library, and allows us to match behavior of ansible (which is looking at the file). Using only python API to get OS information does prevent us from getting some things which are only inside release files. | 14:29 |
| * ykarel|away out | 14:29 | |
| chandankumar | rlandy: we have jobs for rhos-13 and 14 for downstream sf based on tripleo running tempest? | 14:31 |
| chandankumar | it is needed for rhos-13 & rhos-14 patches branches also | 14:32 |
| rlandy | chandankumar: we have a rhos-14 job | 14:33 |
| rlandy | and rhos-15 | 14:33 |
| rlandy | I have not added any rhos-13 jobs | 14:33 |
| rlandy | if I am understanding you correctly | 14:34 |
| *** ykarel|away has quit IRC | 14:34 | |
| *** sanjayu_ has quit IRC | 14:35 | |
| weshay | zbr please renick dude | 14:39 |
| weshay | zbr|ruck | 14:39 |
| panda | amoralej: there's no trunk.rdoproject.org/centos8-master | 14:39 |
| panda | :) | 14:39 |
| weshay | panda jpena I'm fine to rename rhel8-master to centos8-master if you guys want to :) | 14:40 |
| weshay | panda jpena amoralej we may consider el8 | 14:41 |
| chandankumar | rlandy: good, we will use rhos-14 one | 14:41 |
| weshay | as rhel will be a perm fixture in 3rd party henceforth | 14:41 |
| jpena | I wouldn't use el8 unless we're sure we will be using a single builder for both | 14:41 |
| jpena | we can symlink centos8-master to rhel8-master until we have a proper centos8 builder, if that helps | 14:42 |
| weshay | amoralej panda ^ | 14:42 |
| weshay | marios need any help w/ anything? | 14:43 |
| panda | weshay: jpena rhel8 is fine for me. too many places in which we combine DISTRO + REL to assemble the url for delorean repo | 14:44 |
| weshay | k | 14:44 |
| panda | we would need to create an exception | 14:44 |
| marios | weshay: well, pinged mburns will also ping apevec (re the dependency issue) | 14:44 |
| *** zbr is now known as zbr|ruck | 14:44 | |
| marios | weshay: so don't know if there is somethign more we can do right now re that and then the containers/docker/podman thing (filing bug for that before i go) | 14:45 |
| weshay | k | 14:45 |
| weshay | marios I updated https://trello.com/c/27kSfSs4/1014-cixlp1833209tripleociproa-rdotrunkrhel8-dependency-errors-for-overcloud-image-build | 14:48 |
| marios | weshay: ack thanks looking | 14:48 |
| marios | weshay: https://bugs.launchpad.net/tripleo/+bug/1833570 | 14:52 |
| openstack | Launchpad bug 1833570 in tripleo "[rdotrunk][rhel8] ansible-role-container-registry doesn't work on rhel 8" [High,Triaged] - Assigned to Marios Andreou (marios-b) | 14:52 |
| marios | weshay: should i create a trello for that one too then? | 14:52 |
| marios | weshay: hmm maybe not | 14:53 |
| marios | weshay: as in | 14:53 |
| weshay | marios create a bug, mark it alert.. we're the only ones that can fix that | 14:53 |
| marios | weshay: that one is first more on us to look into and get fixed/ | 14:53 |
| marios | weshay: right ack | 14:53 |
| weshay | so it's just a waste to escalate | 14:53 |
| weshay | imho | 14:53 |
| panda | jpena: is this intentional ? https://trunk.rdoproject.org/rhel8-master/delorean-deps.repo | 14:53 |
| *** tosky__ has joined #oooq | 14:54 | |
| *** tosky__ is now known as tosky | 14:54 | |
| amoralej | weshay, panda the point is that it's actually not centos8 :) | 14:54 |
| amoralej | panda, yes it's intentional | 14:54 |
| weshay | k k | 14:54 |
| amoralej | we can do it for convenience | 14:55 |
| amoralej | if it helps | 14:55 |
| amoralej | but at some point, when we have proper centos8 DLRN builder, we will create a centos8-master one | 14:55 |
| jpena | panda: hm, I'll fix that, it should point to the public URL | 14:55 |
| amoralej | until then, if it helps to create a symlink rhel8-master pointing to centos8-repo, i'm ok with it | 14:56 |
| panda | jpena: ah, ok. also is osp15 the right name ? | 14:56 |
| panda | even the right naming ... | 14:56 |
| jpena | panda: it doesn't matter too much, but I'll change it too | 14:57 |
| panda | that's ok, just asking | 14:57 |
| jpena | it got that name because we started with osp-built deps | 14:57 |
| panda | jpena: last thing: rhel8/ should point to rhel8-master. Like before, I think too many scripts are assuming that master link doesn't have the suffix. I could change ii in some places, but I guess some others will break. | 14:59 |
| * jpena sighs at technical debt | 15:08 | |
| * panda joins jpena in sighing | 15:08 | |
| panda | jpena: https://static.tvtropes.org/pmwiki/pub/images/a76c6e55cb369c6ab98e187afb9bed6d.jpg | 15:10 |
| *** jfrancoa has quit IRC | 15:16 | |
| * marios almost out | 15:22 | |
| *** kopecmartin is now known as kopecmartin|off | 15:26 | |
| weshay | panda on that front.. isn't adding rhel8.0 to tripleo-repos instant debt? | 15:27 |
| *** Vorrtex has joined #oooq | 15:28 | |
| sshnaidm | weshay, this task "syncronise.." failed for me twice, but worked last time and job continues to run | 15:31 |
| weshay | sshnaidm ah nice | 15:31 |
| weshay | sshnaidm I'm trying a standalone rdo-cloud job right.. I would think the same thing could occur | 15:31 |
| sshnaidm | weshay, yeah, it's a task in any job | 15:32 |
| weshay | sshnaidm did you try that same review? matbu may be looking to use the env if you can get thing running | 15:33 |
| sshnaidm | weshay, https://github.com/openstack-infra/zuul-jobs/blob/master/roles/prepare-workspace/tasks/main.yaml#L5 | 15:33 |
| sshnaidm | weshay, yeah, it should be same review | 15:33 |
| weshay | sshnaidm I wonder if that task needs a retry | 15:33 |
| sshnaidm | weshay, yeah, maybe | 15:33 |
| weshay | that module can be suspect... synchronize that is | 15:34 |
| weshay | sshnaidm the waiting on logger spam is terrible | 15:38 |
| *** marios has quit IRC | 15:46 | |
| weshay | sshnaidm standalone is just hanging for me | 15:49 |
| sshnaidm | weshay, hmm.. my job is running fine | 15:50 |
| sshnaidm | weshay, I'm afraid it's related to some network issues.. | 15:51 |
| *** ykarel|away has joined #oooq | 15:51 | |
| weshay | ya.. could be | 15:51 |
| sshnaidm | weshay, can you try it from another host on rdo cloud itself? | 15:51 |
| weshay | ya | 15:51 |
| *** ykarel|away has quit IRC | 15:52 | |
| *** fmount- has joined #oooq | 15:56 | |
| *** fmount has quit IRC | 16:03 | |
| *** blueDuckRandom has joined #oooq | 16:08 | |
| *** blueDuckRandomSt has quit IRC | 16:10 | |
| rlandy | brb | 16:12 |
| zbr|ruck | out, returning in ~1h | 16:22 |
| *** ccamacho has quit IRC | 16:25 | |
| *** amoralej is now known as amoralej|off | 16:44 | |
| weshay | k | 16:48 |
| *** panda has quit IRC | 16:50 | |
| *** panda has joined #oooq | 16:52 | |
| rlandy | weshay: you all set with the ovb image? | 16:54 |
| weshay | rlandy ya.. still have issues but still working it | 16:55 |
| weshay | rlandy did you need to chat about perf/brow | 16:55 |
| rlandy | yeah - but maybe in about an hour - if that works? | 16:56 |
| weshay | that's fine | 16:56 |
| weshay | rlandy you see that travel planning spreadsheet? | 16:56 |
| rlandy | I'm learning the rrcockpit setup and sort of in the middle | 16:56 |
| rlandy | yeah - will fill it out | 16:56 |
| rlandy | why would we need a hotel for 6 days fora 2 day meetup? | 16:57 |
| *** derekh has quit IRC | 16:58 | |
| *** tesseract has quit IRC | 17:14 | |
| *** tesseract has joined #oooq | 17:15 | |
| *** jpena is now known as jpena|off | 17:15 | |
| weshay | oh .. it's in brno | 17:16 |
| weshay | rlandy you ok w/ that re: food? | 17:16 |
| weshay | rlandy most people will probably fly in for a week | 17:16 |
| weshay | it's a long trip | 17:16 |
| rlandy | weshay: food, I'll figure out - I'm used to living on granola bars for a few days - I need to figure out shabbat | 17:17 |
| *** tesseract has quit IRC | 17:59 | |
| *** tesseract has joined #oooq | 18:01 | |
| sshnaidm | weshay, ocv job in reproducer failed on overcloud: "Ansible timed out at 3036 seconds" | 18:25 |
| sshnaidm | s/ocv/ovb | 18:25 |
| weshay | sshnaidm in the overcloud? | 18:25 |
| sshnaidm | weshay, yeah | 18:25 |
| weshay | sshnaidm ya.. something seems up w/ rdo-cloud | 18:25 |
| sshnaidm | o, yeah.. | 18:26 |
| sshnaidm | maybe I'll try in vexxhost :) | 18:26 |
| sshnaidm | I have credentials :P | 18:26 |
| weshay | sshnaidm vexhost planning on monday at least | 18:26 |
| sshnaidm | weshay, btw, 002 jobs with 1ctrl ran not bad on upshift, passed but very slow like 3-4 hours | 18:27 |
| zbr|ruck | weshay: do you have few minutes to sync. | 18:27 |
| weshay | zbr|ruck I do | 18:28 |
| weshay | sshnaidm fyi. .have one going http://38.145.35.187:9000/t/tripleo-ci-reproducer/status | 18:28 |
| weshay | but it's sloooooo | 18:28 |
| zbr|ruck | weshay: https://zuul.opendev.org/t/openstack/status 666589 | 18:30 |
| *** tesseract has quit IRC | 18:37 | |
| *** tesseract has joined #oooq | 18:38 | |
| *** gkadam has quit IRC | 18:43 | |
| weshay | zbr|ruck https://sf.hosted.upshift.rdu2.redhat.com/logs/35/173835/3/check/periodic-tripleo-ci-rhel-8-standalone-need-images-rhos-15/f606a4f/logs/undercloud/var/log/tripleo-container-image-prepare.log.txt.gz | 18:51 |
| zbr|ruck | weshay: https://review.opendev.org/#/c/664947/ | 19:00 |
| rlandy | weshay: have time now? later? to review perf tests? | 19:02 |
| weshay | rlandy ya.. wrapping up w/ zbr|ruck but you can join | 19:02 |
| rlandy | k | 19:02 |
| *** Vorrtex has quit IRC | 19:52 | |
| *** matbu has quit IRC | 20:25 | |
| *** matbu has joined #oooq | 20:35 | |
| rlandy | weshay: does this seem right to you ... | 20:50 |
| rlandy | DomainNetworkCIDRMaskGateway | 20:50 |
| rlandy | perf.lab.eng.rdu2.redhat.com10.1.184.0/22255.255.252.010.1.187.254 | 20:50 |
| rlandy | ... /22 CIDR with that mask | 20:51 |
| rlandy | oh actually ok | 20:52 |
| *** jtomasek has quit IRC | 21:10 | |
| * weshay looks | 21:12 | |
| weshay | rlandy https://mxtoolbox.com/subnet/?filter=10.1.184.0/22&source=subnettool | 21:15 |
| weshay | Range: 10.1.184.0 - 10.1.187.255 IP Addresses: | 21:15 |
| weshay | 1024 Mask: | 21:15 |
| weshay | 255.255.252.0 /22 | 21:15 |
| weshay | rlandy so ya.. 10.1.187.254 would typically be the router, the last addr | 21:16 |
| *** tosky has quit IRC | 21:49 | |
| *** sshnaidm is now known as sshnaidm|off | 21:55 | |
| *** vinaykns has quit IRC | 22:24 | |
| *** tesseract has quit IRC | 22:35 | |
| *** rlandy has quit IRC | 23:44 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!
