| rlandy|rover | I guess they are both going tomorrow then | 00:14 |
|---|---|---|
| *** ysandeep|away is now known as ysandeep | 04:09 | |
| *** ysandeep is now known as ysandeep|brb | 05:23 | |
| *** ysandeep|brb is now known as ysandeep | 05:35 | |
| *** pojadhav- is now known as pojadhav | 07:00 | |
| pojadhav | ysandeep, hello | 07:01 |
| * ysandeep will ping back in few min, writing a bug | 07:02 | |
| *** amoralej|off is now known as amoralej | 07:03 | |
| ysandeep | bhagyashris: hey o/ both ruck/rovers are in NA timezone, Is someone covering apac/emea hour ? | 07:04 |
| bhagyashris | ysandeep, nope | 07:08 |
| ysandeep | ack o/ | 07:10 |
| ysandeep | weshay|ruck, dviroel|out hey o/ https://bugs.launchpad.net/tripleo/+bug/1940897 , yesterday's periodic run were mostly skipped because of which I couldn't confirm if this is hitting periodic jobs as well or just check.. Please mark promotion blocker if needed. | 07:11 |
| ysandeep | Feature039 is failing during repo setup, Error: "Failed to set locale, defaulting to C , RPM: error: db5 error(-30969) from dbenv->open: BDB0091 DB_VERSION_MISMATCH: Database environment version mismatch" | 07:13 |
| ysandeep | chandankumar: ^^ fyi.. | 07:14 |
| ysandeep | pojadhav, pong | 07:17 |
| pojadhav | ysandeep, https://sf.hosted.upshift.rdu2.redhat.com/zuul/t/tripleo-ci-internal/builds?pipeline=openstack-periodic-integration-rhos-16.2 can see lot of jobs got fail.. standalone and scenario jobs are failed with connetion timedout to "docker-registry.upshift.redhat.com:443". | 07:18 |
| pojadhav | ysandeep, i thought you are filing the same bug, but that was different one. | 07:19 |
| ysandeep | pojadhav: ack thanks, I will check after lunch | 07:21 |
| pojadhav | ysandeep, yup :) | 07:21 |
| *** ysandeep is now known as ysandeep|lunch | 07:29 | |
| *** jpena|off is now known as jpena | 07:35 | |
| *** ykarel is now known as ykarel|lunch | 08:34 | |
| chandankumar | ysandeep|lunch: looking on that one | 09:03 |
| *** ysandeep|lunch is now known as ysandeep | 09:35 | |
| chandankumar | ysandeep: hello, I am putting a DNM patch with workaroud for fs039 | 09:38 |
| ysandeep | chandankumar: thanks! | 09:38 |
| ysandeep | chandankumar: regarding your comment if its fs039 only: yesterday's periodic run were mostly skipped(i think those were skipped due to infra issues) because of which I couldn't confirm which all jobs are truly impacted... I reviewed openstack-check pipeline on high level - I couldn't find any other fs failure.. | 09:45 |
| ysandeep | but again i didn't check every failure.. there might be some.. | 09:45 |
| arxcruz | frenzy_friday: did some package update recently on er? I had to add some packages to build greenlet and also, now i'm getting http error on rdoproject.org | 09:50 |
| arxcruz | zbr: ^ | 09:50 |
| ysandeep | pojadhav: hey o/ i am going to take a look at that 16.2 failure, If you want to know how to debug that failure.. I can show over gmeet | 09:51 |
| pojadhav | ysandeep, sure. lets do | 09:51 |
| zbr | arxcruz: is this related to er in any way? | 09:52 |
| chandankumar | ysandeep: jobs will finished after 45 mins have the above issue | 09:52 |
| arxcruz | zbr: probably | 09:52 |
| frenzy_friday | arxcruz, I didnt add anything, what error are you getting? | 09:52 |
| ysandeep | pojadhav, meet.google.com/mny-snen-qpc | 09:53 |
| arxcruz | elasticsearch.exceptions.ConnectionTimeout: ConnectionTimeout caused by - ReadTimeoutError(HTTPSConnectionPool(host='review.rdoproject.org', port=443): Read timed out. | 09:53 |
| bhagyashris | folks plz have a lokk when you have free time https://review.rdoproject.org/r/c/config/+/35073 | 09:54 |
| bhagyashris | look* | 09:55 |
| frenzy_friday | arxcruz, elastic_recheck/results.py line 100 params = {"size": size, "request_timeout": 40} should have fixed that. There are some queries for which it takes more than the default timeout to get a response. I added 40 to fix. Check if changing 40 to maybe 60 works? | 09:56 |
| bhagyashris | chandankumar, ^ | 09:56 |
| chandankumar | bhagyashris: ack | 09:56 |
| arxcruz | frenzy_friday: it is 40 | 09:57 |
| arxcruz | i'll update to 100 to check | 09:57 |
| arxcruz | maybe is my network | 09:57 |
| bhagyashris | chandankumar, have a sec to chat ? | 10:05 |
| chandankumar | bhagyashris: sure | 10:05 |
| bhagyashris | https://meet.google.com/xnf-tvdh-pmk?authuser=0 | 10:05 |
| *** ykarel|lunch is now known as ykarel | 10:23 | |
| *** ykarel_ is now known as ykarel | 10:56 | |
| *** ysandeep is now known as ysandeep|afk | 10:58 | |
| *** rlandy is now known as rlandy|rover | 11:02 | |
| rlandy|rover | bhagyashris: hey | 11:04 |
| rlandy|rover | bhagyashris: just wanted to touch base on the centos stream project if you have time to chat | 11:04 |
| rlandy|rover | pojadhav: ysandeep|afk: hello - how's 17 this morning? | 11:05 |
| arxcruz | zbr: frenzy_friday er-cron | /usr/lib/python3.8/site-packages/elastic_recheck/results.py:43: ElasticsearchWarning: The client is unable to verify that the server is Elasticsearch due security privileges on the | 11:05 |
| arxcruz | server side | 11:05 |
| rlandy|rover | arxcruz: hey - we'll merge your patch today | 11:05 |
| rlandy|rover | ping if we have not | 11:06 |
| rlandy|rover | :) | 11:06 |
| arxcruz | rlandy|rover: sure | 11:06 |
| arxcruz | rlandy|rover: it's gtg | 11:06 |
| bhagyashris | rlandy|rover, sure | 11:07 |
| rlandy|rover | bhagyashris: one sec - just pinging infra | 11:07 |
| bhagyashris | rlandy|rover, ok np | 11:07 |
| rlandy|rover | bhagyashris, https://meet.google.com/exk-btgd-exi?pli=1&authuser=0 | 11:10 |
| rlandy|rover | bhagyashris: https://hackmd.io/iYAL3ueUTzWCHE5qFA6jVQ | 11:12 |
| frenzy_friday | arxcruz, unable to verify that the server is Elasticsearch due security privileges warning is normal. It logs the exception but does not raise it | 11:13 |
| pojadhav | rlandy|rover, for 17 due to build push job failure, rest all got skipped in previous run. and for 16.2 there failures but related to connection timed out to "docker-registry.upshift.redhat.com:443" | 11:15 |
| rlandy|rover | pojadhav: sec - one meeting | 11:15 |
| pojadhav | ok | 11:15 |
| arxcruz | frenzy_friday: after that i got exception with readtimeout | 11:16 |
| *** dviroel|out is now known as dviroel|ruck | 11:23 | |
| frenzy_friday | checking | 11:24 |
| bhagyashris | rlandy|rover, https://review.rdoproject.org/r/c/config/+/34709/6/zuul.d/tripleoci.yaml | 11:26 |
| bhagyashris | rlandy|rover, https://review.rdoproject.org/r/c/config/+/35025 | 11:27 |
| bhagyashris | rlandy|rover, https://review.rdoproject.org/r/c/testproject/+/35017 | 11:29 |
| bhagyashris | rlandy|rover, https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/35069 | 11:32 |
| bhagyashris | rlandy|rover, https://review.rdoproject.org/r/c/config/+/35073 | 11:34 |
| *** jpena is now known as jpena|lunch | 11:36 | |
| rlandy|rover | pojadhav: vexxhost had trouble - we might be seeing the end of that | 11:57 |
| pojadhav | rlandy|rover, ok..I have started testrun https://code.engineering.redhat.com/gerrit/c/testproject/+/267410 and https://code.engineering.redhat.com/gerrit/c/testproject/+/267406 | 11:58 |
| *** ysandeep|afk is now known as ysandeep | 12:00 | |
| * pojadhav brb - taking tea break | 12:01 | |
| *** pojadhav is now known as pojadhav|brb | 12:01 | |
| dviroel|ruck | ysandeep: chandankumar tks for openning and investigating the bug | 12:02 |
| dviroel|ruck | chandankumar: will recheck your testproject, it end up in a different error this time | 12:04 |
| arxcruz | frenzy_friday: please, can you try to build from scratch the containers and bring it up? | 12:11 |
| arxcruz | mine isn't working without modifications, for example | 12:11 |
| arxcruz | gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fomit-frame-pointer -g -fno-semantic-interposition -fomit-frame-pointer -g -fno-semantic-interposition -fomit-frame-pointer -g -fno-semantic-interposition -DTHREAD_STACK_SIZE=0x100000 -fPIC -I/usr/include/python3.8 -c src/greenlet/greenlet.c -o build/temp.linux-x86_64-3.8/src/greenlet/greenlet.o | 12:11 |
| arxcruz | #11 17.01 In file included from src/greenlet/greenlet.c:11: | 12:11 |
| arxcruz | #11 17.01 src/greenlet/greenlet.h:8:10: fatal error: Python.h: No such file or directory | 12:11 |
| arxcruz | zbr: ^ | 12:11 |
| frenzy_friday | arxcruz, lemme clean 38.102.83.141 and try | 12:13 |
| frenzy_friday | arxcruz, do you have access to centos@38.102.83.141? (with your key) | 12:15 |
| * weshay|ruck shakes fist at docker.io | 12:18 | |
| rlandy|rover | dviroel|ruck: weshay|ruck: pinged security team re: fs035 bug | 12:19 |
| weshay|ruck | thanks... docker.io rates and plans may have changed again.. trying to figure this out | 12:19 |
| *** pojadhav|brb is now known as pojadhav | 12:19 | |
| weshay|ruck | ah.. guess it worked... rlandy|rover https://hub.docker.com/u/tripleoyoga and will create https://hub.docker.com/u/tripleoyogacentos8 | 12:21 |
| weshay|ruck | and tripleomastercentos9 | 12:22 |
| weshay|ruck | ? | 12:22 |
| weshay|ruck | default for yoga should be 9 when we're all said and done I think | 12:22 |
| weshay|ruck | maybe I'll run it by the team | 12:22 |
| dviroel|ruck | rlandy|rover: still having vexxhost issues? | 12:23 |
| rlandy|rover | dviroel|ruck: the integration lines were stuck | 12:23 |
| rlandy|rover | asked infra to clear them | 12:23 |
| dviroel|ruck | i still see some failures atm | 12:24 |
| weshay|ruck | FYI all.. http://lists.openstack.org/pipermail/openstack-discuss/2021-August/024363.html | 12:24 |
| rlandy|rover | dviroel|ruck: weshay|ruck: there were also issues accessing dlrn | 12:24 |
| rlandy|rover | yay | 12:25 |
| dviroel|ruck | rlandy|rover: ah, ok, just saw this https://review.rdoproject.org/zuul/status#35068 failing a lot with timeouts | 12:26 |
| weshay|ruck | rlandy|rover, https://review.opendev.org/c/openstack/election/+/805840 | 12:26 |
| weshay|ruck | all ^ | 12:26 |
| rlandy|rover | weshay|ruck: we +1 vote? | 12:27 |
| rlandy|rover | that's all? | 12:27 |
| rlandy|rover | do we reply to the email? | 12:27 |
| rlandy|rover | done | 12:27 |
| arxcruz | frenzy_friday: yes i do, i need a few minutes, i'll take my daughter in the school | 12:28 |
| arxcruz | brb | 12:28 |
| frenzy_friday | cool | 12:28 |
| rlandy|rover | we may need to clean up the tenant | 12:29 |
| rlandy|rover | nice | 12:29 |
| rlandy|rover | weshay|ruck: thanks on https://hub.docker.com/u/tripleoyoga | 12:29 |
| rlandy|rover | 8 and 9 | 12:29 |
| weshay|ruck | rlandy|rover, I think.. mastercentos9 and yoga wallabycentos9 | 12:30 |
| rlandy|rover | dviroel|ruck: yeah - we should watch how jobs go from now | 12:30 |
| rlandy|rover | if they still have post failures, that's dlrn reporting jpena is looking into those | 12:30 |
| rlandy|rover | weshay|ruck: ack | 12:31 |
| rlandy|rover | dviroel|ruck: not sure if the molecule jobs are a separate issue | 12:31 |
| rlandy|rover | checking stack status | 12:31 |
| rlandy|rover | pojadhav: yay yay | 12:33 |
| rlandy|rover | scenraio010 periodic-tripleo-ci-rhel-8-scenario010-standalone-rhos-17 | 12:33 |
| *** jpena|lunch is now known as jpena | 12:34 | |
| rlandy|rover | pojadhav: ysandeep: weshay|ruck; chandankumar: dviroel|ruck: going to promote rhos-17 : http://osp-trunk.hosted.upshift.rdu2.redhat.com/api-rhel8-osp17/api/civotes_agg_detail.html?ref_hash=22cde743f0258a0a9ec18e8bf825b96c | 12:37 |
| rlandy|rover | thanks for helping get that straight | 12:38 |
| chandankumar | aye | 12:38 |
| rlandy|rover | chandankumar: hey - can you me on downstream promoter? | 12:43 |
| rlandy|rover | failing promoting 17 | 12:43 |
| rlandy|rover | DEBUG:paramiko.transport.sftp:[chan 0] listdir(b'/var/www/html/images/rhel8/rhos-17/rdo_trunk') | 12:44 |
| rlandy|rover | 2021-08-24 12:44:51,858 2452513 ERROR promoter Qcow-client: Image root dir %s does not exist in the server, or is not accessible | 12:44 |
| *** chem is now known as Guest5258 | 12:46 | |
| sshnaidm | is this error known? 2021-08-24 11:23:58,823 ERROR:dlrn:Known error building packages for tripleo-ansible, will retry later https://logserver.rdoproject.org/69/804869/7/openstack-check/tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001/2527452/logs/undercloud/home/zuul/dlrn.log.txt.gz | 12:49 |
| sshnaidm | jpena, amoralej ^^ | 12:49 |
| jpena | sshnaidm: in that case you need to check the package build log, which is https://logserver.rdoproject.org/69/804869/7/openstack-check/tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001/2527452/logs/undercloud/home/zuul/DLRN/data/repos/component/tripleo/f8/fb/f8fb95e78bf6af82676a5a67964b6cd83b1327fe_dev/rpmbuild.log.txt.gz | 12:50 |
| sshnaidm | oh, it's some problem with proxies: https://logserver.rdoproject.org/69/804869/7/openstack-check/tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001/2527452/logs/undercloud/home/zuul/DLRN/data/repos/component/tripleo/f8/fb/f8fb95e78bf6af82676a5a67964b6cd83b1327fe_dev/root.log.txt.gz | 12:50 |
| sshnaidm | Curl error (7): Couldn't connect to server for https://trunk.rdoproject.org/centos8-wallaby/deps/latest | 12:50 |
| jpena | I think it's a transient error with the trunk.rdo network, a recheck should do | 12:50 |
| sshnaidm | ack | 12:51 |
| rlandy|rover | self.images_dir = self.images_dir.rstrip("/rdo_trunk") | 12:56 |
| rlandy|rover | should strip that | 12:56 |
| amoralej | sshnaidm, what jpena said, transient issue with trunk.rdo | 12:57 |
| *** amoralej is now known as amoralej|lunch | 12:58 | |
| weshay|ruck | FYI.. I have a conflict today w/ community call rlandy|rover | 13:00 |
| rlandy|rover | weshay|ruck: k - np | 13:00 |
| ysandeep | rlandy|rover: ack thanks for 17 promotion, I had to recheck your sc010 testpatch couple of times, but it finally came through | 13:00 |
| rlandy|rover | ysandeep: finally | 13:00 |
| rlandy|rover | but promoter is broken | 13:01 |
| rlandy|rover | trying to fix that | 13:01 |
| arxcruz | frenzy_friday: zbr https://meet.google.com/ezs-bktf-jzg | 13:01 |
| weshay|ruck | rlandy|rover, chandankumar created https://hackmd.io/H4esaBdPSZOTSWp17MP-NA | 13:21 |
| bhagyashris | arxcruz, zbr, sshnaidm, rlandy|rover , marios, ysandeep, bhagyashris, svyas, soniya29, pojadhav, akahat, weshay|ruck , chandankumar, frenzy_friday, dviroel|ruck | 13:24 |
| bhagyashris | Tripleo CI community call in 6 mins | 13:24 |
| bhagyashris | feel free to add agenda here @line no 32 https://hackmd.io/MMg4WDbYSqOQUhU2Kj8zNg?both#2021-08-24-Community-Call | 13:25 |
| bhagyashris | https://meet.google.com/bqx-xwht-wky?authuser=0 | 13:25 |
| rlandy|rover | pojadhav: ysandeep: chandankumar: weshay|ruck: dviroel|ruck: downstream promoter for 17 is broken | 13:26 |
| rlandy|rover | got it promoting half way but it hangs on tripleo component | 13:26 |
| rlandy|rover | will debug more after communitry call | 13:26 |
| rlandy|rover | or use old promoter to get this hash through | 13:26 |
| bhagyashris | ysandeep, hey community call | 13:32 |
| *** ykarel is now known as ykarel|afk | 13:37 | |
| zbr | arxcruz: frenzy_friday lets merge https://review.opendev.org/c/opendev/elastic-recheck/+/805858 | 13:57 |
| zbr | i already opened patch to wrong branch | 13:58 |
| frenzy_friday | ^ looks good | 13:58 |
| *** amoralej|lunch is now known as amoralej | 13:58 | |
| rlandy|rover | pojadhav: hey - let's chat about components | 13:59 |
| pojadhav | rlandy|rover, yup | 13:59 |
| rlandy|rover | pojadhav, : https://meet.google.com/szx-fdhy-ojw?pli=1&authuser=0 | 13:59 |
| chandankumar | zbr: ttps://review.opendev.org/c/openstack/tripleo-quickstart/+/791486 please have a look | 14:10 |
| chandankumar | I will verify the tripleo repos data tomorrow | 14:11 |
| rlandy|rover | dviroel|ruck: rdo still having issues? | 14:22 |
| dviroel|ruck | rlandy|rover: don't think so, didn't see any new failures | 14:23 |
| rlandy|rover | k - let's keep watch | 14:23 |
| rlandy|rover | weshay|ruck: dviroel|ruck: arxcruz: can we merge the skiplist patch now? | 14:23 |
| weshay|ruck | yup | 14:24 |
| rlandy|rover | can we handle fs001 being impacted atm? | 14:24 |
| rlandy|rover | k so we need to merge this one first: | 14:24 |
| rlandy|rover | https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/804396 | 14:25 |
| rlandy|rover | done | 14:25 |
| rlandy|rover | and then | 14:25 |
| rlandy|rover | cool - see you w+'ed | 14:26 |
| rlandy|rover | thanks | 14:26 |
| rlandy|rover | bhagyashris: cool https://review.rdoproject.org/r/c/config/+/35073 merged | 14:28 |
| rlandy|rover | need anything else atm? | 14:28 |
| bhagyashris | rlandy|rover, yup hit the testproject patch https://review.rdoproject.org/r/c/testproject/+/35017 but failing with "msg": "Authentication failed." here https://logserver.rdoproject.org/17/35017/13/check/periodic-tripleo-centos-8-promote-latest-compose/15b0edc/job-output.txt | 14:29 |
| bhagyashris | rlandy|rover, looking into it | 14:29 |
| *** ysandeep is now known as ysandeep|away | 14:30 | |
| rlandy|rover | hmm do you need vars: | 14:30 |
| rlandy|rover | compose_promoter_key_path: '~/.ssh/id_rsa' | 14:30 |
| rlandy|rover | shoulld just use secret | 14:30 |
| * dviroel|ruck looks | 14:30 | |
| bhagyashris | rlandy|rover, no i was just trying | 14:30 |
| rlandy|rover | k | 14:31 |
| bhagyashris | dviroel|ruck, rlandy|rover i think we missed this part https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/34767/7/molecule/compose_promoter/converge.yml#6 in our playbook https://review.rdoproject.org/r/c/config/+/35073/2/playbooks/tripleo-ci-promote-latest-compose/run.yaml | 14:31 |
| rlandy|rover | I see patch before | 14:31 |
| bhagyashris | yeah | 14:31 |
| dviroel|ruck | you need the ssh-agent part to work with secrets | 14:32 |
| bhagyashris | ok than that is with us https://review.rdoproject.org/r/c/config/+/35073/2/playbooks/tripleo-ci-promote-latest-compose/run.yaml#5 | 14:33 |
| bhagyashris | dviroel|ruck, ^ | 14:33 |
| dviroel|ruck | the playbook is corrent, you just need to provide 'ssh_private_key_content' | 14:33 |
| dviroel|ruck | correct* | 14:34 |
| dviroel|ruck | are you testing with secret already? | 14:34 |
| rlandy|rover | dviroel|ruck: the secret already existed | 14:34 |
| bhagyashris | dviroel|ruck, yes the secret already existed | 14:35 |
| rlandy|rover | it's the same one we use for image upload | 14:35 |
| rlandy|rover | so we know the secret works | 14:35 |
| zbr | chandankumar: i did comment but you will not like it. | 14:35 |
| rlandy|rover | dviroel|ruck: pls see https://review.rdoproject.org/r/c/config/+/35073/2/zuul.d/tripleo-rdo-base.yaml | 14:36 |
| chandankumar | zbr: no problem, I will fix it | 14:36 |
| rlandy|rover | secrets: | 14:36 |
| rlandy|rover | - uploader_ssh_key | 14:36 |
| dviroel|ruck | yes, i see now | 14:36 |
| dviroel|ruck | ssh_private_key_content: "{{ uploader_ssh_key.ssh_private_key }}" | 14:36 |
| zbr | let me know if you want to talk about the subject. the patch was working fine for most jobs, i think only ovb one complained and I am sure beause it does somethign shady. | 14:36 |
| bhagyashris | dviroel|ruck, in the log file i see the task is getting skipped | 14:37 |
| zbr | if we use default ansible location we should be fine, (default is user dependent) | 14:37 |
| bhagyashris | 2021-08-24 13:12:52.925980 | TASK [Start a new ssh-agent with the provided private key] | 14:37 |
| bhagyashris | 2021-08-24 13:12:54.118011 | primary | skipping: Conditional result was False | 14:37 |
| *** pojadhav is now known as pojadhav|afk | 14:37 | |
| dviroel|ruck | humm | 14:38 |
| dviroel|ruck | you may want to try provide the secret at https://review.rdoproject.org/r/c/config/+/35073/2/playbooks/tripleo-ci-promote-latest-compose/run.yaml , not sure where zuul actually provide the content of this secret | 14:39 |
| dviroel|ruck | https://www.irccloud.com/pastebin/qXEZsn5n/ | 14:42 |
| dviroel|ruck | bhagyashris: ^ | 14:43 |
| bhagyashris | dviroel|ruck, ack | 14:43 |
| dviroel|ruck | rlandy|rover: another failure here https://ci.centos.org/view/rdo/view/promotion-pipeline/job/rdo_trunk-promote-master-current-tripleo/ | 14:50 |
| rlandy|rover | sec - will check into it after c9 meeting | 14:52 |
| bhagyashris | dviroel|ruck, rlandy|rover chandankumar https://review.rdoproject.org/r/c/config/+/35083 plz have a look when you have a free time | 14:52 |
| dviroel|ruck | bhagyashris: i think you need to remove 'when: ssh_private_key_content is defined' | 14:53 |
| rlandy|rover | dviroel|ruck: ugh tox linters | 14:53 |
| bhagyashris | once this got merge i will test it may be tomorrow | 14:53 |
| bhagyashris | dviroel|ruck, ok | 14:53 |
| rlandy|rover | chandankumar: c9 meeting | 14:53 |
| dviroel|ruck | rlandy|rover: yeah, i was about to open these logs now | 14:54 |
| rlandy|rover | dviroel|ruck: one of those days :) | 14:54 |
| bhagyashris | done | 14:55 |
| chandankumar | rlandy|rover: sorry missed it | 14:55 |
| chandankumar | just joined | 14:55 |
| rlandy|rover | chandankumar: we're just discussing the ansible issue | 14:56 |
| rlandy|rover | thought you would be interested | 14:56 |
| *** ykarel|afk is now known as ykarel | 14:56 | |
| dviroel|ruck | bhagyashris: i'm not a expert, maybe vars need to aligned with include_role, https://docs.ansible.com/ansible/latest/collections/ansible/builtin/include_role_module.html#examples | 15:02 |
| dviroel|ruck | not sure if works in this way too | 15:02 |
| ykarel | rlandy|rover, do you know if marios is on pto's? | 15:06 |
| rlandy|rover | ykarel: yes - for three weeks | 15:06 |
| rlandy|rover | ykarel: anything we can help with? | 15:06 |
| rlandy|rover | ptl stuff> | 15:06 |
| ykarel | rlandy|rover, needed vote on https://review.opendev.org/c/openstack/releases/+/804412 | 15:06 |
| rlandy|rover | weshay|ruck: ^^ | 15:07 |
| rlandy|rover | former ptl | 15:07 |
| ykarel | only liason and ptl can vote, i think both are marios :) | 15:07 |
| * dviroel|ruck brb - lunch | 15:07 | |
| ykarel | would be good to have both different | 15:07 |
| rlandy|rover | ykarel: there must be a plan if ptl is out | 15:08 |
| rlandy|rover | how about upcoming ptl? | 15:08 |
| ykarel | don't think that will work until elected | 15:08 |
| rlandy|rover | ykarel: can sean override that? | 15:10 |
| weshay|ruck | ykarel, so I think because ( no blame ) you updated the patch, now marios has to approve | 15:10 |
| weshay|ruck | or get on #openstack-release | 15:10 |
| ykarel | rlandy|rover, https://github.com/openstack/releases/blob/master/data/release_liaisons.yaml#L157-L160 and https://github.com/openstack/governance/blob/b8d28176698e376358fc5b9bd6eb23bf70e7fa7c/reference/projects.yaml#L3029-L3033 | 15:11 |
| rlandy|rover | ykarel: joined #openstack-release | 15:11 |
| rlandy|rover | there must be a way out of this | 15:11 |
| ykarel | those jobs check that | 15:11 |
| ykarel | weshay|ruck, /me was not aware if he was out and needed to include latest commits | 15:12 |
| ykarel | rlandy|rover, yes let's see | 15:12 |
| rlandy|rover | yeah - don;t want to edit marios out if we don't need tp | 15:12 |
| chandankumar | zbr: I will book something in your calender tomorrow | 15:22 |
| *** amoralej is now known as amoralej|off | 15:23 | |
| rlandy|rover | https://zuul.opendev.org/t/openstack/builds?job_name=tripleo-ci-centos-8-content-provider-victoria | 15:25 |
| rlandy|rover | may have a problem here | 15:25 |
| rlandy|rover | maybe not - only one | 15:27 |
| weshay|ruck | rlandy|rover, I commented on that review | 15:30 |
| weshay|ruck | rlandy|rover, https://review.opendev.org/c/openstack/tripleo-quickstart/+/804399/ | 15:30 |
| rlandy|rover | podman pull issue comment? | 15:31 |
| rlandy|rover | thanks for rechecking | 15:32 |
| weshay|ruck | 2021-08-16 14:21:34.157307 | primary | TASK [list_allowed : List allowed tests] *************************************** | 15:32 |
| *** jpena is now known as jpena|off | 15:33 | |
| weshay|ruck | arxcruz, this may not be logging correctly https://logserver.rdoproject.org/99/804399/3/openstack-check/tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001/0678886/logs/undercloud/home/zuul/tempest/etc/allow-list.yaml.txt.gz | 15:34 |
| weshay|ruck | https://logserver.rdoproject.org/99/804399/3/openstack-check/tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001/0678886/logs/undercloud/home/zuul/tempest/etc/tempest_includelist.txt.gz | 15:34 |
| weshay|ruck | looks fine | 15:34 |
| rlandy|rover | bhagyashris: thanks - looking | 15:41 |
| rlandy|rover | bhagyashris: will merge that so you can try it out | 15:43 |
| arxcruz | weshay|ruck: i think this allow-list is not related to tempest-skiplist | 15:43 |
| arxcruz | i need to check the os_tempest code | 15:43 |
| weshay|ruck | k | 15:43 |
| arxcruz | but the one that you need to look is https://logserver.rdoproject.org/99/804399/3/openstack-check/tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001/0678886/logs/undercloud/home/zuul/tempest/etc/tempest_includelist.txt.gz | 15:43 |
| rlandy|rover | https://logserver.rdoproject.org/99/804399/3/openstack-check/tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001/0678886/logs/undercloud/var/log/tempest/stestr_results.html.gz matches https://logserver.rdoproject.org/openstack-regular/opendev.org/openstack/tripleo-ci/master/tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001/9ab719c/logs/undercloud/var/log/tempest/stestr_results.html.gz | 15:46 |
| rlandy|rover | from before | 15:47 |
| rlandy|rover | akahat: hey - scenario010 patch ready for review? | 15:50 |
| arxcruz | rlandy|rover: yes, but you can see that the list_allowed is running | 15:50 |
| arxcruz | 2021-08-16 14:21:34.157307 | primary | TASK [list_allowed : List allowed tests] *************************************** | 15:50 |
| arxcruz | 2021-08-16 14:21:34.157333 | primary | Monday 16 August 2021 14:21:34 +0000 (0:00:00.099) 1:26:51.229 ********* | 15:50 |
| arxcruz | 2021-08-16 14:21:34.818221 | primary | changed: [undercloud] | 15:50 |
| arxcruz | https://logserver.rdoproject.org/99/804399/3/openstack-check/tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001/0678886/job-output.txt | 15:50 |
| rlandy|rover | arxcruz: so looks right | 15:51 |
| arxcruz | rlandy|rover: yes, it does :) | 15:56 |
| rlandy|rover | k - all good then | 15:56 |
| * dviroel|ruck back | 16:01 | |
| weshay|ruck | we have a handle on scen 001 master / wallaby? need anything from me? | 16:17 |
| dviroel|ruck | bhagyashris: rlandy|rover success https://images.rdoproject.org/centos_compose/centos8/ | 16:19 |
| rlandy|rover | bhagyashris: dviroel|ruck: yay!! | 16:19 |
| rlandy|rover | bhagyashris++ | 16:19 |
| rlandy|rover | so the thing we need here is ... | 16:19 |
| rlandy|rover | for that to go into a release-specific folder | 16:20 |
| rlandy|rover | so the job should be *-master | 16:20 |
| rlandy|rover | and that should upload into the master folder | 16:20 |
| rlandy|rover | bhagyashris: ^^ if you have trouble accessing th eimage server to add those folders, ping me | 16:20 |
| rlandy|rover | thanks for getting this working!!! | 16:21 |
| rlandy|rover | bhagyashris: sorry - one more :) | 16:21 |
| rlandy|rover | I think that's compose-ci-tesing? | 16:22 |
| rlandy|rover | dviroel|ruck,: weshay|ruck: ysandeep|away: ^^ right? | 16:22 |
| rlandy|rover | centos-ci-testing? | 16:22 |
| rlandy|rover | rather than tripleo-ci-testing | 16:22 |
| * rlandy|rover checks hackmd | 16:22 | |
| weshay|ruck | perfect | 16:23 |
| rlandy|rover | centos-ci-testing | 16:23 |
| weshay|ruck | would be more clear in conversations for sure.. to use centos-ci-testing | 16:23 |
| rlandy|rover | per https://hackmd.io/iYAL3ueUTzWCHE5qFA6jVQ | 16:23 |
| weshay|ruck | so.. you're not saying.. wait.. which tripleo-ci-testing | 16:24 |
| dviroel|ruck | rlandy|rover: yeah, didn't pay attention to that, i can fix that | 16:24 |
| rlandy|rover | https://images.rdoproject.org/centos_compose/centos8/ | 16:25 |
| rlandy|rover | weshay|ruck: ^^ | 16:25 |
| rlandy|rover | that should be centos_compose/centos8/master/compose-ci-testing | 16:25 |
| rlandy|rover | dviroel|ruck: thanks | 16:26 |
| dviroel|ruck | rlandy|rover: centos-ci-testing or compose-ci-testing? | 16:27 |
| dviroel|ruck | you mention both :) | 16:27 |
| rlandy|rover | dviroel|ruck: centos-ci-testing per https://hackmd.io/iYAL3ueUTzWCHE5qFA6jVQ | 16:27 |
| rlandy|rover | ^^ final choice :) | 16:28 |
| dviroel|ruck | ok :) | 16:28 |
| *** pojadhav|afk is now known as pojadhav | 16:34 | |
| rlandy|rover | lunch - back in a few | 16:42 |
| dviroel|ruck | weshay|ruck: we have a patch for https://bugs.launchpad.net/tripleo/+bug/1940568 here https://review.rdoproject.org/r/c/openstack/neutron-tempest-plugin-distgit/+/35080 , isn't something that we can testproject? | 16:48 |
| weshay|ruck | dviroel|ruck, not on a rpm spec change.. | 16:51 |
| dviroel|ruck | k | 16:51 |
| weshay|ruck | dviroel|ruck, you can see the updated rpm here.. https://logserver.rdoproject.org/80/35080/1/check/DLRN-build-tripleo-centos8/4dd4691/buildset/ | 16:52 |
| weshay|ruck | if you really really wanted to.. you could test-project, hold the node... install the updated rpm.. and rerun the tempest test in question | 16:52 |
| weshay|ruck | but.. imho all that work is not worth it | 16:52 |
| dviroel|ruck | ok, lets wait that merge first and test after | 16:53 |
| akahat | rlandy|rover, yes. It's ready. | 17:09 |
| rlandy|rover | akahat; thanks - will review | 17:09 |
| rlandy|rover | dviroel|ruck: weshay|ruck: looks like component network is stuck here https://review.rdoproject.org/zuul/status | 17:11 |
| rlandy|rover | going to try dequeue and enqueu | 17:11 |
| dviroel|ruck | rlandy|rover: ok | 17:11 |
| dviroel|ruck | rlandy|rover: https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/35094 passing in molecule job | 17:12 |
| weshay|ruck | rlandy|rover++ | 17:12 |
| dviroel|ruck | ah, periodic-tripleo-ci-centos-8-ovb-1ctlr_2comp-featureset020-ussuri failed :( | 17:14 |
| dviroel|ruck | will test it again in a few | 17:15 |
| dviroel|ruck | timeout in a tempest test | 17:16 |
| rlandy|rover | that didn;t work - will check with infra | 17:21 |
| rlandy|rover | Unknown error code 500: "500 Server Error: Internal Server Error - still | 17:30 |
| rlandy|rover | oh | 17:31 |
| rlandy|rover | nework pipeline rekicked | 17:52 |
| rlandy|rover | network | 17:52 |
| rlandy|rover | dviroel|ruck: +2 on https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/35094 | 17:53 |
| rlandy|rover | do we need an additional change to add the release directory | 17:54 |
| dviroel|ruck | in config, yes | 17:54 |
| rlandy|rover | oh wow master | 17:54 |
| rlandy|rover | dviroel|ruck: k - we can merge that patch | 17:54 |
| rlandy|rover | and then do the change in confi | 17:54 |
| dviroel|ruck | yes | 17:55 |
| dviroel|ruck | chasing ussuri here https://review.rdoproject.org/r/c/testproject/+/34983 | 17:55 |
| rlandy|rover | thanks for ^^ | 17:56 |
| rlandy|rover | 2021-08-24 17:23:33.430095 | primary | | 17:58 |
| rlandy|rover | 2021-08-24 17:23:33.430099 | primary | error: can't find Rust compiler | 17:58 |
| rlandy|rover | 2021-08-24 17:23:33.430104 | primary | | 17:58 |
| dviroel|ruck | rlandy|rover: re compose promoter, destination dir will be 'https://images.rdoproject.org/centos_compose/centos8/master'? | 17:58 |
| rlandy|rover | ack | 17:58 |
| dviroel|ruck | ok, someone need to create that dir | 17:58 |
| rlandy|rover | dviroel|ruck: I'll do it | 17:58 |
| dviroel|ruck | :) | 17:59 |
| rlandy|rover | sec - logging onto to images server | 17:59 |
| rlandy|rover | then will see what's causing mater to bleed like that | 17:59 |
| rlandy|rover | 2021-08-24 17:23:33.430099 | primary | error: can't find Rust compiler | 17:59 |
| akahat | Ready to review: https://review.opendev.org/q/topic:%22utilize-tripleo-operator%22+(status:open%20OR%20status:merged) | 18:00 |
| akahat | Testproject: https://review.rdoproject.org/r/c/testproject/+/31612/10 | 18:00 |
| rlandy|rover | dviroel|ruck: https://images.rdoproject.org/centos_compose/centos8/master/ | 18:02 |
| rlandy|rover | going to delete the entries on that level | 18:02 |
| dviroel|ruck | ok | 18:03 |
| rlandy|rover | dviroel|ruck: k - you should be all set | 18:03 |
| dviroel|ruck | rlandy|rover: ok, will rename the periodic to: 'periodic-tripleo-centos-8-promote-latest-compose-master' | 18:04 |
| dviroel|ruck | i.e, appending 'master' to the end | 18:04 |
| dviroel|ruck | just sent https://review.rdoproject.org/r/c/config/+/35095 | 18:05 |
| rlandy|rover | dviroel|ruck: k- so we have base | 18:05 |
| rlandy|rover | and then *-master etc.? | 18:05 |
| dviroel|ruck | yes | 18:05 |
| dviroel|ruck | ans each one will have a different dest dir https://review.rdoproject.org/r/c/config/+/35095/1/zuul.d/tripleoci.yaml | 18:06 |
| dviroel|ruck | i'm open to rename things too | 18:07 |
| rlandy|rover | dviroel|ruck: +2'ed - left one comment | 18:09 |
| rlandy|rover | we can merge that and change afterwards | 18:09 |
| rlandy|rover | just get the pattern right now | 18:09 |
| rlandy|rover | will w+ when zuul votes | 18:09 |
| rlandy|rover | 2021-08-24 17:23:34.030115 | primary | Failed to build cryptography | 18:10 |
| rlandy|rover | 2021-08-24 17:23:34.030341 | primary | ERROR: Could not build wheels for cryptography which use PEP 517 and cannot be installed directly | 18:10 |
| rlandy|rover | 2021-08-24 17:23:35.714459 | primary | ERROR | 18:10 |
| rlandy|rover | weshay|ruck: dviroel|ruck: ^^ master failure | 18:10 |
| rlandy|rover | 2021-08-24 17:26:09.402138 | primary | running build_rust | 18:11 |
| rlandy|rover | 2021-08-24 17:26:09.402245 | primary | error: can't find Rust compiler | 18:11 |
| rlandy|rover | pip upgrade | 18:11 |
| rlandy|rover | transient thing? | 18:14 |
| rlandy|rover | checking | 18:14 |
| dviroel|ruck | Successfully installed pip-21.2.4 | 18:15 |
| dviroel|ruck | https://pypi.org/project/pip/ | 18:15 |
| rlandy|rover | testproject rerunning standalone master | 18:16 |
| rlandy|rover | let's see | 18:16 |
| dviroel|ruck | ok | 18:16 |
| * rlandy|rover doesn't feel like another disaster now | 18:16 | |
| weshay|ruck | :P | 18:17 |
| dviroel|ruck | rlandy|rover: https://review.rdoproject.org/r/c/config/+/35095 zuul voted +1 | 18:17 |
| rlandy|rover | dviroel|ruck: w+'ed | 18:18 |
| dviroel|ruck | tks | 18:18 |
| rlandy|rover | dviroel|ruck: I will update the program call doc by my EoD ... do you want to report or should I? your choice | 18:18 |
| rlandy|rover | train, wallaby, 16.2 17 | 18:19 |
| dviroel|ruck | rlandy|rover: i can try this time :) | 18:19 |
| rlandy|rover | all the info will be there | 18:19 |
| rlandy|rover | awesome | 18:19 |
| rlandy|rover | all yours | 18:19 |
| dviroel|ruck | will take a look tomorrow morning | 18:19 |
| *** pojadhav is now known as pojadhav|away | 18:20 | |
| dviroel|ruck | will recheck that openstack-tox-linters change that failed | 18:21 |
| rlandy|rover | thanks! | 18:22 |
| rlandy|rover | https://logserver.rdoproject.org/openstack-periodic-integration-stable1/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-containers-multinode-wallaby/cffd36d/ | 18:23 |
| rlandy|rover | juts hit that error | 18:23 |
| dviroel|ruck | :( | 18:28 |
| dviroel|ruck | rlandy|rover: https://bugs.launchpad.net/tripleo/+bug/1921470 | 18:29 |
| dviroel|ruck | rlandy|rover: https://images.rdoproject.org/centos_compose/centos8/master/ :) | 18:32 |
| rlandy|rover | dviroel|ruck: master testproject hit deploy | 18:53 |
| rlandy|rover | maybe it was just a hitch https://review.rdoproject.org/zuul/stream/e226a9bf87954690b84f8f7cb5c4774d?logfile=console.log | 18:53 |
| rlandy|rover | and containers wallaby as wel | 18:54 |
| rlandy|rover | gate is standing | 18:54 |
| rlandy|rover | tripleo-ci-centos-8-scenario010-standalone-wallaby woohoo | 19:15 |
| rlandy|rover | weird - it triggered both | 19:17 |
| * rlandy|rover checks branching | 19:17 | |
| rlandy|rover | akahat: hey https://review.opendev.org/c/openstack/tripleo-ci/+/803867 - commented - needs some branch matching updates - thanks | 19:21 |
| dviroel|ruck | rlandy|rover: will need to go out for a while, run some errands | 19:32 |
| rlandy|rover | dviroel|ruck: sure - I'm around | 19:33 |
| *** dviroel|ruck is now known as dviroel|ruck|afk | 19:33 | |
| rlandy|rover | dviroel|ruck: anything you want me to watch? | 19:33 |
| weshay|ruck | rlandy|rover, ok.. want some hands on w/ the dashboard? | 20:54 |
| rlandy|rover | weshay|ruck: giv eme a few | 20:54 |
| rlandy|rover | weshay|ruck: k - ready to chat when you are | 21:15 |
| weshay|ruck | rlandy|rover, meh.. it turned out to be a one liner | 21:22 |
| weshay|ruck | rlandy|rover, let's chat tomorrow | 21:22 |
| weshay|ruck | about whatever you needed to do | 21:22 |
| rlandy|rover | weshay|ruck: ack ok | 21:24 |
| rlandy|rover | weshay|ruck: program call meeting notes updated - with wallaby :) | 21:40 |
| weshay|ruck | rock | 21:40 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!