Friday, 2020-06-26

« Thursday, 2020-06-25 Index Saturday, 2020-06-27 »
*** rlandy|ruck|bbl is now known as rlandy|ruck02:03
*** rfolco has quit IRC02:09
*** rlandy|ruck has quit IRC02:21
*** yolanda has quit IRC03:02
*** yolanda has joined #oooq03:03
*** ykarel|away is now known as ykarel04:27
*** ysandeep|away is now known as ysandeep04:40
*** jtomasek has joined #oooq04:56
*** jtomasek has quit IRC04:56
*** jtomasek has joined #oooq04:57
*** ratailor has joined #oooq05:12
*** dsneddon has joined #oooq05:28
*** udesale has joined #oooq05:40
ysandeepykarel, is it mirr06:10
ysandeepmirror issue - https://74b83773945845894408-fddd31f569f44d98bd401bfd88253d97.ssl.cf2.rackcdn.com/737923/4/check/tripleo-ci-centos-7-containers-multinode-train/68062e2/logs/undercloud/home/zuul/undercloud_install.log06:10
ykarelysandeep, i think there is a bug for it06:11
ykarelhmm related to registry mirror06:12
ysandeepykarel, thanks! checking RR notes to find the bug06:12
ykareli seen discussion on it last night, not sure if bug was created or not06:12
*** danpawlik is now known as dpawlik|EoD06:15
ysandeepykarel, nothing in RR notes about bug06:17
ysandeepsoniya29|rover, is this issue known? https://74b83773945845894408-fddd31f569f44d98bd401bfd88253d97.ssl.cf2.rackcdn.com/737923/4/check/tripleo-ci-centos-7-containers-multinode-train/68062e2/logs/undercloud/home/zuul/undercloud_install.log06:17
ysandeepI can recheck to confirm if it is consistent but just want to confirm if you have seen this with other jobs as well.06:19
ykarel<rlandy|ruck> Client Error: Too Many Requests for url: http://mirror.regionone.rdo-cloud-tripleo.rdoproject.org:8082/v2/ceph/daemon/blobs/sha256:ea7eb661de71977e5d57c2d6935d5bca6496780665cc7537595b19f8d431b676"], "stdout": "", "stdout_lines": []}06:22
ykarelysandeep, in #oooq channel only06:22
ysandeepykarel, yeah ronelle must have faced this issue with other jobs06:24
*** saneax has joined #oooq06:27
ysandeepykarel, Could you please add https://review.opendev.org/#/c/737975/ and https://review.opendev.org/#/c/737923/ to your review list, not urgent whenever you get free time. CI jobs failure looks unrelated to changes - will push recheck after some reviews.06:42
*** ysandeep is now known as ysandeep|afk07:04
ykarelysandeep|afk, ack will check07:04
ysandeep|afkykarel, thanks!07:04
ykarelrelease var shouldn't be needed though now07:06
ykarelas all centos8 are component based07:06
*** ratailor has quit IRC07:19
*** ratailor has joined #oooq07:19
*** ysandeep|afk is now known as ysandeep07:23
*** bhagyashris|afk is now known as bhagyashris07:27
*** amoralej|off is now known as amoralej07:31
*** jpena|off is now known as jpena07:31
*** jtomasek has quit IRC07:32
*** dtantsur|afk is now known as dtantsur07:33
*** tosky has joined #oooq07:35
*** jtomasek has joined #oooq07:35
*** pojadhav is now known as pojadhav|lunch07:41
ysandeepykarel, thanks! I agree with you, But I am not sure what was the motive behind the recent movement from OS to release in https://review.opendev.org/#/c/720776 .. probably i will check with weshay_pto once he is back07:55
ykarelysandeep, ack that time story was different, but good to confirm07:55
*** jtomasek has quit IRC08:10
*** jtomasek has joined #oooq08:14
ysandeepchandankumar, Test patch https://review.opendev.org/#/c/736174/ failed while pull containers 401 unauthorized - https://d2952c05763cd0040d7f-6b2fbb83e342ba16a98bd56da886e807.ssl.cf2.rackcdn.com/736174/5/check/tripleo-ci-centos-8-containers-multinode-train/f2710f8/logs/undercloud/home/zuul/undercloud_install.log08:21
soniya29|roverysandeep, I haven't came across this isse08:21
soniya29|rovers/isse/issue08:21
ysandeepsoniya29|rover, Thanks! we will a need bug if its hitting other jobs too, I will recheck my job and let you know if i hit it again.08:22
*** derekh has joined #oooq08:43
*** ykarel is now known as ykarel|lunch08:48
soniya29|roverysandeep, ack08:55
*** ysandeep is now known as ysandeep|lunch09:06
*** udesale has quit IRC09:07
*** pojadhav|lunch is now known as pojadhav09:10
*** jfrancoa has joined #oooq09:13
*** udesale has joined #oooq09:15
*** ysandeep|lunch is now known as ysandeep09:22
*** jfrancoa has quit IRC09:29
*** jfrancoa has joined #oooq09:31
*** ykarel|lunch is now known as ykarel09:55
*** ccamacho has quit IRC10:42
ysandeepchandankumar, Hey o/ whenever you get some time - Looking for some pointers on c8 branchful job failure, deployment failed while pull containers - 401 unauthorized - https://d2952c05763cd0040d7f-6b2fbb83e342ba16a98bd56da886e807.ssl.cf2.rackcdn.com/736174/5/check/tripleo-ci-centos-8-containers-multinode-train/f2710f8/logs/undercloud/home/zuul/undercloud_install.log10:46
zbrykarel: chandankumar: please have a read on https://review.opendev.org/#/c/723603/ and put a +1 or -1 on it based on what you find better.11:15
chandankumarysandeep, not sure why it is giving unauthorized error11:20
ykarelzbr, ack11:20
ykarellooks like people have valid concerns there, will post recheck, want to see how it outputs11:21
zbrthe output side is already out, need to build it again.11:22
ysandeepchandankumar, ack thanks!11:26
ykarelysandeep, is there already a train c8 promotion?11:26
ykarelif not the jobs will fail11:27
ykareli don't see any image in https://hub.docker.com/u/tripleotraincentos811:27
ysandeepchandankumar, promotion worked yesterday?11:27
ysandeepchandankumar, grabbing your testpatch11:27
chandankumarykarel, https://review.rdoproject.org/r/28204 once this finishes, we might get an auto promotion11:28
ykarelchandankumar, ack, ysandeep till then it should fail than11:29
ykarelto get check job passing images should be available in https://hub.docker.com/u/tripleotraincentos811:30
ysandeepykarel, ack, query this job don't push container to https://hub.docker.com/u/tripleotraincentos8? - https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-centos-8-train-containers-build-push11:31
ykarelysandeep, ^^ push to rdo registry, to dockerup it pushed by promoter server11:31
ykarels/dockerup/dockerhub11:32
ysandeepykarel, Thank you so much, I will wait then11:32
*** jpena is now known as jpena|lunch11:33
*** kopecmartin is now known as kopecmartin|pto11:37
chandankumararxcruz, akahat soniya29|rover I am cancelling today's tempest meeting as weshay_pto and kopecmartin|pto is on pto11:42
soniya29|roverchandankumar, ack11:42
akahatchandankumar, okay11:52
arxcruzchandankumar: ack12:11
*** rlandy has joined #oooq12:13
*** rlandy is now known as rlandy|ruck12:13
rlandy|ruckpanda: hello12:15
rlandy|rucksoniya29|rover: hi - how are things?12:15
pandarlandy|ruck: hi12:16
rlandy|ruckpanda: hey - been trying to reach you re: rdocloud promoter12:16
rlandy|ruckpanda: would you like to meet?12:16
soniya29|roverrlandy|ruck, hello, things are good12:17
*** rfolco has joined #oooq12:18
rlandy|rucksoniya29|rover: ugh - we still have to get https://review.opendev.org/#/c/738025/ through12:18
soniya29|roverrlandy|ruck, openstack-tox-pep8 is passing now for stable/train12:18
soniya29|roverrlandy|ruck, https://trello.com/c/Ub0c2vyE/1567-cixlp1884989tripleociproa-openstack-tox-pep8-is-failing-when-run-on-tripleo-common-changes-stable-train12:18
rlandy|rucksoniya29|rover: yep - look stable now - you can close out that bug12:19
soniya29|roverrlandy|ruck, okay12:19
pandarlandy|ruck: I took a look this morning. The promoter is up and running again. master ussuri and centos8-train are running. I see that this morning the container push failed, because apparently containers were not found in rdo registry12:19
rlandy|ruckpanda: I tried to disable update again12:20
pandarlandy|ruck: I'm also increasing the number of latest hashes that we consider per release, as with the component, every single promotion takes at least 12 hashes12:20
rlandy|ruckpanda: soniya29|rover: https://meet.google.com/qsr-tsjz-iej - let's discuss in case this happens again12:22
soniya29|roverrlandy|ruck, sure12:22
rlandy|ruckI need to know confirm  exact releases are expected on which promoters12:22
*** udesale has quit IRC12:29
*** derekh has quit IRC12:32
*** rlandy|ruck is now known as rlandy|ruck|mtg12:35
*** jpena|lunch is now known as jpena12:44
rlandy|ruck|mtgsoniya29|rover: https://etherpad.opendev.org/p/tripleo-ptg-victoria12:53
rlandy|ruck|mtglabel:openstack-discuss  tripleo ptg12:56
rlandy|ruck|mtgchandankumar: hey - can we chat about the cirros image?13:02
*** amoralej is now known as amoralej|lunch13:04
chandankumarrlandy|ruck|mtg, give me a min13:04
*** derekh has joined #oooq13:04
chandankumarrlandy|ruck|mtg, now, gmeet link13:04
rlandy|ruck|mtgchandankumar: https://meet.google.com/mbr-neoo-bak13:05
soniya29|roverrlandy|ruck|mtg, we need some more reviews on this patch:- https://review.opendev.org/#/c/737058/413:10
chandankumarrlandy|ruck|mtg, https://logserver.rdoproject.org/openstack-periodic-master/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-scenario001-standalone-master/a4bcfd5/logs/undercloud/var/log/tempest/stestr_results.html.gz13:11
rlandy|ruck|mtgchandankumar: https://bugs.launchpad.net/tripleo/+bug/188527913:14
openstackLaunchpad bug 1885279 in tripleo "Standalone tests on master are failing in tempest with cirros-0.5.1 image" [Critical,In progress] - Assigned to Ronelle Landy (rlandy)13:14
*** ykarel is now known as ykarel|afk13:14
chandankumarrlandy|ruck|mtg, https://logserver.rdoproject.org/openstack-periodic-master/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-scenario001-standalone-master/a4bcfd5/logs/undercloud/home/zuul/tempest/etc/profile.yaml.txt.gz13:15
*** dtantsur is now known as dtantsur|afk13:21
chandankumarrlandy|ruck|mtg, https://review.rdoproject.org/r/#/c/28204/13:21
*** TrevorV has joined #oooq13:25
*** pojadhav is now known as pojadhav|brb13:27
rlandy|ruck|mtgchandankumar: hey - taking this off infra for the moment13:39
chandankumarrlandy|ruck|mtg, https://opendev.org/openstack/devstack/src/commit/a13ab8db2dbd959b5a1f52ac03d3fca07a19eb04/lib/tempest#L20613:40
chandankumarrlandy|ruck|mtg, openstack flavor create --id 42 --ram 128 --disk $disk --vcpus 1 --property hw_rng:allowed=True m1.nano13:40
chandankumarrlandy|ruck|mtg, in tempestconf it is 6413:41
rlandy|ruck|mtgchandankumar: whatever change we make, we need to get it through gates13:41
rlandy|ruck|mtgchandankumar: does tempestconf go through diff gates?13:41
chandankumarrlandy|ruck|mtg, yes13:41
rlandy|ruck|mtglook like we can get the revert through in the mean time13:41
chandankumarlet me prepare a patch13:42
rlandy|ruck|mtglet's qualify the new cirros image more13:42
rlandy|ruck|mtgbefore we create another gate disaster13:42
*** udesale has joined #oooq13:47
*** amoralej|lunch is now known as amoralej13:47
chandankumarrlandy|ruck|mtg, https://review.opendev.org/#/c/738210/13:50
*** rlandy|ruck|mtg is now known as rlandy|ruck13:54
rlandy|ruckchandankumar: k - cool  - let's spin on that :)13:57
chandankumarrfolco, around?14:32
rfolcochandankumar, here14:32
chandankumarrfolco, need help https://logserver.rdoproject.org/94/28194/11/check/periodic-tripleo-build-containers-ubi-8/d7e1429/job-output.txt14:32
chandankumarrfolco, Retag and push image trunk.registry.rdoproject.org/testingmaster/openstack-neutron-metadata-agent-ovn:latest w/ _arch tag suffix]14:33
chandankumar2020-06-26 14:23:54.393062 | primary | Getting image source signatures14:33
chandankumar2020-06-26 14:23:54.393262 | primary | Error: Error copying image to the remote destination: Error trying to reuse blob sha256:f9327c1608ad0ae79b5f4a4c47eda907a980ea9640c70e6036811ea76d641249 at destination: Error checking whether a blob sha256:f9327c1608ad0ae79b5f4a4c47eda907a980ea9640c70e6036811ea76d641249 exists in trunk.registry.rdoproject.org/testingmaster/openstack-neutron-metadata-agent-ovn: unauthorized: authentication required14:33
rlandy|ruckchandankumar: we have bigger problems ^^14:35
*** pojadhav|brb is now known as pojadhav14:35
chandankumarrlandy|ruck, looking14:35
*** ratailor has quit IRC14:36
*** ykarel|afk is now known as ykarel14:37
chandankumarrlandy|ruck, Image prepare failed: Modifying image 192.168.24.1:8787/tripleomaster/centos-binary-ovn-controller:8a49b1e69db255997a89e8527f7270e9 failed here14:38
rlandy|ruckI know14:38
rlandy|ruckpojadhav: hi - can you update the ussuri branch odftripleo-heat-templates to use the ipa template?14:40
chandankumarrfolco, here is the job definition https://github.com/rdo-infra/review.rdoproject.org-config/blob/master/zuul.d/tripleoci.yaml#L493 and triggered here https://review.rdoproject.org/r/#/c/28194/11/zuul.d/build-containers.yaml please have a look, may be I am missing something14:42
pojadhavrlandy|ruck, yup14:42
rlandy|ruckthanks14:43
rfolcochandankumar, will do in a bit14:47
chandankumarrfolco, aye thanks!14:55
rlandy|ruckzbr: you around?14:59
rlandy|ruckif so, we need your help14:59
chandankumarrlandy|ruck, https://review.opendev.org/#/c/738025/ may be we can get it forced merged, minions are also tripleo-ci-centos-8-containers-undercloud-minion (3. attempt)15:00
pojadhavrlandy|ruck, chandankumar : please have a look https://review.opendev.org/#/c/738224/15:11
weshay_ptorlandy|ruck, 2020-06-26 15:01:51.410940 | primary |   "msg": "Depsolve Error occured: \n Problem: cannot install both libnghttp2-1.41.0-1.module_el8+9071+b2b61c14.x86_64 and libnghttp2-1.33.0-1.el8_0.1.x86_64\n  - libnghttp2-1.33.0-1.el8_0.1.i686 has inferior architecture\n  - cannot install the best update candidate for package libnghttp2-1.33.0-1.el8_0.1.x86_64\n  - problem with installed package libnghttp2-1.33.0-1.el8_0.1.i686\n  -15:11
weshay_ptopackage libnghttp2-1.33.0-1.el8_0.1.x86_64 is filtered out by modular filtering",15:11
weshay_pto2020-06-26 15:01:51.411072 | primary |15:11
rlandy|ruckchandankumar: how far are we from getting the tempconf patch in?15:26
rlandy|ruckweshay_pto: https://review.opendev.org/#/c/738210/15:28
rfolcochandankumar, its your namespace I suppose15:28
rfolcohttps://trunk.registry.rdoproject.org:8443/oapi/v1/namespaces/testingmaster/imagestreamtags/15:28
weshay_ptoarxcruz, can you please have a look at https://review.opendev.org/#/c/738210/15:30
arxcruzweshay_pto: sure, need to update the tests as well, would you like me to do it, or chandankumar is already on it?15:32
weshay_ptoarxcruz, it's late in your day but later in his.. just extra eyes and review please15:37
*** jtomasek has quit IRC15:37
rlandy|ruckpojadhav: pls see comments on https://review.opendev.org/#/c/738224/15:38
rlandy|ruckthanks15:38
weshay_ptoarxcruz, it's failing tox and other tests.. and we probably should testproject the patch and run it a few more times.. apparently cirros 5.1 introduces some requirements 4x didn't have15:38
chandankumararxcruz, which test15:44
*** matbu has quit IRC15:46
chandankumarrlandy|ruck, arxcruz, weshay_pto fix tested https://review.opendev.org/#/c/738210/15:48
weshay_ptochandankumar, arxcruz ok.. hit that patch hard w/ additional upstream DNM patches please15:49
chandankumarweshay_pto, rlandy|ruck arxcruz https://review.opendev.org/738229 dnm15:52
*** matbu has joined #oooq15:53
chandankumarrfolco, testingmaster is my namespace15:53
rlandy|ruckchandankumar: thanks - we have a better chance of getting that through15:54
rfolcochandankumar, try a manual push to this namespace with the same credentials used in the job15:59
chandankumarrfolco, will do that15:59
rfolcochandankumar, I don't see any other reasons to fail other than namespace16:00
weshay_ptochandankumar, thanks16:00
chandankumarrfolco, thanks!16:02
chandankumararxcruz, if all passes, please merge that one16:02
chandankumarI am going offline16:02
*** ykarel is now known as ykarel|away16:04
*** matbu has quit IRC16:11
*** matbu has joined #oooq16:12
weshay_ptochandankumar, arxcruz is the cirros 4.x image already loaded in glance here? https://6b5c7688254ddf6bc600-1e37d93b1287908e0942eb2641078389.ssl.cf5.rackcdn.com/737774/2/gate/tripleo-ci-centos-8-scenario010-standalone/5e1471f/job-output.txt16:25
weshay_pto2020-06-26 15:55:40.731231 | primary | 2020-06-26 15:55:40.486 252527 ERROR tempest Details: b'409 Conflict\n\nImage status transition from active to saving is not allowed\n\n   '16:25
weshay_ptoperhaps python-tempestconf needs to check and handle that scenaro16:26
weshay_ptoscenario16:26
weshay_ptorlandy|ruck,  /me logs this bug16:27
weshay_ptothat scenario010 just hit16:27
weshay_ptohttps://bugs.launchpad.net/tripleo/+bug/188531116:31
openstackLaunchpad bug 1885311 in tripleo "python-tempestconf: 409 Conflict Image status transition from active to saving is not allowed'" [Critical,Triaged]16:31
rlandy|ruckweshay_pto: logging a new vexxhost OVB bug16:32
rlandy|ruckhttps://bugs.launchpad.net/tripleo/+bug/1884518 - we get past that16:32
openstackLaunchpad bug 1884518 in tripleo "OVB metalsmith deployment fails: Failed to attach VIF ... to bare metal node, Node ... is locked by host undercloud" [Critical,Triaged]16:32
rlandy|ruck2020-06-26 15:46:34 | 2020-06-26 15:46:34.504293 | fa163e90-5f1c-ebc6-c73f-000000000064 |      FATAL | AllNodesValidationConfig | overcloud-controller-2 | error={"changed": true, "msg": "non-zero return code", "rc": 1, "stderr": "Shared connection to 192.168.24.26 closed.\r\n", "stderr_lines": ["Shared connection to 192.168.24.26 closed."], "stdout": "Trying to ping default gateway 10.0.0.1...Ping to 10.0.0.1 failed.16:33
rlandy|ruckRetrying...\r\nPing to 10.0.0.1 failed. Retrying...\r\nPing to 10.0.0.1 failed. Retrying...\r\nPing to 10.0.0.1 failed. Retrying...\r\nPing to 10.0.0.1 failed. Retrying...\r\nPing to 10.0.0.1 failed. Retrying...\r\nPing to 10.0.0.1 failed. Retrying...\r\nPing to 10.0.0.1 failed. Retrying...\r\nPing to 10.0.0.1 failed. Retrying...\r\nPing to 10.0.0.1 failed. Retrying...\r\nFAILURE\r\n10.0.0.1 is not pingable.\r\n",16:33
rlandy|ruck"stdout_lines": ["Trying to ping default gateway 10.0.0.1...Ping to 10.0.0.1 failed. Retrying...", "Ping to 10.0.0.1 failed. Retrying...", "Ping to 10.0.0.1 failed. Retrying...", "Ping to 10.0.0.1 failed. Retrying...", "Ping to 10.0.0.1 failed. Retrying...", "Ping to 10.0.0.1 failed. Retrying...", "Ping to 10.0.0.1 failed. Retrying...", "Ping to 10.0.0.1 failed. Retrying...", "Ping to 10.0.0.1 failed. Retrying...", "Ping to16:33
rlandy|ruck10.0.0.1 failed. Retrying...", "FAILURE", "10.0.0.1 is not pingable."]}16:33
rlandy|ruck2020-06-26 15:46:34 |16:33
weshay_ptoanyone know what "CANCELLED" status in zuul means?16:35
weshay_ptothat's new to me16:35
weshay_ptorlandy|ruck, ok.. all current gate failures accounted for.. /me digs into a little history16:35
rlandy|ruckweshay_pto: yes ..16:40
*** amoralej is now known as amoralej|off16:40
rlandy|ruckif the job above in the gate fails, the job following has its tests cancelled16:40
*** jpena is now known as jpena|off16:42
rlandy|ruckweshay_pto: for the vexxhost failures ... https://bugs.launchpad.net/tripleo/+bug/188531416:46
openstackLaunchpad bug 1885314 in tripleo "OVB master job running on vexxhost show some nodes failing introspection step" [Critical,Triaged]16:46
rlandy|ruckpatch included16:46
rlandy|rucksagi w-1'ed it am16:46
rlandy|ruckatm16:46
rlandy|ruckwe can test that16:46
*** derekh has quit IRC17:00
rlandy|ruckchandankumar: who would this bug go to? us or packaging? https://bugs.launchpad.net/tripleo/+bug/188531517:00
openstackLaunchpad bug 1885315 in tripleo "tripleo-buildimage-overcloud-full-centos-8 is failing on the update of libnghttp2 package" [Critical,Triaged]17:00
rlandy|ruckweshay_pto: ^^ fyi - one legit retry_limit error17:01
weshay_ptok.. isolating retries to a openvswitch bug17:01
weshay_ptorlandy|ruck, https://review.rdoproject.org/r/#/c/28100/17:06
weshay_ptorlandy|ruck, I have a mtg next week to talk about deps as components17:07
rlandy|ruckcharming - more to watch17:07
rlandy|ruckweshay_pto: what do you want to do about https://review.opendev.org/#/c/735659/? restore it? updating last of CIX cards: https://trello.com/c/ENhCL2zD/1551-cixlp1881642tripleociproa-ussuri-master-fs020-failing-on-testnetworkadvancedserverops17:12
rlandy|ruckhttps://logserver.rdoproject.org/openstack-periodic-master/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-1ctlr_2comp-featureset020-master/996d1f8/logs/undercloud/var/log/tempest/stestr_results.html.gz17:13
rlandy|ruck^^ two failures there17:13
rlandy|ruckwe could clear fs02017:13
weshay_ptofix the merge conflict.. jebus17:13
weshay_ptowhy was that abandoned17:14
rlandy|ruckRafael Folco17:14
rlandy|ruckAbandoned new ruck/rovers will decide what to do17:14
rlandy|ruckweshay_pto: k - I'll fix that merge conflict17:15
chandankumarrlandy|ruck, https://bugs.launchpad.net/tripleo/+bug/1885315 was looking at this libnghttp2-1.33.0-1.el8_0.1.i686 32 bit package from where it is coming17:18
openstackLaunchpad bug 1885315 in tripleo "tripleo-buildimage-overcloud-full-centos-8 is failing on the update of libnghttp2 package" [Critical,Triaged]17:18
chandankumarneed to dig that out17:18
weshay_ptorlandy|ruck, added you to a mtg on monday re: 3rd party deps and the component pipeline17:21
rlandy|ruckchandankumar: ok - just didn't know who to assign it to17:22
rlandy|ruckweshay_pto: on training - could we move it earlier?17:22
rlandy|ruckan hour earlier and I can attend for 30 mins17:23
weshay_ptothere will be follow ups17:23
rlandy|ruckok17:24
weshay_ptorlandy|ruck, I have to relocate.. bbl today17:29
weshay_ptopretty sure we're dealing w/ openvswitch issues17:29
rlandy|ruckweshay_pto: sure - thanks for jumping in17:29
weshay_pto0/17:29
weshay_ptothank you17:29
rlandy|ruckwho is we?17:29
rlandy|ruckroyal we?17:29
weshay_ptoya17:29
weshay_ptolozl17:29
weshay_ptothe retries17:29
rlandy|ruckwho is fixing that?17:30
weshay_ptono one today17:30
rlandy|ruckwe need to put in a patch?17:30
weshay_ptonot sure..17:30
weshay_ptowill pick it up monday17:30
rlandy|ruckok - will watch17:30
*** udesale has quit IRC17:39
*** dsneddon has quit IRC17:55
pojadhavrlandy|ruck, pls check https://review.opendev.org/#/c/738224/218:11
rlandy|ruck2020-06-26 17:47:21,536 19601 DEBUG urllib3.util.retry [  ] Incremented Retry for (url='/v2/tripleotrain/centos-binary-neutron-metadata-agent-ovn/blobs/sha256:8677c78a778099e0855e10ec5216db41ee347c7d787060308f96aee7aaea6e9f'): Retry(total=3, connect=None, read=None,18:14
rlandy|ruckweshay_pto: ^^ another ovn conatiner fail18:14
rlandy|ruckon a diff distro though18:14
*** matbu has quit IRC18:21
*** matbu has joined #oooq18:33
zbrrlandy|ruck: still need my help? i just seen the msg.18:38
rlandy|ruckzbr: we're looking into the retry_limit failures ...18:38
rlandy|ruckweshay_pto thought you could help there ...18:38
rlandy|ruckhttps://trello.com/c/g6p2hyYm/1569-cixlp1885286tripleociproa-increase-in-retrylimit-errors-in-zuulopenstackorg-is-preventing-jobs-from-passing-check-gate18:39
rlandy|ruckzbr: ^^18:39
rlandy|ruckmostly alex and infra are working on it now18:39
rlandy|ruckzbr: I am out on training next week so if there is no progress by then, you can have some fun on this one18:40
rlandy|ruckright now, there is no clear resolution18:40
zbrsomeone please correct me: failure in pre does not mean we failed to setup the node? like in failure to boot the node.18:42
zbri will watch on infra...18:42
rlandy|ruckzbr: it's a long story18:48
rlandy|ruckwe're watching developments on openstack-infra18:48
zbryeah, sad that i am out of popcorn18:55
*** matbu has quit IRC19:07
*** matbu has joined #oooq19:19
*** jfrancoa has quit IRC19:22
*** matbu has quit IRC19:25
arxcruzweshay_pto: checking, sorry didn't saw your message19:25
*** matbu has joined #oooq19:32
*** TrevorV has quit IRC19:50
arxcruzweshay_pto: so i need some glance guy to check if is a valid error or not20:10
arxcruzweshay_pto: https://6b5c7688254ddf6bc600-1e37d93b1287908e0942eb2641078389.ssl.cf5.rackcdn.com/737774/2/gate/tripleo-ci-centos-8-scenario010-standalone/5e1471f/logs/undercloud/var/log/containers/glance/api.log20:10
arxcruzweshay_pto: it seems to be some random error because the first upload passes, and yes, it's using cirros 4.020:10
rlandy|ruckient Error: Too Many Requests for url: http://mirror.mtl01.inap.opendev.org:8082/v2/tripleomaster/centos-binary-cinder-api/blobs/sha256:ff11b65be4e7eda3808f0e94898e97619602e94e215a5627da1dd0d03e51b20:12
rlandy|ruckmore errors20:12
rlandy|ruckugh20:12
*** matbu has quit IRC21:07
*** matbu has joined #oooq21:14
*** chem has quit IRC21:18
*** chem has joined #oooq21:19
*** dsneddon has joined #oooq21:36
*** chem has quit IRC21:48
*** chem has joined #oooq21:54
*** tosky has quit IRC23:00
*** saneax has quit IRC23:05
« Thursday, 2020-06-25 Index Saturday, 2020-06-27 »

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!