Monday, 2021-04-19

« Sunday, 2021-04-18 Index Tuesday, 2021-04-20 »
*** jmasud has joined #oooq00:24
*** jmasud has quit IRC00:51
*** jmasud has joined #oooq02:53
*** jmasud has quit IRC03:01
*** jmasud has joined #oooq03:03
*** jmasud has quit IRC03:18
*** jmasud has joined #oooq03:25
*** ykarel__ has joined #oooq03:51
*** jpodivin has joined #oooq04:52
*** jpodivin has quit IRC04:53
*** ykarel_ has joined #oooq05:02
*** ykarel__ has quit IRC05:05
*** marios has joined #oooq05:06
*** bhagyashris|away is now known as bhagyashris05:33
*** skramaja has joined #oooq05:41
*** udesale has joined #oooq05:43
*** jpodivin has joined #oooq05:46
*** udesale has quit IRC05:49
*** slaweq has quit IRC06:16
*** slaweq has joined #oooq06:19
*** dpawlik7 is now known as dpawlik06:22
*** saneax has joined #oooq06:25
*** amoralej|off is now known as amoralej06:40
*** jmasud has quit IRC06:56
*** matbu has quit IRC07:05
*** matbu has joined #oooq07:05
*** jlarriba has joined #oooq07:15
*** ksambor has joined #oooq07:19
*** anbanerj|ruck has quit IRC07:25
*** frenzy_friday has joined #oooq07:26
*** frenzy_friday is now known as anbanerj|ruck07:29
*** apetrich has joined #oooq07:45
*** jpena|off is now known as jpena07:56
*** udesale has joined #oooq07:59
*** tosky has joined #oooq08:03
*** anbanerj|ruck has quit IRC08:04
*** anbanerj|ruck has joined #oooq08:04
*** derekh has joined #oooq08:17
*** ykarel_ is now known as ykarel|lunch08:46
*** ysandeep is now known as ysandeep|afk09:01
*** ykarel|lunch is now known as ykarel09:41
*** ksambor is now known as ksambor|lunch10:16
*** ysandeep|afk is now known as ysandeep10:24
*** saneax has quit IRC10:41
*** saneax has joined #oooq10:55
*** marios is now known as marios|afk10:55
*** ksambor|lunch is now known as ksambor10:58
*** jpena is now known as jpena|lunch11:34
*** rlandy has joined #oooq11:36
*** rlandy is now known as rlandy|rover11:36
rlandy|roveranbanerj|ruck: hey - can you update the CIX cards you are working on?11:40
*** pojadhav is now known as pojadhav|dentist11:47
rlandy|roverysandeep: hey ... can you look at https://trello.com/c/emw2rEEc/1873-cixbz1939362componentosp17rhel8baremetal-provisioning-is-failing-with-error-no-partition-with-uuid-found-on-device-dev-sda?11:47
rlandy|roverafaict, w should have the fix in 1711:47
rlandy|roverbut same error is showing up11:47
chandankumarrlandy|rover: hello11:52
chandankumarrlandy|rover: https://review.opendev.org/c/openstack/tripleo-quickstart/+/786186 is good to go11:52
rlandy|roverchandankumar: hi there11:52
rlandy|roverchandankumar: thanks +211:54
chandankumar:-)11:54
chandankumarrlandy|rover: just wanted to confirm this https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/32273/3/ci-scripts/dlrnapi_promoter/dlrn_hash.py#181 about extended hash11:55
chandankumarthen I will fix rest of the stuff11:55
chandankumarplease have a look when free11:55
rlandy|roverchandankumar: sure - just updating CIX - will look after that meeting11:56
rlandy|roverhope my comments made sense11:56
rlandy|roverwe need to diff between the full hash and extended hash11:56
*** amoralej is now known as amoralej|lunch12:00
anbanerj|ruckrlandy|rover, periodic-tripleo-ci-centos-8-ovb-1ctlr_2comp-featureset020-master is failing with "No module named 'etcd3gw'" for https://trello.com/c/PQgy05Ai/1906-cixlp1923194tripleociproa-master-promotion-tempestscenariotestminimumbasictestminimumbasicscenariotestminimumbasicscenario-is-fa12:18
*** ksambor has quit IRC12:18
rlandy|roverk - pls ovcmment on cardand we will discuss12:18
anbanerj|ruckyep, updated in card12:25
*** ksambor has joined #oooq12:26
rlandy|roveranbanerj|ruck: lol - they canceled CIX due to PTG12:28
anbanerj|ruckrlandy|rover, yeah, tried to join, the invite vanished12:30
weshay|ruck:)12:30
rlandy|roveryep12:30
rlandy|roverweshay|ruck: hey12:30
rlandy|roverweshay|ruck: looks like ovb is failing in queens - should we fix that or we are not concerned?12:31
rlandy|roveralso testprojecting https://review.opendev.org/c/openstack/tripleo-ci/+/785254/ again - needs to go today - voting or not12:32
weshay|ruckrlandy|rover, let's kill those jobs12:32
weshay|ruckthe 3rd party queens12:32
rlandy|roverweshay|ruck: k - will do12:32
chandankumarweshay|ruck: https://review.opendev.org/c/openstack/tripleo-quickstart/+/786186 good to go12:32
*** jpena|lunch is now known as jpena12:37
chandankumarrlandy|rover: weshay|ruck https://softwarefactory-project.io/r/c/DLRN/+/21576 when free12:39
rlandy|roverfailing12:44
rlandy|roverwaiting for recheck12:45
*** marios|afk is now known as marios|ptg12:45
*** pojadhav|dentist is now known as pojadhav12:45
weshay|ruckrlandy|rover, we can probably chop down the integration line for queens as well in the background today12:50
rlandy|roverweshay|ruck: remove more than just OVB?12:50
weshay|ruckrlandy|rover, ya.. we can match what's voting upstream12:50
weshay|ruckfairly contained12:50
* rlandy|rover checks what else if failing12:50
rlandy|roverweshay|ruck: ack - putting up review12:51
rlandy|roverthen we can update promoter to match12:51
ysandeeprlandy|rover, ack will look12:52
rlandy|roverysandeep: thanks - looks to me like we should have the fix in place12:53
*** jfrancoa has joined #oooq12:54
*** amoralej|lunch is now known as amoralej12:59
*** jfrancoa has quit IRC12:59
*** jfrancoa has joined #oooq13:12
*** ykarel has quit IRC13:12
*** jfrancoa has quit IRC13:16
*** jfrancoa has joined #oooq13:38
*** jfrancoa has quit IRC13:40
*** jfrancoa has joined #oooq13:40
*** jfrancoa has quit IRC13:44
*** fuzzball81 has joined #oooq14:05
rlandy|roverpojadhav: weshay|ruck: plans to add wallaby promotions to http://dashboard-ci.tripleo.org/d/HkOLImOMk/upstream-and-rdo-promotions?orgId=1? if not - will try14:35
pojadhavrlandy|rover, already added wallaby promotions to http://dashboard-ci.tripleo.org/d/HkOLImOMk/upstream-and-rdo-promotions?orgId=1?14:42
*** jlarriba has quit IRC14:44
rlandy|roverpojadhav: thanks -we are still missing wallaby on the top (Tripleo-CI promotions) panel14:45
rlandy|roverunless I need to reload?14:45
*** ykarel has joined #oooq14:47
pojadhavrlandy|rover, we purposely put wallaby panel after master promotion panels..14:47
pojadhavplease have a look after master promotions panels14:48
rlandy|roverpojadhav: I am looking at the top panel14:49
rlandy|roverlists the dates of the last promotions14:49
rlandy|roverall of them14:49
rlandy|roverpojadhav: do you see what I am talking about?14:52
*** jlarriba has joined #oooq14:52
rlandy|roverlet me know if it doesn't make sense14:52
weshay|ruckysandeep, pay attention to this one.. will change our bm deployments afaict15:02
ysandeepyes o/15:04
*** jlarriba has quit IRC15:04
ysandeepweshay|ruck, fyi.. ovb featureset already using network data v2 https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-master/c7811ec/logs/undercloud/home/zuul/overcloud-deploy.sh.txt.gz15:09
weshay|ruckaye.. just keep an aye on changes15:10
*** ykarel has quit IRC15:12
*** jmasud has joined #oooq15:13
*** jmasud has quit IRC15:23
*** jlarriba has joined #oooq15:24
pojadhavrlandy|rover, ahh you mean to say in (Tripleo-CI promotions) wallaby is not present yet..15:25
*** jmasud has joined #oooq15:25
chandankumarrlandy|rover: +w please https://review.opendev.org/c/openstack/tripleo-quickstart/+/78618615:28
*** jlarriba has quit IRC15:33
*** jlarriba has joined #oooq15:34
*** amoralej is now known as amoralej|off15:41
rlandy|roverdone15:43
*** jmasud has quit IRC15:44
*** jmasud has joined #oooq15:44
*** jlarriba has quit IRC15:44
*** jlarriba has joined #oooq15:46
*** jlarriba has quit IRC15:48
*** jlarriba has joined #oooq15:49
*** jpodivin has quit IRC15:54
*** jlarriba has quit IRC15:55
*** jlarriba has joined #oooq15:55
rlandy|roversshnaidm: chandankumar: marios: arxcruz: pls review https://etherpad.opendev.org/p/ci-tripleo-repos - if you wnat to add anything else16:03
rlandy|roverwe are presenting tomorrow16:03
*** marios|ptg is now known as marios|brb16:04
rlandy|roverweshay|ruck: need to go to post office to drop off old license plates ... biab16:06
*** rlandy|rover is now known as rlandy|rvr|biab16:06
weshay|ruckrlandy|rvr|biab, k16:07
weshay|ruckpojadhav, you sure that patch is right?16:10
*** ysandeep is now known as ysandeep|away16:10
pojadhavweshay|ruck, something messed up.. let me correct it..16:11
*** marios|brb is now known as marios|afk16:15
*** udesale has quit IRC16:20
pojadhavweshay|ruck, json file copied at wrong dir.. now corrected - https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/33413/316:24
*** marios|afk is now known as marios16:34
mariosack rlandy|rvr|biab will check that tomorrow morning16:34
*** marios is now known as marios|out16:42
weshay|ruckpojadhav, rlandy|rvr|biab wallaby promotions are visible, thanks http://dashboard-ci.tripleo.org/d/HkOLImOMk/upstream-and-rdo-promotions?orgId=116:47
pojadhavweshay|ruck, have you made any diffrent changes to visible into grafana?16:47
pojadhavin the development server, I was unable to see it16:48
weshay|ruckpojadhav, just merged the change, pulled it and restarted16:50
pojadhavweshay|ruck, okay :)16:51
*** jpena is now known as jpena|off16:59
*** marios|out has quit IRC17:07
*** derekh has quit IRC17:10
*** rlandy|rvr|biab is now known as rlandy|rover17:36
rlandy|roverweshay|ruck: ok ... want to work on scneario004 ... https://review.rdoproject.org/r/c/testproject/+/3158617:40
rlandy|rovergoing  see what is going on what wallaby and ussuri17:40
rlandy|roverwe should merge https://review.opendev.org/c/openstack/tripleo-ci/+/785254/ when zuul passes it17:41
weshay|ruckk17:41
rlandy|roverussuri was deploy thing - reran17:42
rlandy|roverso just wallaby17:42
rlandy|roverchecking on that17:42
*** jfrancoa has joined #oooq17:42
*** jmasud has quit IRC17:43
rlandy|rovergouthamr: hi - we still have wallaby failing scenario004 ... master passes ... python-tempestconf and manila seem to be the same version. Did you change another component that we need to promote to wallaby?17:55
rlandy|rovertripleo-heat-templates bakcport?17:56
rlandy|roverbackport?17:56
gouthamrOh, not that I know of - where is this failure rlandy|rover ? RDO CI?17:57
rlandy|rovergouthamr: https://review.rdoproject.org/r/c/testproject/+/3158617:58
rlandy|roverdepends-on: https://review.opendev.org/c/openstack/tripleo-ci/+/785254/17:58
rlandy|roverussuri failure is in deploy - so that is something else17:58
rlandy|roverwe are seeing a consistent wallaby failure17:58
rlandy|roverI'd like to merge https://review.opendev.org/c/openstack/tripleo-ci/+/785254/ as soon as zuul +1's it17:59
gouthamrAck, looking through the logs18:00
gouthamrhttps://review.rdoproject.org/zuul/build/23f5d3b1eae043459534ee08029aa3d2/log/logs/undercloud/etc/tempest/tempest.conf.txt.gz#8918:00
gouthamrLooks correct18:01
rlandy|roverbut we have the same setup error still18:02
rlandy|roverso master must have something that wallaby doesn't18:04
gouthamryes18:06
gouthamrhttps://review.rdoproject.org/zuul/build/23f5d3b1eae043459534ee08029aa3d2/log/logs/undercloud/var/log/containers/manila/manila-share.log.txt.gz#35018:06
*** saneax has quit IRC18:06
gouthamrperhaps the caps for the manila user? We're getting an "access-denied" when trying to create a share18:07
gouthamrtbarron: ^ maybe the error is familiar to you?18:08
* tbarron looks18:08
rlandy|rovertbarron: thanks - let me know if I missed something18:14
rlandy|roverwould like to close this out today - on all branches18:14
gouthamrceph user caps look alright;18:15
gouthamrnothing warranting a change yet, although we're seeking a caps reduction in manila itself soon: https://review.opendev.org/c/openstack/manila/+/78562318:15
*** jmasud has joined #oooq18:16
gouthamrthe job deployed ceph version 16.2.0 (0c2054e95bcd9b30fdd908a79ac1d8bbc3394442) pacific (stable)18:17
rlandy|roverweshay|ruck: ysandeep|away: ok to merge? https://code.engineering.redhat.com/gerrit/#/c/235463/18:17
weshay|ruckdon't know18:19
tbarrongouthamr: yes18:26
tbarronrlandy|rover: this isn't at all obvious, do we have the passing master job to compare with?18:27
gouthamrtbarron: didn't find anything amiss, i'll look for the package version of the ceph-argparse library18:27
gouthamrtbarron: yes, https://review.rdoproject.org/zuul/build/5da480d18bfd43fd8269264b7e331171/18:28
gouthamrthe ceph versions match between the two jobs18:28
tbarrongouthamr: ty18:28
*** jmasud has quit IRC18:30
gouthamrrlandy|rover: iiuc "master" is actually older than wallaby because the promotion is currently blocked?18:33
gouthamrthe logs on https://review.rdoproject.org/zuul/build/5da480d18bfd43fd8269264b7e331171/log/logs/undercloud/var/log/containers/manila/manila-share.log.txt.gz indicate so at least18:34
rlandy|roverok - looking18:36
rlandy|rovermaster ran in the same testproject18:37
rlandy|roverok - so when we promote master manila - it will also fail18:40
gouthamryes, likely18:40
rlandy|rovergouthamr: so where do we go from here?18:41
rlandy|roverthis was supposed to fix wallaby18:41
tbarronyeah.  We probably don't have vkmc's fix to send those calls to the manager ... but neither did we down-privilege the caps18:41
tbarronrlandy|rover: gouthamr any chance we can get a deployment like this job?18:42
rlandy|rovertbarron: we can hold a node for you to work on18:42
tbarronrlandy|rover: gouthamr our upstream code may not match what is being packaged for centos18:42
gouthamrupstream ceph code?18:43
tbarronrlandy|rover: I think that may be the best way forward unless we happen to see this ...18:43
tbarrongouthamr: yes18:43
rlandy|roverk - let me rekick the testproject and get the node held with your keys18:43
* gouthamr had just added what version of ceph is necessary for Wallaby into vkmc's etherpad for the PTG18:43
gouthamrrlandy|rover: would be very helpful, thank you!18:44
tbarrongouthamr: this ceph fs subvolume stuff isn't getting us permission denied upstream.  I've run both with patched octopus and with pacific, or master rather18:44
tbarrongouthamr: and never hit this.18:45
rlandy|rovertbarron: asking to add these keys: https://github.com/tbarron.keys corrcet?18:45
rlandy|rovercorrcet18:45
rlandy|rovercorrect18:46
gouthamrrlandy|rover: https://github.com/gouthampacha.keys and https://github.com/tombarron.keys18:47
*** vkmc has joined #oooq18:47
tbarronrlandy|rover: yes, ty ^18:47
tbarronrlandy|rover: there's another tbarron I guess18:48
tbarronrlandy|rover: so I had to be tombarron there18:48
vkmco/18:48
rlandy|roverok :)18:48
rlandy|roverasked on rhos-ops18:48
tbarronrlandy++18:49
rlandy|roverweshay|ruck: ^^ we should be able to skip scenario004 in wallaby atm in criteria18:56
weshay|ruckk18:56
rlandy|roverediting criteria18:57
rlandy|roveron  promoter18:57
rlandy|rovergouthamr: tbarron: 38.102.83.191 user zuul18:59
rlandy|roverpls check you can ssh18:59
gouthamryes works!18:59
tbarronditto19:00
gouthamrvkmc: will add you ^19:00
rlandy|roverweshay|ruck: k .. restarting promoter service19:01
rlandy|roverugh - promoting master19:02
rlandy|roverwill wait until done19:02
*** slaweq has quit IRC19:13
*** mjturek has joined #oooq19:26
mjturekweshay|ruck: Hey I know it's been a minute but would love to sync up on publishing ppc containers soon19:27
mjturekwe just switched over to stream19:27
weshay|ruckmjturek, ah nice19:27
weshay|ruckok.. now? or later this week? I'm meeting w/ my promoter people tomorrow morning19:28
weshay|ruckbhagyashris, akahat ^19:28
mjturekwhenever works for you - if you'd like me to come tomorrow I'd be happy to19:30
weshay|ruckmjturek, it's early.. are you EST?.. 830am19:30
mjturektotally fine19:30
mjturekI have a baby boy now, gets me up nice and early :)19:30
weshay|ruckha ha.. ya.. you'll never sleep the same again :(19:31
weshay|ruckk.. adding oyu19:31
weshay|ruckyou19:31
mjturekhahaha, great thank you19:31
weshay|ruck:)19:31
weshay|ruckmjturek, CONGRATS btw19:31
mjturekthank you weshay|ruck !19:32
*** slaweq has joined #oooq19:32
tbarronrlandy|rover: we need to get https://review.opendev.org/c/openstack/manila/+/786912 merged in upstream manila for wallaby20:19
tbarronwill get done today20:19
tbarronDo you need to skip the testcase until it shows up?20:19
rlandy|rovertbarron: np - we are skipping scenario004 until this is sorted20:21
rlandy|roveryou have time :)20:21
tbarronrlandy|rover: ok, we'll stay on course then.  We're pretty sure this fix will do it.20:21
rlandy|rovertbarron: when that gets merged, I'll promote up the wallaby manila component20:21
tbarronfamous last words20:21
*** jmasud has joined #oooq20:22
rlandy|roverthanks for keeping on this20:22
gouthamrhey can we use the same patch to test things work?20:22
rlandy|rovergouthamr: sure - go4it20:22
rlandy|roverall yours20:22
*** slaweq_ has joined #oooq20:22
gouthamrawesome, ty - we'll get this merged in a bit and experiment20:23
rlandy|rovergouthamr++ thanks20:23
rlandy|rover2021-04-19 20:24:01,069 p=3111326 u=promoter n=ansible | fatal: [localhost]: FAILED! => {"changed": false, "msg": "There are missing containers"}20:27
rlandy|roverhmm20:27
rlandy|rover2021-04-19 20:25:43,191 p=3111326 u=promoter n=ansible | PLAY RECAP *********************************************************************20:31
rlandy|rover2021-04-19 20:25:43,191 p=3111326 u=promoter n=ansible | localhost                  : ok=27   changed=17   unreachable=0    failed=1    skipped=8    rescued=0    ignored=020:31
rlandy|roverk .. wallaby fail20:31
rlandy|roverrate-limit", "toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit", "toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit", "toomanyrequests: You have reached your pull rate limit. You may increase the limit by20:32
rlandy|roverauthenticating and upgrading: https://www.docker.com/increase-rate-limit", "toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit", "toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit",20:32
rlandy|rovertoo many requests20:32
rlandy|roverweshay|ruck: ^^ see that lately?20:32
rlandy|roverhttp://10.0.148.74/promoter_logs/container-push/20210419-192401.log20:32
rlandy|rovermaybe it will just rerun later and work?20:35
weshay|ruckrlandy|rover, happens ya20:47
rlandy|roverweshay|ruck: np - I guess it will just rerun20:52
rlandy|roverchandankumar: thanks for update on extended hash - looks much more consistent now20:58
weshay|ruckrlandy|rover, ok.. we'll have a keystone only puppet deployment now for 922:31
weshay|ruckrlandy|rover, got this working.. https://review.opendev.org/c/openstack/puppet-openstack-integration/+/786693/22:31
rlandy|rovernice22:56
*** rlandy|rover has quit IRC22:56
*** tosky has quit IRC23:23
« Sunday, 2021-04-18 Index Tuesday, 2021-04-20 »

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!