Thursday, 2021-01-21

*** jmasud has quit IRC00:46
*** jmasud has joined #oooq00:54
*** jmasud has quit IRC03:17
*** jmasud has joined #oooq03:25
*** skramaja has joined #oooq03:46
*** jmasud has quit IRC03:46
*** pojadhav|away is now known as pojadhav03:47
*** jmasud has joined #oooq04:28
*** ratailor has joined #oooq04:35
*** skramaja_ has joined #oooq04:52
*** skramaja has quit IRC04:52
*** ykarel has joined #oooq04:52
*** chem has quit IRC04:52
*** chem has joined #oooq04:53
*** jmasud has quit IRC05:02
*** udesale has joined #oooq05:22
*** jmasud has joined #oooq05:31
*** ysandeep|away is now known as ysandeep06:06
*** marios has joined #oooq06:15
*** ykarel_ has joined #oooq06:16
*** ykarel has quit IRC06:19
*** jmasud has quit IRC06:26
*** jmasud has joined #oooq06:31
*** saneax has joined #oooq06:37
*** ratailor_ has joined #oooq07:19
*** ratailor has quit IRC07:19
*** ykarel_ is now known as ykarel07:27
*** jpena|off is now known as jpena07:52
*** jmasud has quit IRC07:53
*** slaweq has joined #oooq07:58
*** bogdando has joined #oooq08:05
*** apetrich has joined #oooq08:06
*** amoralej|off is now known as amoralej08:07
*** tosky has joined #oooq08:16
*** jlarriba has joined #oooq08:51
*** jmasud has joined #oooq09:03
*** bogdando_ has joined #oooq09:34
*** ykarel_ has joined #oooq09:34
*** udesale_ has joined #oooq09:35
*** bogdando has quit IRC09:36
*** tosky has quit IRC09:36
*** bogdando_ is now known as bogdando09:36
*** tosky_ has joined #oooq09:36
*** ykarel has quit IRC09:37
*** udesale has quit IRC09:37
*** ykarel_ is now known as ykarel09:39
*** tosky_ is now known as tosky09:43
*** apetrich has quit IRC09:45
*** derekh has joined #oooq09:46
*** ysandeep is now known as ysandeep|afk10:14
*** jmasud has quit IRC10:25
*** apetrich has joined #oooq10:29
arxcruzslaweq: hey, need your help with some neutron stuff10:29
slaweqarxcruz: sure10:29
slaweqwhat's up?10:29
arxcruzslaweq: https://1213954fdcad8296313f-b58d3d982bad04bd40d81285b65fd93d.ssl.cf1.rackcdn.com/770767/5/check/tripleo-ci-centos-7-undercloud-oooq/33bdc5b/job-output.txt10:31
arxcruzslaweq: 2021-01-20 19:26:04.704601 | primary |         "response": "{\"NeutronError\": {\"message\": \"The resource could not be found.\", \"type\": \"HTTPNotFound\", \"detail\": \"\"}}"10:31
arxcruzslaweq: this is queens10:32
arxcruznot sure if we should be trying to add a router on queens or not10:32
slaweqarxcruz: checking10:32
slaweqis this tempest running on undercloud directly?10:33
slaweqif yes, then You should skip all L3 related tests as l3_router service_plugin isn't used on undercloud10:34
slaweqsee config file https://1213954fdcad8296313f-b58d3d982bad04bd40d81285b65fd93d.ssl.cf1.rackcdn.com/770767/5/check/tripleo-ci-centos-7-undercloud-oooq/33bdc5b/logs/undercloud/etc/neutron/neutron.conf10:34
slaweqthere is only "segments" plugin loaded in undercloud neutron10:34
arxcruzslaweq: the issue isn't with the test10:42
arxcruzit's with the tempest-setup10:42
slaweqarxcruz: so tempest-setup shouldn't create router on undercloud10:42
slaweqas there is no l3_plugin enabled there10:42
*** jlarriba has quit IRC10:43
*** jlarriba has joined #oooq10:45
*** jlarriba has joined #oooq10:46
arxcruzslaweq: thanks for the clarification10:47
arxcruzslaweq: should create the public network as well ?10:47
arxcruzi need then to submit a patch on the os_tempest10:47
slaweqpublic network should be fine from the neutron pov10:48
slaweqYou just can't create router/floating ip and such stuff10:48
arxcruzslaweq: the thing is, it can be avoided to create the router if i setup neutron service as false10:49
arxcruzhowever10:49
arxcruzdoing that, other stuff like private network, public network etc, won't be created10:49
arxcruzand i don't know if that's necessary10:49
slaweqtbh I don't know that too10:50
arxcruzlet me do a test10:50
arxcruz:)10:50
slaweqbut if You want to run some neutron related tests on undercloud then probably it may be needed10:50
slaweqworth to check10:50
*** dtantsur|afk is now known as dtantsur10:58
*** udesale__ has joined #oooq11:12
*** udesale_ has quit IRC11:15
*** ysandeep|afk is now known as ysandeep11:36
*** bogdando has left #oooq11:39
*** jpena is now known as jpena|lunch12:30
*** ratailor_ has quit IRC12:45
*** rlandy has joined #oooq12:46
rlandyweshay|ruck: ysandeep: hi - can we meet after scrum ti discuss downstream dlrn reviews?12:56
ysandeeprlandy, ack12:56
bhagyashrisweshay|ruck, sshnaidm|ruck zbr scrum13:02
weshay|ruckaye.. in a recruiting call13:02
bhagyashrisweshay|ruck, ok13:03
*** tosky has quit IRC13:06
*** tosky has joined #oooq13:06
*** amoralej is now known as amoralej|lunch13:09
*** jpena|lunch is now known as jpena13:29
*** ratailor has joined #oooq13:46
*** owalsh has quit IRC13:48
*** amoralej|lunch is now known as amoralej14:05
rlandyysandeep: weshay|ruck: https://review.rdoproject.org/r/#/q/topic:extended-hash+(status:open+OR+status:merged)14:06
*** owalsh has joined #oooq14:08
*** ykarel is now known as ykarel|mtg14:10
rlandyhttps://softwarefactory-project.io/r/#/c/20661/14:13
weshay|ruckbhagyashris, be there in a sec14:13
weshay|ruckbut lets chat14:13
bhagyashrisweshay|ruck, ack14:14
weshay|ruckbhagyashris, I'm in14:15
rlandyysandeep: ^^14:15
bhagyashrisweshay|ruck, https://meet.google.com/pqc-ukza-ihs?authuser=014:16
weshay|ruckakahat|rover, pojadhav jump back in14:19
weshay|ruckwe're done14:19
weshay|ruckbhagyashris, akahat|rover14:30
weshay|ruck   Memory: 35.5M                                                                                                                                                              │················14:30
weshay|ruck   CGroup: /system.slice/dlrn-promoter.service                                                                                                                                │················14:30
weshay|ruck           ├─178270 /bin/bash /home/promoter/ci-config/ci-scripts/dlrnapi_promoter/dlrn-promoter-service.sh                                                                   │················14:30
weshay|ruck           └─311581 sleep 600                                                                                                                                                 │················14:30
weshay|ruck                                                                                                                                                                              │················14:30
weshay|ruckJan 21 14:20:56 promoter-rocky bash[178270]: Traceback (most recent call last):                                                                                               │················14:30
weshay|ruckJan 21 14:20:56 promoter-rocky bash[178270]:   File "/home/promoter/ci-config/ci-scripts/dlrnapi_promoter/dlrnapi_promoter.py", line 135, in <module>                         │················14:30
weshay|ruckJan 21 14:20:56 promoter-rocky bash[178270]:     main()                                                                                                                       │················14:30
weshay|ruckJan 21 14:20:56 promoter-rocky bash[178270]:   File "/home/promoter/ci-config/ci-scripts/dlrnapi_promoter/dlrnapi_promoter.py", line 119, in main                             │················14:31
weshay|ruckJan 21 14:20:56 promoter-rocky bash[178270]:     CONFIG_RELEASE))                                                                                                             │················14:31
weshay|ruckJan 21 14:20:56 promoter-rocky bash[178270]:   File "/home/promoter/ci-config/ci-scripts/dlrnapi_promoter/common.py", line 210, in get_log_file                               │················14:31
weshay|ruckJan 21 14:20:56 promoter-rocky bash[178270]:     config_file = yaml.safe_load(open(log_file_path))                                                                            │················14:31
weshay|ruckJan 21 14:20:56 promoter-rocky bash[178270]: FileNotFoundError: [Errno 2] No such file or dir14:31
*** ratailor has quit IRC14:43
*** ratailor has joined #oooq14:43
*** ratailor has quit IRC14:45
*** ratailor has joined #oooq14:45
*** ratailor has quit IRC14:46
*** sshnaidm|ruck is now known as sshnaidm|afk14:49
*** TrevorV has joined #oooq14:50
rlandymarios: what's left of your reviews to watch this afternoon?14:52
marioshttps://review.opendev.org/c/openstack/tripleo-common/+/771371 is in the gate, then we can merge the rest of https://review.opendev.org/q/topic:%22tripleo-c7-update-upgrade-removal%22+(status:open%20OR%20status:merged) the tripleo-ci ones14:54
mariosrlandy: please thanks ^14:54
zbrarxcruz: can you please help me test https://review.opendev.org/c/openstack/ansible-role-collect-logs/+/771807/ ? --- just download it and run "tox -e sanity".14:54
zbrit should do several things and endup with a like like: ERROR: The 11 sanity test(s) listed below (out of 33) failed.14:55
arxcruzzbr: sure, just a sec14:55
rlandyok14:57
rlandymarios: https://review.opendev.org/c/openstack/tripleo-ci/+/770151 "invalid configuration"?14:59
rlandyI guess that set is all the same - until merge?15:00
mariosrlandy: yeah we need the others to merge first but not sure that one might be legit nit that was posted by weshay15:00
rlandymarios: k - will relook at it15:00
arxcruzzbr: http://paste.openstack.org/show/801826/15:00
zbri know, no worries.15:01
mariosrlandy: but for sure zuul will complain about the config until it merges in all the repos cos we were referring to those jobs. then we can merge the 4 tripleo-ci ones from https://review.opendev.org/q/topic:%22tripleo-c7-update-upgrade-removal%22+(status:open%20OR%20status:merged)15:01
rlandyah ok15:01
zbrarxcruz: i am only interested about results from "tox -e sanity -- --python 3.6" (or whatever python you have)15:02
rlandyweshay|ruck: the internal promotion scripts need a three line change  (and a dlrn update) - if you have time this afternoon we can hack that together15:03
arxcruzzbr: i just ran the tox -e sanity that you asked and that's the output15:06
zbrarxcruz: yeah, that appears to be a bug in ansible which does not list pip as a dependency, i avoided locally by adding it to reqs.15:07
arxcruzzbr: ok, i'll try again15:07
zbrarxcruz: thanks. i will ask you to do it again after if few these few annoyances.15:08
zbrat least I am happy that it went so far.15:08
*** ykarel|mtg is now known as ykarel15:08
arxcruzzbr: you're adding pip on requirements.txt ?15:10
zbrhmm, i looked again. is not ansible itself.15:11
zbrthe error comes from py27~15:11
zbransible-test sanity runs tests with all versions of python found15:12
zbrso this happens because there is no pip on your local py27.15:12
zbrjust run: tox -e sanity -- --python 3.915:12
zbrthis should run it only for python 3.9, and likely you should only 11/3315:13
*** irclogbot_3 has quit IRC15:21
*** irclogbot_3 has joined #oooq15:21
weshay|ruckysandeep, rlandy any objects to me promting all the osp-17 components from component-ci-testing --> promoted-components?15:22
ysandeepweshay|ruck, go for it o/15:22
rlandyweshay|ruck: pls15:28
*** skramaja_ has quit IRC15:28
ykarelweshay|ruck, btw wrt ^ was it known why issue didn't got hit in component(clients) pipeline?15:32
ykareli mean recent escalation https://bugzilla.redhat.com/show_bug.cgi?id=191759515:32
openstackbugzilla.redhat.com bug 1917595 in python-os-brick "cinder_api_db_sync fails w/ ImportError: cannot import name 'md5'" [Urgent,Modified] - Assigned to cinder-bugs15:32
ykarelthis should have been caught in component pipeline itself instead of integration, likely some issue there which lead to false promotion and hit in integration15:33
mariosweshay|ruck: fyi & vote please @ https://review.opendev.org/c/openstack/puppet-tripleo/+/771829 https://review.opendev.org/c/openstack/releases/+/771830 http://lists.openstack.org/pipermail/openstack-discuss/2021-January/019839.html15:33
ykarelwondering if it's issue in framework, then more such issues will be missed15:35
ykarellikely containers were not got updated due to some reason15:36
weshay|ruckfrenzy_friday, be there in a sec15:38
frenzy_fridayweshay|ruck, np15:38
*** bogdando has joined #oooq15:49
bogdandohi, where do we add things from CI logs to logstash ?15:49
*** jmasud has joined #oooq15:51
*** sshnaidm|afk is now known as sshnaidm|ruck15:56
sshnaidm|ruckbogdando, https://github.com/openstack/ansible-role-collect-logs/blob/master/defaults/main.yml#L35715:57
bogdandosshnaidm|ruck: thanks15:58
weshay|ruckpojadhav, akahat|rover is the issue w/ the cockpit resolved?16:14
weshay|ruckfor promotions?16:14
pojadhavweshay|ruck, patch is up and tested locally yesterday with akahat|rover : https://review.rdoproject.org/r/#/c/31318/16:16
pojadhavneed to merge this16:16
weshay|ruckakahat|rover, you going to vote?16:16
weshay|rucksshnaidm|ruck, fyi https://review.rdoproject.org/r/#/c/31318/16:17
weshay|ruckthanks pojadhav16:17
sshnaidm|ruckweshay|ruck, ack16:21
sshnaidm|ruckcan we merge it?16:21
*** pojadhav is now known as pojadhav|away16:23
*** udesale__ has quit IRC16:24
weshay|rucksshnaidm|ruck, /me tries it locally16:25
weshay|ruckpojadhav|away, you have tests coming for this yes?16:25
pojadhav|awayweshay|ruck, yes I will write tests :)16:26
weshay|rucksshnaidm|ruck, doesn't work16:27
sshnaidm|ruck;(16:27
bogdandosshnaidm|ruck: I cannot figure out how errors.txt.txt ends up shown in logstash16:27
bogdandoit is not listed in the logstash files16:27
weshay|ruck(python3) [whayutin@localhost telegraf]$ python last_promotions.py --distro=CentOS-8 --release=victoria16:28
weshay|ruckTraceback (most recent call last):16:28
weshay|ruck  File "/home/whayutin/git/RDO-CI/TRIPLEO-QUICKSTART/ci-config/ci-scripts/infra-setup/roles/rrcockpit/files/telegraf/last_promotions.py", line 119, in <module>16:28
weshay|ruck    promotions = get_promotion(16:28
weshay|ruck  File "/home/whayutin/git/RDO-CI/TRIPLEO-QUICKSTART/ci-config/ci-scripts/infra-setup/roles/rrcockpit/files/telegraf/last_promotions.py", line 53, in get_promotion16:28
weshay|ruck    dlrn_client = promoter_utils.get_dlrn_client(promoter_config)16:28
weshay|ruck  File "/home/whayutin/git/RDO-CI/TRIPLEO-QUICKSTART/ci-config/ci-scripts/infra-setup/roles/rrcockpit/files/telegraf/promoter_utils.py", line 31, in get_dlrn_client16:28
weshay|ruck    api_client = dlrnapi_client.ApiClient(host=config['api_url'])16:28
bogdandoand logstash identifies it as filename:"logs/undercloud/var/log/extra/errors.txt.txt"16:28
sshnaidm|ruckbogdando, where is it..?16:29
sshnaidm|ruckbogdando, maybe it's not in logstash, but indexed separately16:29
sshnaidm|ruckbogdando, we send a few to logstash, console, combined logstash file from various logs, and error.txt separately I think16:30
sshnaidm|ruckbogdando, this config is somewhere in elastic-* repos16:30
sshnaidm|ruckzbr sure knows16:30
*** ysandeep is now known as ysandeep|away16:31
sshnaidm|ruckweshay|ruck, we'll need to do something with it eventually http://dashboard-ci.tripleo.org/d/3-DYSmOGk/jobs-exploration?var-influxdb_filter=job_name%7C%3D~%7C%2Fcontent-provider%2F&var-influxdb_filter=cloud%7C%3D%7Cvexxhost&var-influxdb_filter=region%7C%3D%7Cca-ymq-1&var-influxdb_filter=result%7C%3D%7CTIMED_OUT16:37
weshay|rucksshnaidm|ruck, is it isolated to just that region?16:38
sshnaidm|ruckpass rate of content providers on vexxhost upstream is about 65%16:38
sshnaidm|ruckyes16:38
weshay|rucksshnaidm|ruck, will you please write a tracker bug in lp ... promotion-blocker16:38
sshnaidm|ruckweshay|ruck, remove result and you'll see overall rate16:38
weshay|ruckand I'll have rdo - infra open a ticket w/ mohammed16:38
sshnaidm|ruckweshay|ruck, ok16:38
weshay|rucksshnaidm|ruck, please include the negative case w/ ca-ymq-1 and then the results from a working region16:39
*** marios is now known as marios|out16:44
bogdandosshnaidm|ruck, weshay|ruck: https://review.opendev.org/c/openstack/ansible-role-collect-logs/+/77185116:48
sshnaidm|ruckbogdando, ack16:48
sshnaidm|ruckweshay|ruck, https://bugs.launchpad.net/tripleo/+bug/191266316:48
openstackLaunchpad bug 1912663 in tripleo "Content provider jobs are often timeouting on upstream vexxhost ca-ymq-1 region" [Critical,Triaged]16:48
rlandysshnaidm|ruck: weshay|ruck: ha - ok - https://review.rdoproject.org/zuul/build/e082f36d83e148f08f6d42a2581413ad16:48
weshay|rucksshnaidm|ruck++16:49
rlandynice16:49
sshnaidm|ruckrlandy, great16:49
rlandysshnaidm|ruck: weshay|ruck: k - merging https://review.rdoproject.org/r/#/c/31500/16:49
rlandyoh wait16:50
rlandydefinition in wring place16:50
sshnaidm|ruckrlandy, did you see my change there?16:50
rlandyupdating16:50
rlandysshnaidm|ruck: yep - but bot changes should be on job down16:50
rlandyupdate coming up16:50
rlandynot all multiple dependency will be stream16:51
sshnaidm|ruckrlandy, I set there ansible version in parent job, otherwise it was running on 2.816:52
rlandysshnaidm|ruck: ++ the change is right - it's just one parent job too high up16:52
weshay|ruckrlandy, promoted osp-17 component-ci-testing -> promoted-components16:54
rlandyweshay|ruck++16:54
rlandyovercloud builds should pass there now as well16:54
rlandyhttps://review.rdoproject.org/r/31500 Run centos 8 streamdeps jobs on centos 8 stream nodes16:55
rlandysshnaidm|ruck: ^^ only when the dependency is stream we shoudl change the nodeset and ansible version16:55
rlandyweshay|ruck: ^^16:55
rlandyrerunning testproject check16:56
sshnaidm|ruckrlandy, tbh idk why not to run 2.9 everywhere16:56
rlandysshnaidm|ruck: that's a different change16:57
rlandyin dependencies we are changing one thing at a time16:57
sshnaidm|ruckyeah16:57
rlandyif we switch to ansubel 2.9 everywhere, there is fine but the change should be different jobs16:57
rlandyie: centos 8 released with ansible 2.9 may be a diff ball game16:58
rlandyprogress!!16:58
*** jpena is now known as jpena|off16:59
sshnaidm|ruckrlandy, actually we'll need to change it here: https://github.com/openstack/tripleo-ci/blob/master/zuul.d/base.yaml#L10217:00
rlandyyep17:00
rlandyor even base rdo17:03
rlandyif we wanted periodics first17:03
zbrarxcruz: see if you can download it again.17:03
*** marios|out has quit IRC17:03
*** pojadhav|away is now known as pojadhav17:08
*** ykarel has quit IRC17:09
*** ykarel has joined #oooq17:11
*** amoralej is now known as amoralej|off17:11
*** ykarel has quit IRC17:17
akahat|roverweshay|ruck, i've voted on: https://review.rdoproject.org/r/#/c/31318/17:22
weshay|ruckyou test it?17:22
akahat|roverweshay|ruck, yes. it's working http://paste.openstack.org/show/801844/17:22
weshay|ruckk.. ya.. that looks right17:23
weshay|ruckthanks17:23
*** rlandy is now known as rlandy|biab17:43
*** bogdando has quit IRC17:48
*** frenzy_friday has quit IRC18:11
*** frenzy_friday has joined #oooq18:12
*** dtantsur is now known as dtantsur|afk18:32
*** amoralej|off is now known as amoralej18:40
*** rlandy|biab is now known as rlandy18:41
*** amoralej is now known as amoralej|off19:24
rlandyweshay|ruck: anything we need to touch base on?19:29
rlandyhttps://softwarefactory-project.io/r/#/c/20661/ - waiting on that voting patch to merge19:29
rlandyhope to merge dlrn changes tomorrow19:29
rlandyplus on server19:29
rlandyhttps://review.opendev.org/c/openstack/tripleo-ci/+/770151 - invalid config - checking on that19:30
rlandyah19:53
*** slaweq has quit IRC19:55
weshay|ruckrlandy, few min20:00
rlandyweshay|ruck: no rush20:03
*** irclogbot_3 has quit IRC20:14
*** tosky has quit IRC20:14
*** tosky has joined #oooq20:15
*** irclogbot_0 has joined #oooq20:16
*** jmasud has quit IRC20:34
*** jmasud has joined #oooq20:44
*** TrevorV has quit IRC21:12
weshay|ruckrlandy, sorry.. need anything?21:13
rlandyweshay|ruck: lol - actually need to run out in a few ... but  - your changes to remove updates have errors21:14
rlandyone I commented on21:14
rlandytwo actualy21:14
rlandyall the rest is merged there21:15
rlandythere are four invalid config errors21:15
weshay|ruckk21:15
rlandyweshay|ruck: other than that, nothing urgent - 8.4 node worked - centos8 stream node change is merged21:15
rlandybusy writing spec21:16
rlandywill ping you tomorrow re: dlrn changes21:16
rlandysandeep reviwed21:16
rlandythink we are ok there21:16
*** rlandy is now known as rlandy|biab21:17
*** owalsh has quit IRC21:26
*** derekh has quit IRC21:37
*** owalsh has joined #oooq21:44
*** derekh has joined #oooq22:06
*** derekh has quit IRC22:06
*** jmasud has quit IRC22:10
*** jmasud has joined #oooq22:12
*** owalsh has quit IRC22:16
*** rlandy|biab is now known as rlandy22:20
*** owalsh has joined #oooq22:25
*** holser has quit IRC23:17
*** jmasud has quit IRC23:19
*** holser has joined #oooq23:19
*** jmasud has joined #oooq23:22
*** rlandy has quit IRC23:26
*** holser has quit IRC23:35

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!