Wednesday, 2019-05-29

*** rlandy has quit IRC00:10
*** hamzy has joined #oooq00:20
*** Goneri has joined #oooq00:21
*** Vorrtex has quit IRC00:31
*** sshnaidm has quit IRC00:44
*** Goneri has quit IRC01:00
*** dsneddon has quit IRC01:32
*** sshnaidm has joined #oooq01:50
*** dsneddon has joined #oooq01:50
*** dsneddon has quit IRC01:55
*** apetrich has quit IRC01:57
*** dsneddon has joined #oooq02:09
*** dsneddon has quit IRC02:14
*** dsneddon has joined #oooq02:21
*** dsneddon has quit IRC02:27
*** dsneddon has joined #oooq02:35
*** dsneddon has quit IRC02:39
*** dsneddon has joined #oooq02:58
*** dsneddon has quit IRC03:03
*** panda|ruck has quit IRC03:03
*** panda has joined #oooq03:03
*** dsneddon has joined #oooq03:04
*** dsneddon has quit IRC03:11
*** dsneddon has joined #oooq03:19
*** altlogbot_1 has quit IRC03:44
*** altlogbot_3 has joined #oooq03:45
*** dsneddon has quit IRC03:47
*** ykarel|away has joined #oooq03:53
*** ykarel|away has quit IRC03:53
*** ykarel|away has joined #oooq03:55
*** ykarel|away has quit IRC03:55
*** ykarel|away has joined #oooq03:56
*** ykarel|away is now known as ykarel03:58
*** ykarel has quit IRC04:05
*** ykarel has joined #oooq04:08
*** ykarel has quit IRC04:14
*** ykarel has joined #oooq04:34
*** dsneddon has joined #oooq04:34
*** ratailor has joined #oooq04:34
*** raukadah is now known as chandankumar04:37
*** dsneddon has quit IRC04:38
*** altlogbot_3 has quit IRC04:38
*** altlogbot_1 has joined #oooq04:39
*** rascasoft has quit IRC04:39
*** altlogbot_1 has quit IRC04:40
*** altlogbot_3 has joined #oooq04:41
*** rascasoft has joined #oooq04:42
*** marios has joined #oooq05:25
*** dsneddon has joined #oooq05:28
*** dsneddon has quit IRC05:33
*** quiquell has joined #oooq05:34
*** dsneddon has joined #oooq05:59
*** dtantsur|afk is now known as dtantsur06:00
*** jfrancoa has joined #oooq06:05
chandankumarmarios: https://review.opendev.org/#/c/661699/ and https://review.opendev.org/#/c/661697/ are good to go06:38
mariosack chandankumar06:45
chandankumarmarios: thanks!06:45
marioschandankumar: and https://review.opendev.org/#/c/661694 you forgot that one06:45
marios;)06:45
chandankumarmarios: yes, thanks :-)06:46
*** saneax has joined #oooq06:49
*** ykarel is now known as ykarel|afk06:51
*** holser_ has joined #oooq06:52
*** tesseract has joined #oooq07:05
*** tosky has joined #oooq07:11
*** amoralej|off is now known as amoralej07:22
*** dsneddon has quit IRC07:27
chandankumarmarios: https://code.engineering.redhat.com/gerrit/#/c/171227/ please have a look at this oen07:27
*** ykarel|afk is now known as ykarel07:30
*** apetrich has joined #oooq07:32
chandankumarykarel: marios it is quite confusing now queens fs020 https://logs.rdoproject.org/openstack-periodic-24hr/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset020-queens/00e8f34/logs/tempest.html.gz is passing now07:33
ykarel:( u checked the relevant tests?07:35
ykarelbackup ones and tempest config07:35
chandankumarit is getting skipped07:36
ykarelyup because of backup = False in tempestconf07:37
ykarelso u need to find what brings ^^07:37
ykarellast what i checked tempestconf takes care of it07:38
ykarelbut how it started coming suddenly that needs to be find07:38
chandankumarhttps://logs.rdoproject.org/openstack-periodic-24hr/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset020-queens/2a849bc/logs/tempest.html.gz -> from 20th may, it was getting skipped07:39
*** tosky has quit IRC07:40
*** tosky has joined #oooq07:42
*** bogdando has joined #oooq07:45
*** ykarel_ has joined #oooq07:55
*** ykarel has quit IRC07:56
*** dsneddon has joined #oooq07:57
*** ykarel_ is now known as ykarel07:58
zbrrfolco: why we have no job testing https://code.engineering.redhat.com/gerrit/#/c/171297/6 ? it will never merge if it does not have a job. or is something broken?07:58
*** jpena|off is now known as jpena07:58
mariosbiab08:00
*** marios has quit IRC08:00
*** kopecmartin|off is now known as kopecmartin08:09
*** panda is now known as panda|ruck08:23
*** ykarel is now known as ykarel|lunch08:30
*** sshnaidm has quit IRC08:30
*** sshnaidm has joined #oooq08:31
chandankumarpanda|ruck: Hello08:35
panda|ruckchandankumar: hi08:35
chandankumarpanda|ruck: regarding 2 compute nodes patch https://review.rdoproject.org/r/#/c/20902/ is it ok? do we need to add something for branches attr for check jobs?08:36
panda|ruckchandankumar: is your goal to run it in periodic ?08:36
chandankumarpanda|ruck: there are two types of jobs one in check and other in periodic08:37
panda|ruckchandankumar: also after reading zuul documentation, I stil don't understand the dynamic of implicit branches: attribute, so yes, even knowing what I said to you, it's probably better to just put branches: ".*" just to be sure that all the branches are processed08:38
chandankumarpanda|ruck: we have to run both jobs08:38
chandankumarpanda|ruck: ok, then!08:38
panda|ruckchandankumar: ok than you're just missing the pipeline configuration08:38
*** derekh has joined #oooq08:39
chandankumarpanda|ruck: https://review.rdoproject.org/r/#/c/20898/ and https://review.rdoproject.org/r/#/c/20919/ answers the query08:39
panda|ruckugh standalone fedora is failing a bunch again08:43
panda|ruckin gate08:43
panda|ruckand it's voting08:43
* panda|ruck charges the cannon08:43
chandankumarpanda|ruck: https://review.rdoproject.org/r/#/c/20965/08:56
chandankumarpanda|ruck: does rhel-8 container images available? I need to some tempest package testing for os_tempest08:57
panda|ruckchandankumar: we have successfully built them08:58
panda|ruckchandankumar: rfolco is working on making them available08:58
chandankumargreat, I will wait for him to come online08:58
panda|ruckchandankumar: an ambush. I like it.08:59
*** marios has joined #oooq09:04
*** ykarel|lunch is now known as ykarel09:11
panda|ruckmaster phase1 promotion job is running09:14
mariospanda|ruck: how are things generally do you need help with something/any bugs unattended - and are you tracking somewhere ?09:24
chandankumarkopecmartin: Hello09:28
rfolcopanda|ruck, chandankumar I am working on rhel7 containers.09:28
rfolconot rhel809:28
chandankumarkopecmartin: do we want to update tempest version for queens?09:28
kopecmartinchandankumar, yes, as the task says https://tree.taiga.io/project/tripleo-ci-board/task/103909:29
mariossshnaidm: rfolco: panda|ruck fyi updated that with .* please check if you have time - pinged fbo in sf-ops too09:29
rfolcozbr, I will test container push after merging that patch09:29
marioshttps://code.engineering.redhat.com/gerrit/#/c/171703/3/zuul.d/standalone-jobs.yaml  'that' sshnaidm rfolco panda|ruck09:29
kopecmartinchandankumar, i don't know about upstream, but downstream for sure09:29
kopecmartinchandankumar, i would bacported the required patches and build packages, but I'm stuck on this https://review.opendev.org/#/c/657018/ i'm still waiting for review09:30
kopecmartinchandankumar, I'll attend QA meeting tomorrow and I'll try to remind it09:30
kopecmartinget some attention to it09:31
chandankumarkopecmartin: https://tree.taiga.io/project/tripleo-ci-board/task/1039?kanban-status=144727509:31
*** saneax has quit IRC09:32
rfolcomarios, is zuul happy in this modern world? does he need more?09:32
mariosrfolco: well don't know yet and i think we'll only find out if we merge ;)09:32
*** saneax has joined #oooq09:32
rfolcomarios, k :)09:33
chandankumarrfolco: sshnaidm panda|ruck please have a look at these series of patches for skip list https://review.opendev.org/#/q/topic:tempest_skip_massupdate+(status:open+OR+status:merged)09:37
chandankumarkopecmartin: you mean upstream qa meeting?09:39
kopecmartinchandankumar, yes09:39
chandankumarok09:40
panda|ruckmarios: everything's green, there are a bunch of bugs in alert, but they don't really seem workable or urgent09:43
*** tcw has quit IRC09:43
*** tcw has joined #oooq09:45
mariospanda|ruck: do not repeat those words again... ack09:47
panda|ruckmarios: ?09:47
mariospanda|ruck: ('everything's fine' ... then its not)09:47
mariospanda|ruck: "don't jinx it"09:48
panda|ruckmarios: taht's ok, I didn't pretend I was being optimistic about it09:48
zbrwas anyone able to use zuul API from internal sf to gather build status? apparently it returns [] on any tenant.10:00
zbr(i was trying to make the greasemonkey script we use upstream and on rdo to work with it)10:00
chandankumarmarios: ykarel https://bugs.launchpad.net/tripleo/+bug/1830351/comments/7 what do you think?10:06
openstackLaunchpad bug 1830351 in tripleo "periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset020-queens fails tempest run for tempest.api.volume.admin.test_volumes_backup test group" [Critical,Triaged] - Assigned to chandan kumar (chkumar246)10:06
ykarelchandankumar, u got what put backup=false in current periodic fs020 job?10:07
ykarelas it's passing already10:07
* ykarel read back if u already answered it10:08
chandankumarykarel: from the run it is coming false positive sometime it is passing and sometime it is failing10:08
ykarelchandankumar, test can be false positive, but i don't think tempest config should be generated by tooling only10:09
ykarels/should/is10:09
ykarelhttps://logs.rdoproject.org/openstack-periodic-24hr/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset020-queens/2a849bc/logs/undercloud/home/zuul/tempest/etc/tempest.conf.txt.gz10:10
ykarel[volume-feature-enabled]10:10
ykarelbackup = False10:11
chandankumarykarel: https://logs.rdoproject.org/openstack-periodic-24hr/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset020-queens/6bc873a/logs/undercloud/home/zuul/tempest.log.txt.gz#_2019-05-29_06_51_3910:11
chandankumarin failed one10:11
chandankumarNo volume service found, skipping backup service check10:11
mariosykarel: chandankumar ack so bumping to 19 should get rid of the problem? a bit cnfused by " 2. Then merge the revert" then " 2.2 Merge the revert"10:12
marioschandankumar: we can merge that - i.e. get rid of fake v1 endpoints, when v3 support comes in v19 tempest?10:12
marioschandankumar: 'that' https://review.opendev.org/#/c/661707/3/puppet/services/cinder-api.yaml ?10:13
ykarelchandankumar, yes that is only the question10:13
ykarelhow backup = False is set in passing one and not in failing one10:13
chandankumarykarel: that is the question from where it is coming10:13
ykarel^^ possible with tht revert, but how it set without revert10:13
ykarelchandankumar, yes, good to find that10:15
chandankumarykarel: do we capture openstack endpoint list or catalog list?10:16
chandankumarin logs?10:16
ykarelchandankumar, i don't think that is explictilty collected, but u might find in tempest and other services logs10:16
*** jfrancoa has quit IRC10:19
*** jfrancoa has joined #oooq10:22
chandankumarpanda|ruck: sshnaidm https://review.rdoproject.org/r/#/c/20965/ this one is needed for 2 comp branches fixes10:24
chandankumarotherwise jobs got triggered due to branch issue it might not work10:24
chandankumarykarel: that data is not available in tempest as it totally depends on tempest.conf values10:25
ykarelchandankumar, which data?10:26
chandankumarykarel: openstack endpoint list10:26
chandankumarto better guess whether it is volume or volume310:26
ykarelchandankumar, ^^ is part of token afaik10:26
ykareli can see it in logs10:26
chandankumarone in10:27
*** dtantsur is now known as dtantsur|lunch10:43
chandankumarmarios: ykarel take a alook at comment 8 https://bugs.launchpad.net/tripleo/+bug/1830351/10:56
openstackLaunchpad bug 1830351 in tripleo "periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset020-queens fails tempest run for tempest.api.volume.admin.test_volumes_backup test group" [Critical,Triaged] - Assigned to chandan kumar (chkumar246)10:56
marioschandankumar: maybe you need to ask someone in storage to check that about why/how its sometimes created/not maybe there is oemthign else you can check in logs we don't know about10:57
ykarelchandankumar, sure will check in few minutes11:00
*** dsneddon has quit IRC11:17
*** ratailor has quit IRC11:18
*** jpena is now known as jpena|lunch11:30
*** ykarel_ has joined #oooq11:39
*** ykarel has quit IRC11:43
*** dsneddon has joined #oooq11:46
*** tesseract has quit IRC11:53
*** tesseract has joined #oooq11:53
*** ykarel_ is now known as ykarel11:55
panda|ruckwe're starting to have some problems with the gates11:57
panda|ruckbut the failures don't seems to be connectted11:57
panda|ruckfedora standalone failing a bunch, but last failure was a network error to github during build-test-package11:58
chandankumarrfolco: panda|ruck I am trying to use rhel8 ubi image for testing12:10
chandankumarfrom here https://access.redhat.com/containers/?tab=images#/registry.access.redhat.com/ubi8/ubi12:10
weshaychandankumar panda|ruck can you guys trigger some osp-15 standalone jobs on the latest need_images12:14
weshayrlandy can assist12:15
panda|ruckweshay: ok12:15
sshnaidmchandankumar, commented https://review.rdoproject.org/r/#/c/20965/12:17
*** Goneri has joined #oooq12:17
chandankumarsshnaidm: the places you commented is going to removed in a review12:18
chandankumarthose are 1comp jobs12:18
chandankumarwe have added the 2comp fs020/fs021 jobs for the same by duplicating it12:19
*** Goneri has quit IRC12:20
sshnaidmchandankumar, I see, +212:20
*** marios has quit IRC12:20
chandankumarsshnaidm: thanks!12:20
chandankumarsshnaidm: panda|ruck https://review.rdoproject.org/r/#/c/20965/ once it merges, we can merge this one https://review.rdoproject.org/r/#/c/20919/ to consume in promotion12:21
*** aakarsh has quit IRC12:24
*** udesale has joined #oooq12:25
chandankumarI found the os_tempest downstream issue12:25
ykarelchandankumar, i checked ur comment in fs020-queens, i think the issue is happening because of duplicated cinder service names(cinderv3) with different service type(volume,volumev3)12:30
ykareland that's why we see random SUCCESS in periodic pipeline12:30
*** amoralej is now known as amoralej|off12:30
ykarelabishop and u can relate more on it as it's related to tempestconf and cinder12:30
*** amoralej|off is now known as amoralej|lunch12:30
chandankumarpanda|ruck: sshnaidm zbr rfolco https://gitlab.cee.redhat.com/snippets/926 some fun with ubi12:34
*** jpena|lunch is now known as jpena12:34
*** rlandy has joined #oooq12:35
sshnaidmchandankumar, nice, saw these ubi images, didn't have chance to play with it yet12:35
sshnaidmchandankumar, can we install undercloud on it? :D12:36
chandankumarsshnaidm: never tried that on undercloud12:36
chandankumarsshnaidm: we just need the login credentails and we are done12:37
chandankumarit is just like other images12:37
chandankumarrlandy: https://gitlab.cee.redhat.com/snippets/926 some fun with ubi12:37
rlandywow - interesting12:38
*** Goneri has joined #oooq12:38
chandankumarrlandy: now I am preparing the patch for os_Tempest stuff12:38
panda|ruckrlandy: hey, can you help me triggering a standalone run on the need_images ?12:41
rlandypanda|ruck: yep - getting job12:41
rlandypanda|ruck: https://code.engineering.redhat.com/gerrit/#/c/171119/12:42
rlandypanda|ruck: there is an issue with the updated containers12:44
rlandyweshay had emailed steve baker12:44
rlandythe suggestion was to remove the port12:44
rlandywe can try that if you need the job working12:45
rlandyin fact, I am trying it in my regular rhos-15 job12:45
weshayrlandy ya, thanks I find it odd that is not needed in tripleo12:46
rlandyweshay: I still don't get that12:46
rlandynote that it is defined in two places ...12:46
rlandystandalone_parameters.yaml and containers-prepare-parameters.yaml12:47
rlandyboth use the port12:47
rlandythe one passed last is the standalone_parameters.yaml12:47
rfolcopanda|ruck, my job is merged in config, why it is not running in check https://code.engineering.redhat.com/gerrit/#/c/171729?12:47
weshayrlandy so we need one more release file for osp-15.. just talking to burns12:48
weshayhttp://download.devel.redhat.com/rcm-guest/puddles/OpenStack/15.0-RHEL-8/latest-RHOS_TRUNK-15-RHEL-8/12:48
rlandyweshay: that is what I was going to ask you about yesterday12:48
weshayso the link is .. latest-RHOS_TRUNK-15-RHEL-8/12:49
rlandythe current rhos-15 job is passing12:49
rlandybut it's not updating the right containers12:49
rlandyI think the tag combination is wrong12:49
rlandylook at this12:49
weshayrlandy ya.. we're not running in check, so not updating the containers is fine for now12:50
weshayzbr panda|ruck so we need to help debug https://rhos-qe-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/view/ReleaseDelivery/view/OSP15/job/phase1-15_director-rhel-8.0-virthost-1cont_1comp-ipv4-geneve-lvm/  which the standalone on osp-15 will help w/ that12:50
rlandyhttps://sf.hosted.upshift.rdu2.redhat.com/logs/periodic/code.engineering.redhat.com/openstack/tripleo-ci-internal-jobs/master/tripleo-ci-rhel-8-standalone-rhos-15/622f38b/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz#_2019-05-29_00_30_2512:50
rlandyworks fine12:50
rlandy: Pull 192.168.24.1:8787/rhosp15/openstack-cinder-volume:20190523.112:50
weshayrlandy me looks..12:50
rlandy^^ is legit12:50
rlandybut this is not12:51
panda|ruckrfolco: because you marked it as abstract12:51
rfolcopanda|ruck, holy sheet! spreadsheet!12:52
panda|ruckwax on, wax off.12:52
rlandy2019-05-28 13:06:00 | TASK [tripleo-container-tag : Pull 192.168.24.1:8787/rhosp15/openstack-cinder-volume:20190523.1-updated-20190528084845 image] ***]12:52
rlandyidk12:53
rlandywierdness12:53
weshayrlandy I think it doesn't matter re: the tag.. ie the container update will be skipped if we have  modify_only_with_labels:12:53
weshay        - kolla_version12:53
weshayin the params12:53
rlandyI did12:53
rlandyhttps://review.opendev.org/#/c/659794/11/roles/standalone/tasks/main.yml line 10312:54
zbrweshay: i am going to try to run it on my hw. i hope running it on centos-7 virthost does not invalidate it.12:54
rlandyweshay: will skip the whole thing and run again12:55
*** aakarsh has joined #oooq12:55
weshayzbr ya.. it's fine as long as the vm is rhel-813:00
* rascasoft waves all his old friends :)13:00
weshayrascasoft helloooooo13:01
rlandyrascasoft: hey - how goes it??13:01
rlandyweshay: going to do one run w/o port number for sanity13:01
rlandybut we can't just skip the one kolla setting - we need to skip them all13:01
rlandy^^ if we decide to do that13:02
rascasoftweshay, rlandy, now that I'm reading you not just good, I'd say best13:02
rlandybecause the modify_append_tag: "-updated-20190528084845" is still added13:02
rfolcopanda|ruck, rlandy: would you please merge this one? disable abstract attribute: https://code.engineering.redhat.com/gerrit/#/c/171820/13:05
panda|ruckrfolco: does this fiel contain only bases ?13:07
rlandyrfolco: done13:07
rfolcopanda|ruck, not sure what you mean13:07
*** marios has joined #oooq13:08
*** amoralej|lunch is now known as amoralej13:12
*** dsneddon has quit IRC13:13
*** ykarel is now known as ykarel|afk13:16
panda|ruckok, now we have a standalone centos7 job in gates consistently failing with ssh timeout to cirros13:18
chandankumarpanda|ruck: you mean here http://zuul.openstack.org/builds?job_name=tripleo-ci-centos-7-standalone ? few have issues13:23
chandankumarkopecmartin: Hello13:24
chandankumarkopecmartin: https://bugzilla.redhat.com/show_bug.cgi?id=171005213:24
openstackbugzilla.redhat.com bug 1710052 in openstack-nova "[RHOS 15] Tempest test failure: test_list_extensions" [Unspecified,Modified] - Assigned to alifshit13:24
chandankumarkopecmartin: check last comment13:24
*** dmellado has quit IRC13:25
*** dmellado has joined #oooq13:25
kopecmartinchandankumar, so create a bug on storyboard and we'll remove it from the code13:26
chandankumarkopecmartin: will do it13:27
chandankumarzbr: need some help here https://review.opendev.org/#/c/661979/1/vars/redhat.yml@30 for ternary operator13:28
sshnaidmrlandy, panda|ruck please review: https://review.opendev.org/#/c/661505/213:28
zbrchandankumar: why not using jinja2 conditionals instead? they are much easier to read than ternary construct.13:29
chandankumarzbr: yes, that is doable, but lot of if repeative13:31
zbrthat code seems really hard to read, is clearly incompatible with fedora. i wonder if we cannot avoid it and build/load the vars at runtime.13:32
*** dtantsur|lunch is now known as dtantsur13:35
zbrchandankumar: give me 10mins and i will show you how to do it. we already used a better approach in other places.13:35
zbrchandankumar: i am working on it, will be ready soon13:45
chandankumarzbr: cool!13:54
*** dsneddon has joined #oooq13:56
*** rf0lc0 has joined #oooq13:58
chandankumarkopecmartin: https://storyboard.openstack.org/#!/story/200579413:59
kopecmartinchandankumar, great, thanks13:59
chandankumarkopecmartin: once this merge, we can cut a new release?14:00
kopecmartinsure14:00
*** rfolco has quit IRC14:00
*** dsneddon has quit IRC14:02
chandankumarzbr: honey badger call time14:02
zbrchandankumar: tx for the reminder, i was about to miss it.14:03
chandankumarzbr: :-)14:04
chandankumarpanda|ruck: https://trello.com/c/28FhkqpT/987-cixossphase1osp15-floating-ip-seems-unreachable-phase1-fails14:06
chandankumarpanda|ruck: we might need to switch to new cirros imahe14:06
chandankumar4.014:06
*** dsneddon has joined #oooq14:06
chandankumarssh timedout issue on phase 1 jobs coming so14:07
*** ykarel|afk has quit IRC14:07
*** dsneddon has quit IRC14:12
chandankumarrlandy: rf0lc0 https://code.engineering.redhat.com/gerrit/#/c/171227/ please have a look when free!14:16
*** rf0lc0 is now known as rfolco14:16
rlandychandankumar: wrt ^^ ... looking at the test job, the results are failures14:19
chandankumarrlandy: extra repo issue is gone14:19
chandankumarrlandy: for python3 fixes, I am working here https://review.opendev.org/#/c/661979/14:20
chandankumaronce this one is done, we will be able to consume os_tempest in standalone14:20
rlandychandankumar:s orry - just to be clear  ... 2019-05-23 08:31:32.145012 | primary | An exception occurred during task execution. To see the full traceback, use -vvv. The error was: dnf.exceptions.MarkingError: no package matched14:21
rlandy^^ error in the test job14:21
rlandythat is the python error you mentioned?14:22
chandankumarrlandy: https://code.engineering.redhat.com/gerrit/#/c/171227/1/config/release/rhos-15.yml14:22
panda|ruckchandankumar: yep14:22
chandankumarrlandy: it is about extra repos14:22
panda|ruckchandankumar: some gate is failing on this14:22
chandankumarrlandy: /me can wait also till python3 os_temepst patch merges14:24
rlandychandankumar: no - I am just trying to check that standalone still deployed ok14:24
rlandy2019-05-23 08:06:35.947121 | primary |14:25
rlandy2019-05-23 08:06:35.947158 | primary | TASK [standalone : Deploy the standalone] **************************************14:25
rlandy2019-05-23 08:06:35.947168 | primary | Thursday 23 May 2019  08:06:35 -0400 (0:00:02.332)       0:09:03.890 **********14:25
rlandy2019-05-23 08:31:11.914726 | primary | changed: [undercloud]14:25
rlandy2019-05-23 08:31:11.947552 | primary |14:25
rlandychandankumar: ^^ ok - so I think that's alright then14:25
chandankumarpanda|ruck: you mean fs28 and centos-7 standalone ssh timedout error14:25
rlandywill merge14:25
chandankumarrlandy: great, thansk :-)14:25
panda|ruckchandankumar: yes14:28
rfolcorlandy, how did you unstuck zuul yesterday? no jobs running again14:28
zbrchandankumar: can you please explain me how am I supposed to run tox on openstack-ansible-os_tempest repo?14:30
rlandyrfolco: yep14:30
rlandywe were short of nodes14:30
rlandyrfolco: jobs are running now14:30
rfolcorlandy, https://sf.hosted.upshift.rdu2.redhat.com/zuul/t/local/status14:31
zbrit references files that are not inside repository: like https://github.com/openstack/openstack-ansible-os_tempest/blob/master/tox.ini#L10914:31
rfolcorlandy, not even tox-linters :(14:31
rlandyrfolco: https://sf.hosted.upshift.rdu2.redhat.com/zuul/t/tripleo-ci-internal/status14:31
rlandy^^ you need to specify the tenant14:31
rfolcorlandy, :-O14:32
rfolcorlandy, cool my job is running thanks :)14:32
chandankumarzbr: we just run tox for running linters apart from that all other jobs we depends on osa14:34
chandankumarzbr: you can find a script named https://opendev.org/openstack/openstack-ansible-os_tempest/src/branch/master/run_tests.sh14:34
chandankumarfor running the same14:34
*** ykarel|afk has joined #oooq14:35
*** ykarel|afk is now known as ykarel14:35
chandankumarzbr: https://docs.openstack.org/openstack-ansible/latest/contributor/testing.html14:36
rlandychandankumar: to get back to the logs upload from tripleo-ci-internal tenant ... I am modifying the job to add an option whether to log to rdo public or not ...14:36
rlandywhich jobs do you need to log to rdo?14:37
chandankumarrlandy: standalone rhos-1514:37
chandankumarso that I can link back in os_tempest review14:37
rlandychandankumar: ok - of I add the option, it will default to internal logging14:37
rlandychandankumar: k - will put that change up for your review14:38
chandankumarrlandy: sure, great :-)14:38
zbrchandankumar: https://media.giphy.com/media/mBjulVQHWumozyk6O2/source.gif --- that's how I feel when I discover that someone decides to reinvent the wheel and make it square ;)14:38
chandankumarrlandy: panda|ruck sshnaidm https://review.rdoproject.org/r/#/c/20919/ last review for getting 2ctlr in prmotion piepline14:38
chandankumarzbr: hahah14:39
zbrchandankumar: and obviously that if fails to run big style becase it relies on readlink -f which is does not exist on bsd14:40
sshnaidmchandankumar, better ruck and rover will merge it ^^ <- marios panda|ruck14:40
panda|ruckchandankumar: wait, so I saw the 2comp REPLACING most of the jobs in the pipeline configuration14:40
panda|ruckchandankumar: here you are adding it to the criteria14:41
chandankumarpanda|ruck: yes,14:45
chandankumarpanda|ruck: do we want to run somewhere else for few days?14:45
*** saneax has quit IRC14:46
panda|ruckchandankumar: no I mean, if the old job is replaced, that it should be removed from the criteria14:46
chandankumarpanda|ruck: ok, do we want to add like this, add 2 ctlr job first then remove 1cltr job?14:48
marioschandankumar: sshnaidm panda|ruck isn't it a bit premature i mean e.g. https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-7-ovb-1ctlr_2comp-featureset020-master14:50
mariospremature to put into criteria i mean14:50
mariosof course i used my bash script to get that information !14:50
marios;)14:50
marioschandankumar: panda|ruck sshnaidm seriously why are we in a rush to put that into the promotion criteria14:51
sshnaidmmarios, yeah, I'd wait a few successful times14:51
mariosi agree with panda let it run first get some green builds and then ...14:51
chandankumarmarios: agree, let's hold this patch for a week14:52
mariossshnaidm: fwiw i added the json query with jq (thanks panda|ruck ) in https://github.com/marios/tripleo_ruck_job_tool/blob/b53a448d181fc104fee85ad286c3879be4bb8e60/ooo-job-tool.sh#L5314:52
chandankumarwe should have the run from periodic pipeline14:52
mariossshnaidm: panda|ruck thanks !14:52
panda|ruckyes, let the next ruck and rover deal with this new job :D14:53
sshnaidmmarios, 👍14:53
*** Vorrtex has joined #oooq14:53
chandankumarok, time to leave now, see ya tomorrow14:53
*** chandankumar is now known as raukadah14:54
ykarelmarios, chandankumar, so blocking promotion for 1 week?14:54
ykarelraukadah, ^^14:54
raukadahykarel: we are not blocking14:54
mariosykarel: the proposal is to ADD to criteria14:54
ykarelraukadah, but with current promotion criteria it will be blocked14:55
mariosykarel: but anyway it doesn't have to be a week14:55
mariosjust saying14:55
raukadahfs020 with 1 compute and 2 compute14:55
mariosright now thtere is nothing there from periodic runs https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-7-ovb-1ctlr_2comp-featureset020-master for example14:55
zbr... chandan left before me asking who was behind the design of run_tests.sh approach....14:55
ykarelraukadah, marios me seeing https://review.rdoproject.org/r/#/c/20898/ , is there some other patch also to adjust promotion criteria14:55
raukadahzbr: check with odyssey4me and cloudnull14:55
marioshttps://review.rdoproject.org/r/#/c/20919/ ykarel14:55
* ykarel looks14:56
ykarelmarios, yes i meant ^^ only14:56
zbrraukadah: thanks, I will make a note and I will delay contacting them because i have very high negative load.... i need to cooldown first.14:56
ykarelmarios, raukadah for example periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset020-master job is no longer running and it exist in promotion criteria14:57
ykarelso there will be no promotion with ^^ state14:57
raukadahykarel: oh, yes14:57
ykareland i see Holding for a week to see the performance of the jobs then run. comment14:57
mariosykarel: ack then we'll need another patch to remove those in the meantime if we need promotions14:57
mariosykarel: but14:57
mariosykarel: maybe couple days is OK14:57
mariosykarel: if we're not red14:58
* marios checks14:58
raukadahykarel: let me put a patch14:58
marioshttp://rhos-release.virt.bos.redhat.com:3030/rhosp doesn't look too bad14:58
zbrbtw, is anyone using podman as non-root user under centos-7? asking because I ended up with lots of issues, even not able to pull an image.14:58
ykarelmarios, raukadah i would say good to add new job to promotion criteria, it's good to promote periodically14:58
mariospanda|ruck: wdyt? raukadah no rush can be tomorrow ... and maybe we don't need a new patch ... we 'block' for couple days and then merge raukadah patch at https://review.rdoproject.org/r/#/c/20919/14:59
ykarelmarios, raukadah i assume those jobs are tested already14:59
mariosykarel: yeah we all want to add them but they are just too new14:59
raukadahykarel: jobs are tested already14:59
mariosykarel: there are no runs of them in periodics yet literally 0 @ https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-7-ovb-1ctlr_2comp-featureset020-master14:59
mariosraukadah: ack i don't dispute that but i've been caught by things running fine in check/in the review then breaking in periodic14:59
marioscos of some variable/somethign we're doing differently etc14:59
ykarelmarios, raukadah okk, u can wait till tomorrow, but blocking for long is not good i think15:00
ykarelby tomorrow there will be few runs15:00
ykareland atlesast 1 run in older runs15:00
mariosykarel: ack15:00
* marios shutdown sequence 15:13
*** dsneddon has joined #oooq15:14
weshayrlandy zbr fyi https://code.engineering.redhat.com/gerrit/#/c/171845/15:15
rlandyweshay: what am I look at with ^^?15:16
weshayrlandy yet another type of osp-15 build..  we may not need that if / when we get container update to work.  However it may help our guys debug phase115:17
weshayso atm.. 3 osp-15 standalone jobs15:18
zbrweshay: i do not understand it.... bit confused by it.15:18
weshayneed_images, latest, passed_phase_115:18
*** dsneddon has quit IRC15:20
rlandyweshay: ok - so here we go ... if I remove the port number, rhos-15 passes15:21
rlandyrhos-14 fails15:21
rlandythis is so much fun15:21
*** dsneddon has joined #oooq15:22
rlandyhttps://sf.hosted.upshift.rdu2.redhat.com/logs/19/170519/11/check/tripleo-ci-rhel-8-standalone-rhos-15/a317265/logs/undercloud/etc/containers/registries.conf.txt.gz15:22
rlandy^^ you can see the change in the container registry15:22
rlandyhttps://sf.hosted.upshift.rdu2.redhat.com/logs/19/170519/11/check/tripleo-ci-rhel-8-standalone-rhos-15/a317265/logs/undercloud/var/log/tripleo-container-image-prepare.log.txt.gz15:23
rlandy^^ that indeed works15:23
weshayoh.. /me looks15:24
weshayzbr what questions do you have?15:24
weshayit's just a different build of osp15:24
zbri added comment there15:24
*** quiquell has quit IRC15:24
rlandyweshay: so rhos-14 and rhps-15 are definitely diff15:25
rlandyweshay: but stevebaker was right about rhos-1515:25
weshayk cool15:25
rlandyweshay: I think I will need to switch case the replace15:25
weshayzbr regarding your comments.. yes we should trigger.. etc.. but the need is right now, so i wanted to see something immediately15:26
weshayrlandy nice.. so modify works15:29
rlandyweshay: sec - changing review to only switch insecure registry on stein and highre15:30
rlandyhigher15:30
rlandylet's see if we can get them all to pass15:30
* rlandy doesn't understand why15:30
rlandythen I'll try needs_images15:31
weshaywhy that works in one version and not the other15:31
weshayrlandy I think Steve made reference the trace was directly from podman15:31
rlandyweshay: ack15:32
rlandydocker must be different15:32
weshayso osp-15 and stein would be podman15:32
weshayrlandy ya15:32
rlandybut then how is stein passing (upstream)???15:32
* rlandy is a little confused15:32
*** marios has quit IRC15:33
weshayrlandy ya..agree15:34
* weshay looks15:34
*** dsneddon has quit IRC15:34
rlandyweshay: also I am editing the 'replace' tasks not the merge tasks in the origina l file15:34
*** altlogbot_3 has quit IRC15:35
*** altlogbot_1 has joined #oooq15:35
*** irclogbot_1 has quit IRC15:35
*** irclogbot_3 has joined #oooq15:36
zbrweshay: or rlandy can you login to bugzilla as associate? i get a perl error.15:39
rlandyzbr: ack ... do this15:40
rlandyhttps://bugzilla.redhat.com/15:40
rlandygo to main page15:40
rlandylog in as associate15:40
rlandyit doesn't work for me to log in from a bug15:40
rlandyonce you are logged in15:41
rlandyyou can access the internal bug info15:41
*** tesseract has quit IRC15:44
rlandyman, I am totally guessing here15:45
rlandyzbr: work for you??15:45
zbrrlandy: thanks. worked. we need to follow-up this issue. already sent an email.15:47
*** dtantsur is now known as dtantsur|afk15:51
*** jfrancoa has quit IRC15:52
*** ykarel is now known as ykarel|away15:59
*** bogdando has quit IRC16:18
*** dsneddon has joined #oooq16:23
rfolcorlandy, I'm getting error with the encrypted secret for rdo_registry.... who did create it on rdo config ?16:25
rfolco"Logging into trunk.registry.rdoproject.org for user unused failed - 401 Client Error: Unauthorized (\"Get https://trunk.registry.rdoproject.org/v2/: unauthorized: authentication required\")"16:25
rfolcooh jpena did16:26
rfolcojpena, need help on the rdo_registry secret16:26
rfolcohttps://github.com/rdo-infra/review.rdoproject.org-config/commit/a99425996a8d190c9251b6813c988efade6da63c16:26
jpenarfolco: isn't it working?16:27
rfolcojpena, no, I paste the password into a file and then ran the command to generated the encrypted secret... wrong password?16:28
rfolcojpena, the one in sf is not working, not in rdo config16:29
jpenarfolco: ah ok16:29
jpenaunderstood16:29
jpenaso you did the zuul_encrypt thing and it did't work, right?16:29
rfolcojpena, yes, I got the password in the promoter server16:30
rfolcojpena, pvt16:30
jpenarfolco: I think I've seen that, let me remember16:30
*** dsneddon has quit IRC16:31
rfolcojpena, differently from uploader_ssh_key, which I paste the entire private key... for rdo_registry I paste only password. Not sur eif this is ok.16:31
jpenarfolco: the password is a very long one, right?16:32
jpenathat's actually a token, but serves as a password16:32
rfolcojpena, yes, correct16:32
*** ykarel|away has quit IRC16:35
*** holser_ is now known as holser|eod16:47
*** derekh has quit IRC16:53
*** dsneddon has joined #oooq16:59
rfolcorlandy, secret did not work, suspecting EOF char.... attempt to fix: https://code.engineering.redhat.com/gerrit/171865 - would you merge it please ?17:00
*** aakarsh has quit IRC17:02
rfolcoor jpena if you are around ^ :)17:03
rlandyrfolco: k - done17:04
rfolcorlandy, thanks17:04
*** holser|eod has quit IRC17:06
*** dsneddon has quit IRC17:08
*** jpena is now known as jpena|off17:15
*** dsneddon has joined #oooq17:26
*** sshnaidm is now known as sshnaidm|off17:33
rlandysshnaidm|off: how big is your bmc node?17:37
rlandysshnaidm|off: the bmc node may be too small17:38
rlandychecking rdocloud17:39
*** kopecmartin is now known as kopecmartin|off17:47
*** aakarsh has joined #oooq17:48
*** tosky__ has joined #oooq17:59
*** tosky has quit IRC18:00
*** tosky__ is now known as tosky18:00
*** amoralej is now known as amoralej|off18:17
*** panda|ruck has quit IRC18:35
*** panda has joined #oooq18:37
*** Vorrtex has quit IRC19:34
*** rlandy is now known as rlandy|brb19:47
*** vinaykns has joined #oooq19:51
*** rlandy|brb is now known as rlandy20:15
*** dsneddon has quit IRC20:35
rlandyweshay: ok - Running buildah pull 192.168.24.1:8787/rhosp15/openstack-swift-proxy-server:20190523.120:42
rlandymodified_append_tag: -updated-2019052915041820:43
rlandynot used20:43
rlandy2019-05-29 19:23:38 | TASK [tripleo-container-tag : Pull 192.168.24.1:8787/rhosp15/openstack-cinder-volume:20190523.1-updated-20190529150418 image] ***20:45
rlandy2019-05-29 19:23:39 | changed: [standalone]20:45
rlandy^^ that works20:47
*** jtomasek has quit IRC20:55
sshnaidm|offrlandy, yeah, it seems smaller, will increase it now21:27
rlandysshnaidm|off: already did21:27
rlandysee the latest job21:27
rlandyadded that setting to the latest run21:27
rlandywe still see an introspection timeout21:27
rlandywaiting for latest logs to see if it's the same errro21:28
rlandyerror21:28
sshnaidm|offrlandy, maybe now to try the original ipxe image21:28
rlandysshnaidm|off: sure go4it - but we may get the original error back21:28
rlandywe can try though21:29
rlandyaccording to kforde, the upgrades guys have their jobs working now21:29
sshnaidm|offrlandy, ovb jobs on upshift?21:29
rlandykforde-> rlandy, sshnaidm any progress on the ipxe issue in upshift?21:29
rlandy<kforde-> AFAIK the upgrades guys had it working today21:29
rlandyno idea what the upgrades guys have going21:30
rlandyweshay rhos-15 needs images is passing now21:31
sshnaidm|offok, trying now with original image..21:32
rlandysshnaidm|off: so here's the error from the last run: workflow failed due to message status. Status:FAILED21:35
rlandydifferent from before21:35
rlandyttps://sf.hosted.upshift.rdu2.redhat.com/logs/68/171268/12/check/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-internal/0704815/logs/undercloud/var/log/containers/ironic/ironic-conductor.log.txt.gz#_2019-05-29_19_40_56_48621:36
rlandy^^ same errors here though21:36
sshnaidm|offyeah, timeouts..21:37
*** vinaykns has quit IRC21:41
*** udesale has quit IRC21:43
rlandysshnaidm|off: seen this error: https://sf.hosted.upshift.rdu2.redhat.com/logs/68/171268/12/check/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-internal/0704815/logs/undercloud/var/log/extra/errors.txt.txt.gz#_2019-05-29_19_37_04_644 and this bug: https://bugzilla.redhat.com/show_bug.cgi?id=170088321:45
openstackbugzilla.redhat.com bug 1700883 in podman "Impossible to recreate a container with the same name that a container already removed" [Urgent,Assigned] - Assigned to mheon21:45
*** tosky has quit IRC21:49
*** aakarsh|2 has joined #oooq22:04
*** aakarsh has quit IRC22:04
*** Goneri has quit IRC22:18
*** rlandy is now known as rlandy|bbl22:22
*** dsneddon has joined #oooq22:31
*** dsneddon has quit IRC22:35
*** ccamacho has quit IRC23:58

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!