Friday, 2019-06-14

*** dtrainor_ has joined #oooq00:42
*** dtrainor has quit IRC00:43
*** dsneddon has quit IRC01:11
*** dsneddon has joined #oooq01:25
*** apetrich has quit IRC01:57
*** Goneri has quit IRC02:14
rlandy|ruckugh - rax failing out gates again02:18
*** rlandy|ruck has quit IRC02:24
*** Goneri has joined #oooq02:25
*** ykarel has joined #oooq02:44
*** Goneri has quit IRC02:49
*** ykarel has quit IRC02:51
*** dmellado_ has joined #oooq03:08
*** dmellado has quit IRC03:08
*** dmellado_ is now known as dmellado03:08
*** jschlueter has quit IRC03:22
*** jbadiapa has quit IRC03:22
*** hamzy has quit IRC03:22
*** jschlueter has joined #oooq03:37
*** jbadiapa has joined #oooq03:38
*** hamzy has joined #oooq03:38
*** irclogbot_0 has quit IRC03:41
*** irclogbot_3 has joined #oooq03:41
*** ratailor has joined #oooq03:49
*** udesale has joined #oooq03:53
*** ykarel has joined #oooq04:10
*** skramaja has joined #oooq04:25
*** ykarel has quit IRC04:29
*** ykarel has joined #oooq04:48
*** ksambor_ is now known as ksambor04:49
*** tcw has quit IRC05:07
*** jfrancoa has joined #oooq05:15
*** raukadah is now known as chandankumar05:21
*** marios has joined #oooq05:37
*** tcw has joined #oooq05:41
*** jbadiapa has quit IRC05:49
*** marios has quit IRC06:23
*** marios has joined #oooq06:26
*** udesale has quit IRC06:34
*** udesale has joined #oooq06:37
*** kopecmartin|off is now known as kopecmartin06:40
*** tesseract has joined #oooq07:08
*** tosky has joined #oooq07:15
*** jbadiapa has joined #oooq07:27
*** apetrich has joined #oooq07:35
*** holser_ has joined #oooq07:36
*** holser_ has quit IRC07:40
chandankumarsshnaidm|off: Hello07:42
*** amoralej|off is now known as amoralej07:42
chandankumarsshnaidm|off: https://review.opendev.org/#/c/665244/ this patch might fix the ansible 2.7 issue for os_tempest07:43
chandankumarykarel: ^^07:43
*** dtantsur|afk is now known as dtantsur07:44
*** d0ugal has quit IRC07:47
*** d0ugal has joined #oooq07:48
*** jpena|off is now known as jpena07:50
*** ykarel is now known as ykarel|afk07:52
*** ykarel|afk is now known as ykarel|lunch07:53
*** udesale has quit IRC07:56
*** zbr|ooo is now known as zbr|ruck08:01
*** bogdando has joined #oooq08:03
*** sanjayu_ has joined #oooq08:16
*** holser_ has joined #oooq08:21
*** holser__ has joined #oooq08:22
zbr|ruckI just observed some weird yum repos on my c7 machine, quickstart* ones. which seem to duplicate the default OS ones. What is the reasoning behind creating them instead of using the original OS repo names?08:22
*** holser__ has quit IRC08:24
*** holser_ has quit IRC08:25
zbr|ruckanother question is related to delorean repos which contain the checksum inside .repo file, which renders them broken after a number of days. While for a CI build I can understand the need to assure we use special version, for non CI use, I usually prefer a perma-symlink to avoid getting them obsolete.08:27
*** derekh has joined #oooq08:29
*** holser_ has joined #oooq08:29
*** ccamacho has joined #oooq08:30
*** holser_ has quit IRC08:31
*** holser__ has joined #oooq08:31
*** holser__ has quit IRC08:35
*** holser_ has joined #oooq08:37
*** jtomasek has joined #oooq08:38
*** holser_ has quit IRC08:44
*** holser_ has joined #oooq08:53
*** udesale has joined #oooq08:55
*** holser__ has joined #oooq08:57
*** holser_ has quit IRC08:59
*** ykarel|lunch is now known as ykarel08:59
*** holser__ has quit IRC09:01
*** holser_ has joined #oooq09:02
*** holser_ has quit IRC09:04
*** holser_ has joined #oooq09:05
*** holser_ has joined #oooq09:06
chandankumarmarios: Hello09:23
chandankumarmarios: regarding rhos-release changes https://code.engineering.redhat.com/gerrit/#/c/173155/09:23
marioschandankumar: o/09:23
chandankumarin order to use this does the command is rhos-release -p rdotrunk-master or we call it with rdotrunk-train?09:23
chandankumarand the release file name should be master-rhel or train-rhel?09:24
marioschandankumar: well it just overrides the default behaviour... so if you use rdotrunk-whatever and this is rhel8 it'll use the experimental repos yeah09:24
marioschandankumar: don't know re the name but i've been using 'train' in the job names fwiw09:25
chandankumarok let me put the patch then we can take it from there09:25
chandankumarpanda: marios https://code.engineering.redhat.com/gerrit/#/c/173480/09:35
*** holser_ is now known as holser|lunch09:40
chandankumarmarios: one more question https://code.engineering.redhat.com/gerrit/#/c/172857/ adds overcloud image build, its stuff can be consumed in release file https://code.engineering.redhat.com/gerrit/#/c/173480/2/config/release/train-rhel.yml@69 ? Am i correct?09:45
marioschandankumar: do you mean containers? you're poinging to the images review . but what stuff do you have in mind09:54
chandankumarmarios: yes09:55
chandankumarmarios: docker_registry_host build command and other stuff09:55
mariospanda: seen this before? (its not the dns, and i've already done the cert bit from https://code.engineering.redhat.com/gerrit/#/c/173350/1/playbooks/tripleo-buildcontainers/pre-rhos-release-rhel.yaml@18  2019-06-14 09:51:46.822 | > Cannot download 'https://cdn.redhat.com/content/dist/rhel8/8.1/x86_64/appstream/os': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried.09:55
marioschandankumar: yeah they will likely be the same as the containers review but honestly haven't worked on those bits yet still trying to build images :/09:56
pandamarios: where's the full log ?09:56
chandankumarmarios: ok09:56
mariospanda: here thanks http://paste.openstack.org/show/752923/09:58
mariospanda: i already tried a yum clean all && sudo yum clean metadata && sudo yum clean dbcache && sudo yum makecache and that completed fine09:59
pandamarios: never seen this, but it looks like the problem is with the client certificate, and that line you pasted deals only with the server certificate10:03
pandayou install the CA cert, but you don't do anything for the client10:03
mariospanda: ack thanks for looking10:04
pandaI think the server whats to know who you are and if you realy are who you're saying you are10:04
*** sanjayu_ has quit IRC10:11
*** ratailor_ has joined #oooq10:11
*** sanjayu_ has joined #oooq10:12
*** rfolco has quit IRC10:13
*** rfolco has joined #oooq10:14
*** ratailor has quit IRC10:14
*** blueDuckRandomSt has joined #oooq10:16
*** ratailor_ has quit IRC10:16
*** ratailor has joined #oooq10:18
*** blueDuckRandom has quit IRC10:18
zbr|ruckthis has nothign to do with custom CA cert. you need a client-SSL certificate, which acts as credentials for CDN. This is how subscription-manager works, each computer gets its own client-SSL certificate.10:20
zbr|ruckstored in /etc/pki/...10:20
pandathat's what I said.10:21
mariosthanks zbr|ruck panda10:23
*** udesale has quit IRC10:35
*** sanjayu_ has quit IRC10:36
*** udesale has joined #oooq10:38
*** saneax has joined #oooq10:38
zbr|ruckmarios: thanks for the mention regarding stderr/stdout ordering with pytest. I think i found a workaround, not perfect but clearly better than having all stderr lines at the bottom.10:39
*** holser|lunch is now known as holser_10:57
*** jbadiapa has quit IRC10:57
*** jaosorior has joined #oooq11:20
*** panda is now known as panda|lunch11:28
*** jpena is now known as jpena|lunch11:36
*** udesale has quit IRC11:48
weshaymarios are you building overcloud images?11:53
weshayoh no.. that's containers11:53
mariosweshay: well theres two stories/tasks so both11:54
* weshay just looking at that error11:55
mariosweshay: they both share the rhos-release rdotrunk-master issue with https://code.engineering.redhat.com/gerrit/#/c/173155/11:55
mariosweshay: where you looking? latest issues are on the 'manual deploy/build foo' tasks11:56
mariosweshay: https://tree.taiga.io/project/tripleo-ci-board/task/1114 i guess this is where you're looking (also see that 12:59 < marios> panda: here thanks http://paste.openstack.org/show/752923/ )11:57
weshayk.. will look..   marios /me trying to figure out what is requesting  cdn.redhat11:57
weshaymarios  #511:58
weshayhave a log?11:58
mariosweshay: well just what's on the paste there but if you like you can check yourself on the box right now just copy past those build commands11:59
mariosweshay: sec pvt will give you the box info11:59
weshaymarios is that image or container build11:59
weshaysub-man is enabled on that machine or container11:59
mariosweshay: thats image build see command in Reviews on the task12:00
weshaysubsciption-manager unsubscribe; subscription-manager unregister12:00
weshaythat needs to be executed on quickvm's12:01
weshaymarios it's 100% on those vms12:01
weshayour internal tooling should never never hit cdn.redhat.com12:01
chandankumararxcruz: kopecmartin https://etherpad.openstack.org/p/tripleo-tempest-squad-meeting add the agenda below line 2712:02
mariosweshay++12:03
*** jaosorior has quit IRC12:09
chandankumarrfolco: weshay https://code.engineering.redhat.com/gerrit/#/c/173480/ please have a look at this patch12:12
weshayrfolco please block rhel7/8 ovb jobs on internal12:13
weshayrfolco block w/ this card https://trello.com/c/W4gs0iAX/54-upshift-performance-concerns12:13
rfolcoweshay, moving to rdo w/ rhui ?12:13
*** rf0lc0 has joined #oooq12:21
chandankumarmarios: https://sf.hosted.upshift.rdu2.redhat.com/logs/37/173337/5/check/tripleo-ci-rhel-8-standalone-train/e847881/job-output.txt.gz#_2019-06-14_08_16_29_74874412:21
*** rfolco has quit IRC12:21
chandankumarmarios: seen this issue, do we first needs to merge rhos-relese patch?12:21
marioschandankumar: yeah you're hitting the same issue you need that https://code.engineering.redhat.com/gerrit/#/c/17315512:23
chandankumarmarios: it is there, but depends-on does not working12:24
chandankumarmarios: https://code.engineering.redhat.com/gerrit/#/c/173337/12:24
*** derekh has quit IRC12:25
zbr|ruckon Monday i will be out till ~3pm BST, can someone cover me?12:26
*** hamzy has quit IRC12:27
chandankumarmarios: I think we can add it as a depends -on project then install it from then make rpm12:28
chandankumars/depends-on/required-projects12:28
chandankumarin tripleo-ci-internal jobs repo12:29
chandankumar?12:29
rf0lc0weshay, ok, only ovb jobs. Got it.12:30
chandankumarbrb12:30
zbr|ruck>1.5yr old typo https://review.opendev.org/#/c/542057/ --- this makes me think we may not do a very good job reviewing code.12:30
*** jpena|lunch is now known as jpena12:37
*** rlandy has joined #oooq12:38
*** rlandy is now known as rlandy|ruck12:38
rlandy|ruckyay master promotion12:40
rlandy|ruckykarel++12:40
chandankumarsshnaidm|off: will I edit this dnm patch https://review.opendev.org/#/c/664537 ?12:47
zbr|ruckweshay: i think we may want to store an ansible version constraint in a shared place, similar to openstack requirements. what do you think?12:48
rlandy|ruckhmmm ... got  a legit issue on queens fs03512:51
*** skramaja has quit IRC12:52
*** panda|lunch is now known as panda12:55
weshayrlandy|ruck fyi https://trello.com/c/W4gs0iAX/54-upshift-performance-concerns12:58
*** derekh has joined #oooq12:58
weshayrlandy|ruck blocking ovb jobs on upshift for now12:58
ykarelrlandy|ruck, i reran fs035 queens again with https://review.rdoproject.org/r/#/c/21064/ 1.5 hours back, it should report soon, if it pass queens will promote, i reran because it didn't seemed a real issue, tempest test failed as metadata responded in 13 seconds, while cirros timesout in 10 seconds12:59
ykarelit happens from time to time12:59
chandankumarweshay: meeting time13:01
*** hamzy has joined #oooq13:02
rlandy|ruckweshay: ack - understand13:04
rlandy|ruckykarel: that error happened twice - yesterday and today13:05
rlandy|ruckthere were some patches to queens to lately13:05
rlandy|ruckso it is possible there is an issue13:05
ykarelrlandy|ruck, ack but i checked yesterdays run and it passed, let me check again13:05
chandankumarweshay: arxcruz kopecmartin https://etherpad.openstack.org/p/tripleo-tempest-squad-meeting13:05
rlandy|ruckopenstack/tripleo-cimasteropenstack-periodic-24hrmaster71082019-06-14T05:11:09FAILURE13:06
rlandy|ruckperiodic-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-queensopenstack/tripleo-cimasteropenstack-periodic-24hrmaster77322019-06-13T05:02:24FAILURE13:06
rlandy|ruckykarel: ^^13:06
rlandy|ruckykarel: from yesterday's chat conversation ...13:07
rlandy|ruck<rlandy|ruck> weshay: hi - you worked with ssl changes to queens?13:07
rlandy|ruck<weshay> rlandy|ruck maybe? not sure13:07
rlandy|ruck<weshay> the tls work'13:07
rlandy|ruckwe have auth issues13:07
rlandy|ruck2019-06-14 07:01:16 |     2019-06-14 07:00:17,239 84939 WARNING  [tempest.lib.common.ssh] Failed to establish authenticated ssh connection to cirros@10.0.0.100 (Authentication failed.). Number attempts: 23. Retry after 24 seconds.13:09
ykarelrlandy|ruck, yes right it failed yesteray, looks like when i checked https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-queens, todays promotion pipeline was running13:10
ykarelrlandy|ruck, so if u check the logs you will find failed to get http://169.254.169.254/2009-04-04/meta-data/public-keys13:11
ykarel^^ is the issue, public keys didn't get injected into test vm13:11
rlandy|ruckykarel: your job is still going though13:12
rlandy|ruck2019-06-14 13:09:46.187987 | primary | TASK [validate-tempest : Execute tempest] **************************************13:12
rlandy|ruck2019-06-14 13:09:46.208384 | primary | Friday 14 June 2019  13:09:46 +0000 (0:00:00.063)       0:00:58.819 ***********13:12
*** hamzy has quit IRC13:13
rlandy|ruckhow is your job not failing then?13:13
ykarelrlandy|ruck, so for both todays and yesterdays run metadata returned after 13 seconds, the current run will fail if it's a real issue13:15
ykarelconsidering the timing it may be a real issue, let's see it will report in few minutes13:15
*** hamzy has joined #oooq13:17
*** amoralej is now known as amoralej|lunch13:17
ykarelrlandy|ruck, it passed tempest in current run: echo 'Playbook run of baremetal-full-overcloud-validate.yml passed successfully'13:18
*** ykarel is now known as ykarel|afk13:19
*** hamzy has quit IRC13:19
*** hamzy has joined #oooq13:20
rlandy|ruckykarel|afk: yeah - I see the pass13:24
rlandy|ruckweird that we got two of the same failures13:24
rlandy|ruckdid you run with tripleo-ci-testing???13:25
rlandy|ruckyes - with depends on13:26
rlandy|ruckok then13:26
rlandy|ruckqueens will promote13:26
*** tcw has quit IRC13:34
*** rf0lc0 has quit IRC13:35
rlandy|ruckweshay: just because you like pictures of fields of green ... http://rhos-release.virt.bos.redhat.com:3030/rhosp13:37
*** rfolco has joined #oooq13:42
*** rfolco has quit IRC13:46
*** rfolco has joined #oooq13:46
*** jbadiapa has joined #oooq13:50
chandankumarHappy Weekend, See ya on monday :-)13:59
*** chandankumar is now known as raukadah13:59
mariosraukadah: u 214:00
*** saneax has quit IRC14:06
*** rfolco has quit IRC14:20
*** rfolco has joined #oooq14:21
weshaymarios sent14:21
mariosweshay: thank you14:21
*** Vorrtex has joined #oooq14:22
rfolcoweshay, sent you an email. Sorry for the inconvenience.14:25
*** rfolco is now known as rfolco|off14:25
*** ykarel|afk is now known as ykarel|away14:27
*** amoralej|lunch is now known as amoralej14:28
*** ykarel_ has joined #oooq14:30
*** ykarel_ has quit IRC14:30
*** ykarel|away has quit IRC14:32
rlandy|ruckweshay: wrt non-voting and failing tests ... the next two on the list to consider are scenario010 (octavia) and scenario012. Want to leave those where they are for the moment?14:36
*** hamzy has quit IRC14:36
*** hamzy has joined #oooq14:37
weshayrlandy|ruck so .. geez14:37
weshaysend an email to rh-openstack-dev..   the networking and ironic teams need to know whats up14:38
rlandy|ruckk14:38
weshayrlandy|ruck essentially say..  please join the tripleo-ci community call to help push these jobs to passing/gating status or we will be moving them to experimental14:38
weshaydue to upstream resource constraints14:39
weshayrlandy|ruck put a draft in docs for review before sending14:39
rlandy|ruckweshay: on second thoughts standalone 010 is pretty good14:39
rlandy|ruckso really 1214:39
*** ratailor has quit IRC14:50
*** hamzy has quit IRC15:01
weshayarxcruz 1-1?15:01
arxcruzweshay i'm already in your bj15:01
*** hamzy has joined #oooq15:02
*** ratailor has joined #oooq15:04
*** aakarsh has joined #oooq15:05
*** ratailor has quit IRC15:12
*** rfolco has joined #oooq15:24
*** rfolco|off has quit IRC15:25
* marios almost out15:30
*** Goneri has joined #oooq15:35
*** marios has quit IRC15:41
*** ykarel has joined #oooq15:43
*** ykarel is now known as ykarel|away15:52
*** Vorrtex has quit IRC15:53
*** holser_ has quit IRC15:54
weshayrlandy|ruck I have an external mtg.. will be out for a bit15:55
weshayrlandy|ruck need anything before I roll?15:55
*** holser_ has joined #oooq15:55
rlandy|ruckweshay: we're good atm15:55
weshayk k..15:55
weshaythanks15:55
*** holser__ has joined #oooq15:58
*** holser_ has quit IRC16:00
*** kopecmartin is now known as kopecmartin|off16:06
*** holser__ has quit IRC16:14
*** bogdando has quit IRC16:22
*** dtantsur is now known as dtantsur|afk16:26
*** jfrancoa has quit IRC16:31
*** derekh has quit IRC16:37
*** tcw has joined #oooq16:48
*** jpena is now known as jpena|off16:57
*** hamzy has quit IRC17:23
*** hamzy has joined #oooq17:24
*** amoralej is now known as amoralej|off17:32
*** hamzy has quit IRC17:47
*** ykarel|away has quit IRC17:54
*** hamzy has joined #oooq17:56
*** jaosorior has joined #oooq18:16
*** jaosorior has quit IRC18:18
*** tesseract has quit IRC18:18
*** jdennis has quit IRC18:38
*** hamzy has quit IRC19:05
*** jbadiapa has quit IRC19:40
weshayrlandy|ruck need anything?19:43
*** hamzy has joined #oooq19:45
rlandy|ruckweshay: no - seems ok19:56
rlandy|ruckgate is backed up - but it's not our queue19:57
rlandy|ruckstarted the rocky perf testing19:57
rlandy|ruckmaster might promote again19:57
*** dsneddon has quit IRC20:23
*** holser_ has joined #oooq20:27
*** holser_ has quit IRC20:31
rlandy|ruck2019-06-14 19:43:29,972 5951 INFO     promoter SUCCESS promoting tripleo-ci-testing as current-tripleo f28 also promoted20:33
*** holser_ has joined #oooq20:39
*** aakarsh has quit IRC21:10
*** zbr|ruck has quit IRC21:13
*** chem has quit IRC21:15
*** zbr has joined #oooq21:30
*** aakarsh has joined #oooq21:42
*** holser_ has quit IRC21:55
*** rlandy|ruck has quit IRC22:09
*** rfolco has quit IRC22:44
*** tosky has quit IRC22:48
*** dsneddon has joined #oooq23:08
*** jtomasek has quit IRC23:19
*** dsneddon has quit IRC23:19
*** sabedevops has joined #oooq23:24
*** sabedevops has quit IRC23:44

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!