Friday, 2018-06-01

hubbotFAILING CHECK JOBS on master: tripleo-quickstart-extras-gate-newton-delorean-full-minimal @ https://review.openstack.org/56044500:25
*** links has joined #oooq00:44
*** d0ugal__ has quit IRC00:58
*** d0ugal__ has joined #oooq01:00
EmilienMrlandy|rover|bbl: ack, thanks!01:24
EmilienMrlandy|rover|bbl: ok this time it reproduced my error, so perfect. Thanks again01:25
*** dougbtv__ has joined #oooq02:15
*** dougbtv_ has quit IRC02:15
hubbotFAILING CHECK JOBS on master: tripleo-quickstart-extras-gate-newton-delorean-full-minimal @ https://review.openstack.org/56044502:25
myoungrlandy|rover|bbl: pike finished promotion without issue02:26
myoungrlandy|rover|bbl, arxcruz|ruck, now that's a pretty picture: https://photos.app.goo.gl/RjOCGMa0ZmXP0VVK202:29
myoung\o/02:29
*** rlandy|rover|bbl is now known as rlandy|rover03:31
rlandy|roverEmilienM:cool - happy it worked out - I need to get that patch into a reviewable state so we don't hit this again03:32
rlandy|rovermyoung++03:32
hubbotrlandy|rover: myoung's karma is now 103:32
rlandy|roverthank you - looks amazing -ocata03:32
EmilienMrlandy|rover: it allowed me to find this bug https://bugs.launchpad.net/tripleo/+bug/177455703:34
openstackLaunchpad bug 1774557 in tripleo "Ansible timeout with containerized undercloud + IPv6 overcloud" [High,In progress] - Assigned to Emilien Macchi (emilienm)03:34
rlandy|roverEmilienM: do you want to hold that env - or should I nuke it?03:34
rlandy|roverno rush atm if you'd like to keep it03:35
*** dougbtv__ has quit IRC03:44
*** jaganathan has quit IRC03:44
*** jjoyce has quit IRC03:44
*** Tengu has quit IRC03:44
*** tbarron has quit IRC03:44
*** ajo has quit IRC03:44
*** EmilienM has quit IRC03:44
*** EmilienM has joined #oooq03:45
*** EmilienM has quit IRC03:46
*** EmilienM has joined #oooq03:46
*** udesale has joined #oooq03:48
*** d0ugal__ has quit IRC03:52
*** ykarel|away has joined #oooq03:55
*** rlandy|rover has quit IRC03:59
*** ykarel|away is now known as ykarel04:11
*** d0ugal__ has joined #oooq04:14
hubbotFAILING CHECK JOBS on master: tripleo-quickstart-extras-gate-newton-delorean-full-minimal @ https://review.openstack.org/56044504:25
*** strattao has quit IRC04:29
*** strattao has joined #oooq04:29
*** pgadiya has joined #oooq04:41
*** pgadiya has quit IRC04:41
*** d0ugal__ has quit IRC04:49
*** d0ugal__ has joined #oooq04:58
*** holser has joined #oooq05:08
*** marios has joined #oooq05:11
*** holser__ has joined #oooq05:17
*** holser has quit IRC05:18
*** holser has joined #oooq05:21
*** holser__ has quit IRC05:22
*** gvrangan has joined #oooq05:54
*** holser has quit IRC05:59
*** ratailor has joined #oooq06:07
*** saneax has joined #oooq06:09
hubbotFAILING CHECK JOBS on master: tripleo-quickstart-extras-gate-newton-delorean-full-minimal @ https://review.openstack.org/56044506:25
*** d0ugal__ has quit IRC06:26
*** quiquell|off is now known as quiquell06:34
*** sshnaidm_pto has joined #oooq06:35
*** d0ugal__ has joined #oooq06:48
*** amoralej|off is now known as amoralej07:09
*** quiquell is now known as quiquell|afk07:09
*** bogdando has joined #oooq07:10
*** ccamacho has quit IRC07:16
*** ccamacho has joined #oooq07:16
*** florianf has joined #oooq07:22
*** florianf has quit IRC07:22
*** florianf has joined #oooq07:24
*** tesseract has joined #oooq07:25
*** gvrangan has quit IRC07:28
*** dougbtv__ has joined #oooq07:47
*** jaganathan has joined #oooq07:47
*** jjoyce has joined #oooq07:47
*** Tengu has joined #oooq07:47
*** tbarron has joined #oooq07:47
*** ajo has joined #oooq07:47
*** skramaja has joined #oooq07:48
*** ykarel is now known as ykarel|lunch07:53
*** d0ugal__ has quit IRC08:03
*** d0ugal has joined #oooq08:03
hubbotFAILING CHECK JOBS on master: tripleo-quickstart-extras-gate-newton-delorean-full-minimal @ https://review.openstack.org/56044508:25
*** sshnaidm_pto has quit IRC08:29
*** sshnaidm_pto has joined #oooq08:30
*** kopecmartin has joined #oooq08:31
*** jaganathan has quit IRC08:33
*** jaganathan has joined #oooq08:33
*** sshnaidm_pto has quit IRC08:36
*** ykarel|lunch is now known as ykarel08:40
*** quiquell|afk is now known as quiquell08:44
*** sshnaidm_pto has joined #oooq08:51
*** ykarel_ has joined #oooq09:05
*** ykarel has quit IRC09:07
*** sshnaidm_pto has quit IRC09:20
*** udesale_ has joined #oooq09:28
*** udesale has quit IRC09:30
*** udesale_ has quit IRC09:31
*** udesale has joined #oooq09:32
*** d0ugal has quit IRC09:33
ajoinstack_undercloud.validator.FailedValidation: undercloud_nameservers "fe80::1%enp0s31f6" must be a valid IP address09:35
ajohas anybody saw this ^ ?09:35
*** zoli is now known as zoli|lunch09:35
ajoIt happens to me anytime I try to do a quickstart09:35
ajook09:37
ajoI have those nameservers in my virthost09:37
ajoI'm oing to remove09:37
ajoEmilienM: do you have some time today ? I wanted to talk about a spec tripleo has :)09:39
ajoEmilienM: https://blueprints.launchpad.net/tripleo/+spec/all-in-one09:39
ajowe were talking yesterday on the DFG Networking meeting09:39
ajoand we thought it could be very useful to help developers constrain their devel environments09:40
ajoand that it could even be more interesting if we had some sort of capability to... on every release, snapshot an "overcloud-aio" image to qcow2 so....09:40
ajodevelopers could grab that and use it for development09:40
ajowith the help of some additional scripts to setup git / containers, etc...09:40
ajolike a very effective replacement of devstack in many cases09:41
*** panda|off is now known as panda09:51
*** d0ugal has joined #oooq09:53
quiquellpanda: Good morning09:55
pandaquiquell: morning09:59
quiquellpanda: Do you know if any of the cars is already in progress ?09:59
quiquellpanda: I am planning on continue with emit releases script to be able to test n -> n + 1 sooner10:00
quiquellIs easier to hardcore just one pareter10:00
quiquellparameter10:00
*** udesale_ has joined #oooq10:01
*** udesale has quit IRC10:03
pandacars ? hardcore ? too much drumming yesterday dude10:05
pandametal band ?10:05
pandaquiquell: I was discussing with Matt yesterday, I suggested him to take the unit test part of the script10:06
quiquellpanda: Humm... unit testing a implementation is the same, he has to take the implementation too10:06
quiquellpanda: This cannot be splitted10:06
quiquellpanda: Unit testing is iterative, you don't write all the test and the write the code or reverse10:07
quiquellpanda: You write one test that fails, and make it pass, a go on on this.10:07
pandaquiquell: TDD10:07
quiquellTDD10:07
quiquellIf we want to do TDD Matt also do the implementation10:07
pandawhy10:07
quiquellpanda: becouse unit test is part of the code, the ownership of fixing it is from the commiter10:08
quiquellpanda: If you add a new feature to the script you also add tests for it10:08
quiquellthe same guy10:08
quiquellSince it have to be iterative, is a lot of overhead of two epeople involve10:10
quiquellpanda: one option is pairing10:10
quiquellMaybe Matt and Trown can pair on it10:10
quiquellpanda: btw I am going to update my proposal for the interface, they can go from that10:10
pandaPair was my initial idea, and I thought they started some cards, but apparently nothing moved10:13
quiquellpanda: Ok, I will just update the interface proposal, and do a hack on the script to have the n -> n + 1 to test it10:15
quiquellI want to take the injection part to learn a little about the roles10:15
quiquellIs ok ?10:15
*** d0ugal has quit IRC10:16
pandaquiquell: ok for me10:16
quiquellpanda: Cool cool10:16
ykarel_arxcruz|ruck, ocata phase 1 passed, but not promoted yet, is there some issue with promoter?10:17
*** ykarel_ is now known as ykarel10:17
arxcruz|ruckykarel: i'll check in a few, i'm heading for lunch10:18
ykarelarxcruz|ruck, Ok, take ur time10:18
quiquellpanda: https://review.openstack.org/#/c/570779/10:24
*** d0ugal has joined #oooq10:25
hubbotFAILING CHECK JOBS on master: tripleo-quickstart-extras-gate-newton-delorean-full-minimal @ https://review.openstack.org/56044510:25
quiquellpanda: fs050 it at gates now, going to move the card10:28
*** amoralej is now known as amoralej|off10:29
panda\o/10:36
quiquellpanda: To really test fs037 and queens we need a queens change ?10:37
quiquellA tht queens change with a Depends-On on activate queens will run the job ?10:38
arxcruz|ruckquiquell: is the promotion script running how many times per day ?10:50
quiquellarxcruz|ruck: Promotion is done now with the myoung script, don't know how it works10:51
quiquellarxcruz|ruck: old one was running sequencially every 10 minutes, so it depends how each release takes10:52
*** zoli|lunch is now known as zoli10:57
*** d0ugal has quit IRC11:06
*** panda is now known as panda|lunch11:07
*** dougbtv__ is now known as dougbtv11:11
*** udesale__ has joined #oooq11:12
*** udesale_ has quit IRC11:15
*** udesale__ has quit IRC11:18
*** d0ugal has joined #oooq11:23
*** sshnaidm_pto has joined #oooq11:26
*** apetrich has quit IRC11:29
*** apetrich has joined #oooq11:31
arxcruz|ruckmyoung: around ?11:42
*** quiquell is now known as quiquell|lunch11:46
*** dtrainor has quit IRC11:48
*** dtrainor has joined #oooq11:48
*** d0ugal has quit IRC11:53
ajoEmilienM: ping :)11:55
*** rfolco_ has joined #oooq11:56
*** d0ugal has joined #oooq11:59
*** pgadiya has joined #oooq12:03
myoungarxcruz|ruck: yes...quasi, still early here12:05
arxcruz|ruckmyoung: can you check ocata promoter script?12:05
myoungarxcruz|ruck: to be clear, the promoter is running presently as a stop gap on internal server, it's the same script12:06
arxcruz|ruckit was promoted but the logs still say is missing a job, however the job was promoted12:06
myoungarxcruz|ruck: looking, is your key there too?12:06
myoungit is12:12
myoungpromoter is in middle of pike promotion12:12
myoung2018-06-01 07:42:58,084 22580 INFO     promoter Promoting the container images for dlrn hash 351c320c191252c496f6911f972d31390469585b on pike to current-tripleo-rdo12:12
myounglooking at why ocata didn't promote too12:12
*** ykarel has quit IRC12:15
*** ykarel has joined #oooq12:15
*** d0ugal has quit IRC12:16
myoungarxcruz|ruck, ykarel, quiquell|lunch, the promoter does loop around every 10 mins, but an actual promotion (the way the script works) takes a little over an hour to push/pull images from rdo/docker.io12:19
myoungthe ocata job passed a few hours ago, and we've been promoting other branches.  the last ocata promotion attempt was a few hours ago12:19
myoungarxcruz|ruck: I confirmed you have ssh access12:19
arxcruz|ruckmyoung: last good run of ocata was 4 hours ago, and promoter finished 3 hours ago12:20
arxcruz|ruckmyoung: yes, i log there but i didn't want to touch anything without you12:20
arxcruz|rucksince i'm not an expert12:20
myoungalso , I would like to return the promoter to running in rdocloud :)12:20
myoungit's only running on my server because we were stuck last friday night12:20
myounghttps://bugs.launchpad.net/tripleo/+bug/177422012:21
openstackLaunchpad bug 1774220 in tripleo "Promoter server is impacted by networking issues and is offline (running private instance presently)" [Critical,Triaged]12:21
* myoung will be back in a few, it's so early in the morning here he doesn't have contacts in yet12:21
myoungarxcruz|ruck: back...whoo!  that's better12:24
myoungarxcruz|ruck: walking it thru now...can BJ if you like or just do here.  let's see what's up with ocat12:24
*** rfolco_ is now known as rfolco12:25
hubbotFAILING CHECK JOBS on master: tripleo-quickstart-extras-gate-newton-delorean-full-minimal @ https://review.openstack.org/56044512:25
*** ykarel has quit IRC12:25
*** ykarel has joined #oooq12:26
*** pgadiya has quit IRC12:29
myoungarxcruz|ruck: ok confirmed...so last time ocata attempted to promote was about 4 hours ago12:29
myoung2018-06-01 04:41:18,211 19708 INFO     promoter Skipping promotion of current-tripleo to current-tripleo-rdo, missing successful jobs: ['tripleo-quickstart-promote-ocata-rdo_trunk-minimal']12:29
myoungarxcruz|ruck: on the server it's running on now if we dont' want to wait we can kick it by hand in parallel.12:30
myoungthe script running there (which is not different from what's on rdo cloud instance and not running now) serializes the promotions because the instance could not handle the load from network and/or storage perspective.12:31
*** quiquell|lunch is now known as quiquell12:31
quiquellmyoung: Do you have time for some bj ?12:31
myoungarxcruz|ruck: confirmed as well pike is at 80 of 89 containers uploaded to docker.io, it's nearly complete, then ocata should kick in and promote12:33
myoungquiquell: I do...i have some responses to your review of that WIP patch as well12:33
myoungquiquell: can we meet in 10?12:34
quiquellmyoung: sure12:34
*** d0ugal has joined #oooq12:35
*** rlandy has joined #oooq12:36
rlandyarxcruz|ruck: hello12:38
*** ratailor has quit IRC12:38
arxcruz|ruckrlandy: hi12:38
rlandyykarel: a green ocata job - I think you are fast becoming our favorite person!!!12:38
arxcruz|ruckrlandy: was ykarel who fix the problem on that job on ocata?12:39
rlandyarxcruz|ruck: we can actually promote ocata!!!!!12:39
rlandyyep - he found a missing backport12:39
rlandygoing to see what's up with the promoter for ocata12:39
chandankumarrlandy: oooq-undercloud job was failing with few tempest tests in check jobs12:39
rlandychandankumar: ok - do you a specific link/bug - otherwise I'll look in a sec12:40
ykarelrlandy, yeah i like Green :)12:41
rlandyykarel:yep - just need to kick the promoter12:41
rlandy2018-06-01 04:41:18,211 19708 INFO     promoter Skipping promotion of current-tripleo to current-tripleo-rdo, missing successful jobs: ['tripleo-quickstart-promote-ocata-rdo_trunk-minimal']12:41
rlandy2018-06-01 04:41:18,211 19708 INFO     promoter FINISHED promotion process12:41
rlandymaybe e missed it12:41
chandankumarrlandy: https://review.openstack.org/#/c/57008112:42
myoungrlandy: pike is nearly done12:42
myoungrlandy:  (see scrollback above), ocata's about to kick12:43
myoungthat's from 4 hours ago12:43
rlandymyoung: cool - so we'll run ocata again?12:43
rlandynice12:43
arxcruz|ruckman, i wonder what kind of black magic ykarel do to find this stuff12:43
chandankumarrlandy: https://review.openstack.org/57168712:43
rlandyoh happy day!!!12:43
rlandychandankumar: looking12:43
myoungrlandy: yes, pike promotion is just finishing now12:43
myoungrlandy: ocata's on deck12:43
*** trown|outtypewww is now known as trown12:43
rlandychandankumar:looking12:44
myoungrlandy, arxcruz|ruck - we *could* run them all in parallel on the server it's on now...but would like to get promoter working back up on rdocloud preferably and address https://bugs.launchpad.net/tripleo/+bug/177422012:44
openstackLaunchpad bug 1774220 in tripleo "Promoter server is impacted by networking issues and is offline (running private instance presently)" [Critical,Triaged]12:44
arxcruz|ruckmyoung: the problem is how to fix the network issues, according rdocloud everything is working fine12:45
myoungarxcruz|ruck: ack12:45
myoungquiquell: avail now12:45
myoung?12:45
quiquellSUre12:45
rlandychandankumar: hmmm ... http://cistatus.tripleo.org/ doesn't show a lot of failures12:46
rlandybut I see 2018-06-01 02:49:51 |  - Failed: 212:46
myoungquiquell: i'm in bj/matyoung (7050859455)12:47
quiquellYes yes12:47
rlandyI see on eon the gate12:48
quiquellmyoung: I am at your bj12:48
quiquellwait12:48
rlandyok - hasn't reported yet12:49
rlandymyoung: should we merge this? https://code.engineering.redhat.com/gerrit/#/c/140417/12:53
*** rlandy is now known as rlandy|rover12:55
rlandy|roveroops - always forget that nick thing12:56
rlandy|roverchandankumar: does this match what you are seeing: https://bugs.launchpad.net/tripleo/+bug/1773461?13:01
openstackLaunchpad bug 1773461 in tripleo "ocata undercloud job failing on tempest tests" [Critical,Triaged]13:01
chandankumarrlandy|rover: yup13:01
rlandy|roverchandankumar: ok - saves me creating a new bug - the failure look intermittent - is this something the tempest team would take or does the CI team look into it more?13:03
chandankumarrlandy|rover: I am digging into it currently13:03
rlandy|roverchandankumar: ah ok - can I assign it to you? if it becomes a CI thing, pls send it back to me13:03
chandankumarrlandy|rover: sure13:03
rlandy|roverchandankumar: cool - if you need a reproducer or whatever, pls ping and I'll do that13:04
chandankumarrlandy|rover: I have created the reproducer13:04
chandankumari am looking into it13:04
*** links has quit IRC13:37
myoungrlandy|rover, ykarel, arxcruz|ruck: ocata promotion appears to have finished.  no containers so it's fast13:40
myoung2018-06-01 08:46:58,712 19649 INFO     promoter SUCCESS promoting current-tripleo as current-tripleo-rdo ({'timestamp': 1527835528, 'distro_hash': '6b6ecb1c8c901f05db8b5e40a5ac6d6aeb212eff', 'promote_name': 'current-tripleo', 'user': 'ciuser', 'repo_url': 'https://trunk.rdoproject.org/centos7-ocata/99/5f/995fa4b984d2630312ce7417b387f2a980883816_6b6ecb1c', 'full_hash': '995fa4b984d2630312ce7417b387f2a980883816_6b6ecb1c', 'repo_hash': '13:41
myoung995fa4b984d2630312ce7417b387f2a980883816_6b6ecb1c', 'commit_hash': '995fa4b984d2630312ce7417b387f2a980883816'})13:41
myoung2018-06-01 08:46:58,713 19649 INFO     promoter FINISHED promotion process13:41
ykarelmyoung, yeah dashboard shows it Green13:41
rlandy|rovergreat13:43
rlandy|rovermyoung: watching phase 213:43
rlandy|roverchecking requirements there13:43
rlandy|roverwe may need to adjust :(13:43
* rlandy|rover hates doing that13:44
EmilienMajo: sup?13:46
ajohey EmilienM  :)13:46
ajoEmilienM: do you have some time today ? I wanted to talk about a spec tripleo has :)13:47
ajo11:39 EmilienM: https://blueprints.launchpad.net/tripleo/+spec/all-in-one13:47
ajo11:39 we were talking yesterday on the DFG Networking meeting13:47
ajo11:39 and we thought it could be very useful to help developers constrain their devel environments13:47
ajo11:40 and that it could even be more interesting if we had some sort of capability to... on every release, snapshot an "overcloud-aio" image to qcow2 so....13:47
ajo11:40 developers could grab that and use it for development13:47
ajo11:40 with the help of some additional scripts to setup git / containers, etc...13:47
ajo11:40 like a very effective replacement of devstack in many cases13:47
ajocopypasting  ^  :)13:47
ajowe wanted to share those use cases with you13:47
ajowe believe the all-in-one idea is a very good one13:47
ajoEmilienM: and we wanted to know what's the status / priority for "S", we believe that "Medium" is may be not representative of how useful it could be13:48
quiquelltrown: We can abandon https://review.openstack.org/#/c/570779/, just add the args to your pat ch13:51
quiquellIt's more natural13:51
trownquiquell: ok, sure13:51
quiquelltrown: Cool13:52
trownquiquell: I am sure my choice in names will be no better :P13:52
quiquelltrown: Naming is super hard13:52
trownthe hardest problem in computer science13:52
ajoEmilienM: ping ^   ;)13:52
EmilienMajo: give me one sec please, just started my day, I'll reply when I can ;-)13:54
ajoack sorry :D13:54
myoungpanda|lunch:  or trown, could you please join quiquell and I in my room ?13:59
quiquellHave a good weekend14:11
*** quiquell is now known as quiquell|off14:11
hubbotFAILING CHECK JOBS on master: tripleo-quickstart-extras-gate-newton-delorean-full-minimal @ https://review.openstack.org/56044514:26
*** panda|lunch is now known as panda14:28
*** d0ugal has quit IRC14:31
*** d0ugal has joined #oooq14:35
rlandy|roveroh newton - what now??14:37
*** d0ugal has quit IRC14:49
*** d0ugal has joined #oooq14:57
*** marios has quit IRC14:58
*** saneax has quit IRC14:58
*** ykarel is now known as ykarel|away15:00
*** gkadam has joined #oooq15:05
*** ykarel|away has quit IRC15:06
*** zoli is now known as zoli|gone15:12
myoungpanda: thanks for the time for backlog grooming :)15:14
pandarlandy|rover: I think we have some problem with folsom too15:14
*** bogdando has quit IRC15:14
myoungrlandy|rover: sorry for delay, was in BJ15:15
myoungregarding https://code.engineering.redhat.com/gerrit/#/c/140417, yes i think we should merge yes?  now that we're at 7.5 prep internal will either fail, or do bad things no (without it)15:15
rlandy|rovermyoung: k - pls merge - so I don't self merge so much :)15:16
* myoung engages the warp drive (merged)15:17
myoungrlandy|rover: re: changing requiresments in rdo2, what did you mean?15:17
myoungrlandy|rover: promo criteria?15:18
rlandy|rovermyoung: ocata promotion criteria still includes two bm envs15:19
rlandy|rovernot sure how successful we will be there15:19
rlandy|roverwill edit per jobs runs15:19
rlandy|rover30 days out - we need to promote15:19
*** gkadam_ has joined #oooq15:20
rlandy|roverugh - build image failure15:22
rlandy|roverwe should expect that after 30 days15:23
myoungrlandy|rover: link?15:23
myoungrlandy|rover: and ack15:23
rlandy|roverhttps://rhos-dev-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/rdo-ocata-promote-rdo_trunk-build-images/169/15:23
rlandy|roverI'm checking it out15:23
*** gkadam has quit IRC15:24
myoungrlandy|rover: https://thirdparty.logs.rdoproject.org/jenkins-rdo-ocata-promote-rdo_trunk-build-images-169/rdo-ci-fx2-01-s5.v101.rdoci.lab.eng.rdu.redhat.com/home/oooq-images/ooib.log.txt.gz15:29
myoungwe might need to keep ocata image builds on a 7.4 box after all, or see if we can update build inputs15:29
myoungi bet we can do the latter15:29
rlandy|roveryeah - was just looking at that15:29
myoungError: Package: 2:container-selinux-2.55-1.el7.noarch (rhelosp-rhel-7.4-extras)15:30
myoung           Requires: selinux-policy >= 3.13.1-18315:30
myoung           Installed: selinux-policy-3.13.1-166.el7_4.9.noarch (@rhelosp-rhel-7.4-server)15:30
rlandy|rovermyoung: we still have those boxes you had labeled as something else15:30
rlandy|roverwe could make one of those into a rhel 7.4 build15:30
rlandy|roverand assign that machine to ocata15:30
myoungor... (checking input config)15:30
myoungupdate this http://git.app.eng.bos.redhat.com/git/tripleo-environments.git/tree/image-build/ocata/latest15:31
myoungrlandy|rover: specifically this http://git.app.eng.bos.redhat.com/git/tripleo-environments.git/tree/image-build/ocata/latest/ospd-build-image-settings.yml#n515:32
myoung^^ --> 7.5 guest image15:32
rlandy|roverrhn-kvm-guest-74/rhel-server-7.4-x86_64-kvm.qcow215:32
rlandy|roveryep - let's try that15:32
myoungmoving the base guest to 7.5 imho will have fewer issues...because we'll get package bolts in a blender15:32
rlandy|roverI'll propose a patch for your review15:32
rlandy|roversec15:32
myoungneed to d/l new guest image from access.redhat.com --> rhos-release.REDACTED.com/ci-images and update config15:33
myoungcool cool15:33
*** d0ugal has quit IRC15:34
rlandy|rovermyoung: hmmm ... ok no 7.5 images here http://rhos-release.virt.bos.redhat.com/ci-images/15:35
rlandy|roveroh - I see what you are saying15:36
rlandy|rovermyoung: ^^ do you have access to do that ?15:36
myoungI do, as does the normal key15:38
myoungwant me to copy it there or have you already dl?15:38
* myoung opens a tmate and PM's link15:38
EmilienMrlandy|rover: hey, please don't trash the reproducer yet15:41
EmilienMI'm still using it today15:41
*** jaosorior has quit IRC15:41
*** d0ugal has joined #oooq15:41
*** skramaja has quit IRC15:42
*** gkadam__ has joined #oooq15:46
myoungrlandy|rover: upload in progress...15:46
*** d0ugal has quit IRC15:48
*** gkadam_ has quit IRC15:49
* myoung will brb15:50
*** d0ugal has joined #oooq15:51
*** gkadam_ has joined #oooq15:52
rlandy|rovermyoung: nice - thanks15:53
*** gkadam__ has quit IRC15:55
myoungrlandy|rover: should be good to go. http://rhos-release.virt.bos.redhat.com/ci-images/rhn-kvm-guest-75/rhel-server-7.5-update-1-x86_64-kvm.qcow215:59
myoung^^ it the latest, there's a pre-update1 version as well but don't think we should mess with the past16:00
myoungunless we need to pseudo-binary-search w.r.t deps16:00
*** tesseract has quit IRC16:01
*** kopecmartin has quit IRC16:01
myoungrlandy|rover: actually...16:04
myoungrlandy|rover: ocata is the only place today where we even still do RDO on RHEL, we've removed it from the other pipelines16:04
myoungI think we can just stop building RDO on RHEL images at this point, and nuke the jobs in ocata rdo2 that still do it.16:05
myounglike we've done for all the other releases16:05
* myoung preps a patch for that16:05
rlandy|rovermyoung: whatever you think is best - as long as we can run promotion jobs today16:06
myoungrl https://code.engineering.redhat.com/gerrit/140505 Update build config for ocata to use 7.5 , just in case we DO want this16:07
myoungpatch to stop doing this alltogther inc...sec16:07
*** jrist has quit IRC16:08
rlandy|rovermyoung: merged https://code.engineering.redhat.com/gerrit/#/c/140505/16:09
rlandy|rovermyoung: need to run out for a few - brb16:14
*** rlandy|rover is now known as rlandy|rover|brb16:14
myoungrlandy|rover|brb: I think this is the real answer --> https://code.engineering.redhat.com/gerrit/140507 Remove RDO on RHEL build+test jobs for ocata16:16
*** jrist has joined #oooq16:25
hubbotFAILING CHECK JOBS on master: tripleo-quickstart-extras-gate-newton-delorean-full-minimal @ https://review.openstack.org/56044516:26
myoungrlandy|rover|brb: associated housekeeping review to remove RDO on RHEL ref's from promoter criteria: https://review.rdoproject.org/r/14002 promoter: remove references to RDO on RHEL (ocata, pike)16:26
*** gkadam__ has joined #oooq16:33
*** gkadam_ has quit IRC16:36
*** gkadam_ has joined #oooq16:37
*** EmilienM is now known as EvilienM16:40
*** gkadam__ has quit IRC16:40
rlandy|rover|brbmyoung: https://code.engineering.redhat.com/gerrit/#/c/140507 approved - merge whenever yo are ready16:44
*** sshnaidm_pto has quit IRC16:44
myoungrlandy|rover|brb: merging and rekicking ocata then16:46
myoungrlandy|rover|brb: waiting for https://rhos-dev-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/tripleo-environments-jjb-push/81/console to finish first...16:47
rlandy|rover|brbmyoung: awesome16:55
*** rlandy|rover|brb is now known as rlandy|rover16:55
*** gkadam__ has joined #oooq16:57
*** gkadam has joined #oooq17:00
*** gkadam_ has quit IRC17:00
*** gkadam__ has quit IRC17:02
myounghttps://rhos-dev-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/rdo-promote-ocata-rdo_trunk/173 in flight17:04
myoungrlandy|rover: ^^ is past the build/cache phase and into tests17:06
rlandy|rovermyoung: I am still holding onto this review .. https://review.rdoproject.org/r/#/c/14002/17:07
rlandy|roverwe'll probably have to edit the bm entries in there17:07
rlandy|roverif we one bm - let's go with it17:10
*** ykarel|away has joined #oooq17:11
myoungrlandy|rover: ack, that just removes the promote-rhel-{ocata, pike} jobs that no longer exist.  housekeeping ;)17:13
*** trown is now known as trown|lunch17:13
rlandy|rovermyoung: ok - I'll approve17:14
rlandy|roverif we need to change it again , we will17:14
* myoung nods17:17
myoungrlandy|rover: since ocata is now gone, and newton is defunct anyway, at least for rdo on rhel, i'll prep a bigger patch that completely removes all the cruft entirely17:18
* myoung is taking bets on how much JJB can evaporate17:18
myoungbidding starts at 200 lines17:18
*** gkadam has quit IRC17:18
rlandy|roverlet's just see the ocata stuff through first17:20
myoungrlandy|rover: aye.17:30
EvilienMrlandy|rover: you can remove the reproducer, thanks again17:38
*** ykarel_ has joined #oooq17:39
rlandy|roverEvilienM: sure17:40
*** ykarel|away has quit IRC17:41
myoungrlandy|rover: looks like fs020 failed during UC install (still collecting logs), BM's failed for a mixture of libvirt install and that kernel missing, BMU job in flight and probably ok17:44
myoungrlandy|rover: what's wierd is https://code.engineering.redhat.com/gerrit/#/c/140417 merge 2 hours ago, but https://rhos-dev-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/tripleo-quickstart-ocata-rdo_trunk-baremetal-hp_dl360_envD-single_nic_vlans/69/console is showing No package matching 'kernel-3.10.0-693.el7.x86_64' found available, installed or updated"17:45
myoungsomething's smiss17:45
myoungamiss*17:46
*** myoung is now known as myoung|bbl17:47
* myoung|bbl steps out for appt17:47
rlandy|roverlooking17:54
*** ykarel_ is now known as ykarel|away17:57
*** ykarel|away has quit IRC18:02
*** trown|lunch is now known as trown18:04
rlandy|rover[root@rdoci-hp-01 ~]# cat /etc/redhat-release18:20
rlandy|roverCentOS Linux release 7.5.1804 (Core)18:20
rlandy|roverha - so it is 7.518:20
*** florianf has quit IRC18:22
rlandy|rovermyoung|bbl: it's the stable braches18:24
hubbotAll check jobs are working fine on stable/ocata, stable/ocata, master, stable/queens.18:26
rlandy|roveroh how nice19:05
rlandy|rover2018-05-14 ZuulMerge "Remove explicit --config-download" stable/master stable/pike stable/queens stable/rhos-12 stable/rhos-13 commit | commitdiff | tree | snapshot19:13
rlandy|roverno stable/ocata19:13
rlandy|rovermyoung|bbl: can I just ff the ocata branch?19:23
*** myoung|bbl is now known as myoung19:30
myoungrlandy|rover: yup19:31
rlandy|roverhttps://rhos-dev-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/periodic-ocata-rdo_trunk-featureset020-1ctlr_1comp_64gb/46/console19:31
rlandy|rover^^ failing19:31
rlandy|roverso I don't know if I can just ff19:31
* myoung looks19:31
rlandy|rovermyoung: wdyt?19:31
myoungrlandy|rover: yeah, tq/tqe are both 2+ months old at this point19:32
myoungrlandy|rover: let's rebase em...we can always put it back.  I recall hitting this on other branches19:33
rlandy|rovermyoung: ok - otherwise we will never promote19:33
rlandy|roverI think the same test will fail the ocata promotion though19:33
rlandy|roverbut hard to tell at this point19:33
rlandy|rover: 'enable_vbmc'  - that might hit us but we will have to fix it in line, I think19:34
myoungrlandy|rover: pretty sure we hit that in other branches as well...going back thru notes19:43
rlandy|roverhere we go again19:44
* myoung watches #17419:44
*** d0ugal has quit IRC19:47
myoungrlandy|rover: ick...the fs20 error we were hitting on ocata UC install was this again: (os-refresh-config) [ERROR] Could not lock /var/run/os-refresh-config.lock. [Errno 11] Resource temporarily unavailable19:54
myoung^^ https://bugs.launchpad.net/tripleo/+bug/166911019:55
openstackLaunchpad bug 1669110 in tripleo "undercloud fails with " [ERROR] Could not lock /var/run/os-refresh-config.lock."" [High,Triaged]19:55
*** d0ugal has joined #oooq20:05
myoungrlandy|rover: heh...wow...taking hp_dl360_envE offline to troubleshoot https://rhos-dev-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/tripleo-quickstart-ocata-rdo_trunk-baremetal-hp_dl360_envE-single_nic_vlans/87.  heh btw looks like 97 changes to TQ and 142 changes to TQE!  the error appears to be the same libvirt package deps.  getting on the virthost to confirm20:13
rlandy|rovernot sure why20:16
myoungrdoci-hp-07.v100.rdoci is rhe. 7.420:18
myoungi'm in there now (same tmate) - want me to kick off the upgrade?20:18
myoungi've disabled hp_dl360_envE jobs for all the branches so it won't be kicked20:18
rlandy|roverwe need to upgrade20:19
hubbotAll check jobs are working fine on stable/queens, stable/ocata, stable/ocata, master.20:26
myounghubbot: that's wonderful news20:30
hubbotmyoung: Error: "that's" is not a valid command.20:30
myounghubbot: kill -9 is a command20:30
hubbotmyoung: Error: "kill" is not a valid command.20:30
* myoung begs to differ20:30
rlandy|roverhttps://rhos-dev-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/periodic-ocata-rdo_trunk-featureset020-1ctlr_1comp_64gb/47/console20:33
rlandy|rovermaybe we got by it?20:33
*** yolanda has quit IRC20:35
rlandy|roverhmmm ... promotion failure ... 18:33:09 *** FAILED with status: 220:42
rlandy|rover18:33:09 Build step 'Execute shell' marked build as failure20:42
rlandy|rover18:33:09 [PostBuildScript] - Execution post build scripts.20:42
rlandy|roverlet's hope that's a one off20:42
*** dougbtv_ has joined #oooq20:43
*** dougbtv has quit IRC20:45
EvilienMajo: hey, sorry how can I help?20:52
ajohi EvilienM lol, you went evil ;)20:52
ajodid you see my messages ?20:53
ajoEvilienM: basically, from the network DFG, lots of love for the allinone tripleo rfe20:53
ajoEvilienM: we found a bunch of interesting use cases20:53
ajofrom, shifting developers into using tripleo deployments as a base (where you develop on your component but keep the versions of the other stuff)20:54
ajoto... eventually being able to snapshot allinones that one can download and use20:54
ajoif we were able to run the overcloud with the undercloud off20:54
ajoand some basic instructions about how to setup the network, etc20:54
ajoEvilienM: what do you think?20:54
EvilienMajo: so today we got something working20:55
EvilienMI was about to prepare some documentation20:55
ajooh wow20:55
EvilienMwe were able to deploy openstack on one node with just keystone20:55
ajoEvilienM: with just keystone? :)20:55
ajosounds like a very minimal openstack :D20:56
EvilienMyeah we start small :D20:56
ajohehe20:56
EvilienManyway20:57
EvilienMajo: it's a bit early now to test but if you want I can show you how we do20:57
ajoEvilienM: if you can loop me in into the documentation patches, that'd be great20:58
ajonow I was getting into bed, :D20:58
*** trown is now known as trown|outtypewww20:58
ajoEvilienM: what is your timezone?, sorry to rush you this morning ;)20:59
ajosorry for the rush, I mean ;)20:59
ajoI thought you were on an EU timezone20:59
EvilienMajo: I'm in Vancouver20:59
EvilienMhttp://paste.openstack.org/show/722539/20:59
EvilienMthat's how you can get started21:00
EvilienMbut AGAIN this is super experimental21:00
EvilienMyou can remove etcd also21:00
EvilienMI added it for testing21:00
*** rfolco has quit IRC21:06
*** d0ugal has quit IRC21:08
*** d0ugal has joined #oooq21:15
*** EvilienM has quit IRC21:18
*** EvilienM has joined #oooq21:21
*** holser__ has joined #oooq21:34
*** rfolco has joined #oooq21:35
myoungrlandy|rover: woot...fs20: 01:47:48.910     "overcloud_deploy_result": "passed"21:43
rlandy|rovernice21:45
*** holser__ has quit IRC21:52
hubbotAll check jobs are working fine on stable/queens, stable/ocata, stable/ocata, master.22:26
*** rlandy|rover has quit IRC22:39

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!