*** rlandy has quit IRC | 00:12 | |
*** bana_k has quit IRC | 00:20 | |
*** bana_k has joined #tripleo | 00:31 | |
*** sarath has quit IRC | 00:34 | |
*** apetrich has quit IRC | 00:40 | |
*** apetrich has joined #tripleo | 00:42 | |
*** dprince has quit IRC | 00:44 | |
*** limao has joined #tripleo | 00:45 | |
*** limao has quit IRC | 00:47 | |
*** limao has joined #tripleo | 00:47 | |
*** limao has quit IRC | 01:00 | |
*** limao has joined #tripleo | 01:00 | |
*** ramishra has quit IRC | 01:04 | |
*** bana_k has quit IRC | 01:10 | |
*** akshai has joined #tripleo | 01:23 | |
*** gfidente has quit IRC | 01:24 | |
*** akshai has quit IRC | 01:35 | |
*** saneax is now known as saneax_AFK | 01:37 | |
*** fzdarsky_ has joined #tripleo | 01:58 | |
*** fzdarsky has quit IRC | 02:02 | |
*** coolsvap has joined #tripleo | 02:19 | |
*** ramishra has joined #tripleo | 02:21 | |
*** tzumainn has quit IRC | 02:26 | |
*** ramishra has quit IRC | 02:26 | |
*** ramishra has joined #tripleo | 02:28 | |
*** yamahata has quit IRC | 02:28 | |
*** r-mibu has quit IRC | 03:24 | |
*** r-mibu has joined #tripleo | 03:24 | |
*** Nakato has quit IRC | 03:27 | |
*** colonwq is now known as colonwq_afk | 03:32 | |
*** Nakato has joined #tripleo | 03:32 | |
*** ayoung has quit IRC | 03:48 | |
*** links has joined #tripleo | 04:10 | |
*** masco has joined #tripleo | 04:29 | |
*** julim has quit IRC | 05:02 | |
*** jaosorior has joined #tripleo | 05:03 | |
*** yamahata has joined #tripleo | 05:05 | |
*** saneax_AFK is now known as saneax | 05:09 | |
openstackgerrit | Merged openstack/puppet-tripleo: Run local CA trust before haproxy deployment https://review.openstack.org/349412 | 05:11 |
---|---|---|
jaosorior | marios: Hey dude, if you get a chance to give the undercloud certmonger review another check https://review.openstack.org/#/c/332774/ Here's a commit that actually uses the code and it's passing CI https://review.openstack.org/#/c/349606/ | 05:12 |
*** mbound has joined #tripleo | 05:16 | |
*** bana_k has joined #tripleo | 05:19 | |
*** yamahata has quit IRC | 05:19 | |
*** mbound has quit IRC | 05:21 | |
*** bana_k has quit IRC | 05:22 | |
*** mburned has quit IRC | 05:27 | |
*** liverpooler has joined #tripleo | 05:28 | |
*** rcernin has joined #tripleo | 05:28 | |
*** mburned has joined #tripleo | 05:30 | |
*** liverpooler has quit IRC | 05:30 | |
*** liverpooler has joined #tripleo | 05:31 | |
marios | jaosorior: ack | 05:34 |
jaosorior | marios: Anything needing review on your side? | 05:35 |
marios | jaosorior: do not worry i will hassle you at the apropriate time ;) | 05:36 |
jaosorior | marios hahaha alright then, no problem | 05:37 |
*** fragatina has joined #tripleo | 05:45 | |
*** fragatina has quit IRC | 05:51 | |
*** fragatina has joined #tripleo | 05:52 | |
*** julim has joined #tripleo | 05:57 | |
bandini | morning folks | 06:00 |
*** aufi has joined #tripleo | 06:03 | |
openstackgerrit | Babu Shanmugam proposed openstack/tripleo-heat-templates: OVN heat templates https://review.openstack.org/307734 | 06:07 |
*** oshvartz has joined #tripleo | 06:10 | |
*** leanderthal|afk is now known as leanderthal | 06:12 | |
jaosorior | bandini: que onda wei | 06:13 |
*** karthiks|afk is now known as karthiks | 06:13 | |
bandini | jaosorior: ;) todo bien y tu? | 06:13 |
bandini | still looking into that aodh failure, it's a bit weird | 06:14 |
jaosorior | bandini: service_manage is set to false. Might it be an issue with the puppet manifest? | 06:14 |
bandini | dunno, still looking | 06:15 |
*** florianf has joined #tripleo | 06:23 | |
*** mburned has quit IRC | 06:24 | |
*** bana_k has joined #tripleo | 06:27 | |
*** mburned has joined #tripleo | 06:30 | |
*** hewbrocca-afk is now known as hewbrocca | 06:31 | |
*** bootsha has joined #tripleo | 06:34 | |
*** milan has joined #tripleo | 06:36 | |
*** pgadiya has joined #tripleo | 06:36 | |
*** panda|Zz is now known as panda | 06:44 | |
*** tesseract- has joined #tripleo | 06:45 | |
*** paramite has joined #tripleo | 06:48 | |
*** tremble has joined #tripleo | 06:57 | |
*** tremble has joined #tripleo | 06:57 | |
openstackgerrit | Juan Antonio Osorio Robles proposed openstack/instack-undercloud: Refactor endpoint env variable population https://review.openstack.org/350396 | 06:58 |
*** jbadiapa has joined #tripleo | 07:05 | |
ccamacho | morning! | 07:12 |
*** jaosorior is now known as jaosorior_brb | 07:15 | |
*** jpena|off is now known as jpena | 07:16 | |
*** zoli_gone-proxy is now known as zoliXXL | 07:19 | |
*** ohamada has joined #tripleo | 07:19 | |
*** pcaruana has joined #tripleo | 07:21 | |
*** tremble has quit IRC | 07:22 | |
*** links has quit IRC | 07:23 | |
*** mcornea has joined #tripleo | 07:24 | |
*** hewbrocca is now known as hewbrocca-afk | 07:25 | |
*** tremble has joined #tripleo | 07:26 | |
*** tremble has joined #tripleo | 07:26 | |
*** anshul has joined #tripleo | 07:27 | |
*** ebarrera has joined #tripleo | 07:29 | |
*** jpich has joined #tripleo | 07:35 | |
*** tremble has quit IRC | 07:36 | |
*** links has joined #tripleo | 07:36 | |
*** tremble has joined #tripleo | 07:40 | |
*** ebalduf has quit IRC | 07:41 | |
*** bana_k has quit IRC | 07:45 | |
*** hewbrocca-afk is now known as hewbrocca | 07:52 | |
*** Goneri has joined #tripleo | 07:54 | |
hewbrocca | Could we get a final ack for paramite on https://review.openstack.org/#/c/347399/ please? | 07:55 |
*** Goneri has quit IRC | 07:56 | |
*** Goneri has joined #tripleo | 07:56 | |
*** devvesa has joined #tripleo | 07:56 | |
*** gfidente has joined #tripleo | 07:57 | |
*** gfidente has quit IRC | 07:57 | |
*** gfidente has joined #tripleo | 07:57 | |
*** shardy has joined #tripleo | 07:59 | |
*** jistr is now known as jistr|training | 07:59 | |
*** radek_ has quit IRC | 08:01 | |
openstackgerrit | Julie Pichon proposed openstack/tripleo-common: Mistral workflow for creating a deployment plan from JSON input https://review.openstack.org/347830 | 08:02 |
*** dsariel has joined #tripleo | 08:02 | |
gfidente | shardy, fwiw, we still didn't get to the bottom of the mitaka/ci/tripleoclient failure | 08:05 |
gfidente | yesterday sshnaidm was looking into that as well and he was able to deploy with the RPM | 08:05 |
gfidente | wonder if we aren't using tripleoclient from a virtualenv | 08:06 |
gfidente | as it's only failing in ci | 08:07 |
*** coolsvap is now known as coolsvap_ | 08:08 | |
gfidente | so... still going through tripleo-ci looking for possible causes | 08:08 |
*** zoliXXL is now known as zoli|brb | 08:08 | |
*** limao_ has joined #tripleo | 08:10 | |
shardy | gfidente: Hmm, Ok. That is strange, as you can see the RPM get installed :( | 08:11 |
gfidente | yeah and the RPM is good | 08:11 |
shardy | gfidente: maybe we could ask someone with CI access to take a look when the test is actually running later | 08:11 |
shardy | or is there anything we can push as a WIP tripleo-ci patch which proves the version? | 08:12 |
*** jaosorior_brb has quit IRC | 08:12 | |
*** limao has quit IRC | 08:13 | |
*** jaosorior_brb has joined #tripleo | 08:13 | |
gfidente | shardy, well I could try to print the plugin version in overcloud_deploy | 08:14 |
gfidente | but to do so I'd have to use a depends_on | 08:14 |
gfidente | in which case we will *want* a non-rpm version of tripleoclient | 08:14 |
gfidente | well I could trigger this to see if the tht submission passes though | 08:15 |
sshnaidm | gfidente, shardy I ran a patched CI version with discovering version of tripleoclient, I didn't find anything other than installed | 08:16 |
sshnaidm | also didn't find installation of tripleoclient within the scripts other than "yum install", and of course delorean building | 08:17 |
gfidente | sshnaidm, hey, so the same version which passes locally, doesn't work in ci? | 08:17 |
shardy | sounds like we do need access to a CI testenv to debug this | 08:18 |
openstackgerrit | Giulio Fidente proposed openstack/python-tripleoclient: DO NOT MERGE: test tripleoclient in mitaka/ci https://review.openstack.org/350434 | 08:18 |
sshnaidm | gfidente, not really, I just tried tripleo-ci in master | 08:18 |
openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates: We don't need to set a default for the CephX keys and cluster FSID https://review.openstack.org/348900 | 08:19 |
*** oshvartz has quit IRC | 08:19 | |
sshnaidm | gfidente, tripleo-ci doesn't have branches, should be same for all | 08:19 |
*** snecklifter has joined #tripleo | 08:20 | |
shardy | well we do use different codepaths in tripleo.sh for stable branches | 08:20 |
gfidente | sshnaidm, right but yesterday you tested the tripleoclient RPM to deploy the overcloud, correct? | 08:20 |
sshnaidm | gfidente, maybe there is possible to do a few dependent patches - tripleoclient with version printing, tripleo-ci patch which run client to detect the version, and then the tht patch for mitaka, if all of them will run, maybe we'll discover something? | 08:20 |
sshnaidm | gfidente, yes, it succeeded | 08:21 |
gfidente | sshnaidm, yeah I tried the RPM and worked for me too | 08:21 |
*** oshvartz has joined #tripleo | 08:21 | |
gfidente | sshnaidm, so what I did for now is just pushing a fake tripleoclient update to stable/mitaka and make tht depends on it | 08:21 |
gfidente | if we checkout tripleoclient from stable/mitaka branch for the depends_on it should be overriding any local version and the tht submission pass | 08:22 |
sshnaidm | right | 08:22 |
gfidente | but let's add the version number too | 08:23 |
*** jaosorior_brb is now known as jaosorior | 08:23 | |
sshnaidm | gfidente, the version number is just for being sure we use particular client, not something installed in venv | 08:24 |
gfidente | sshnaidm, yep, ideally we want the git commit would | 08:25 |
sshnaidm | gfidente, you will see in logs when it's built by delorean | 08:26 |
*** kbyrne has quit IRC | 08:29 | |
shadower | jtomasek: ping | 08:29 |
jtomasek | shadower: pong | 08:30 |
shadower | jtomasek: question about the legacy API+UI: can you run a group of validations at once? And if so, how is that done? Does the UI run all validations individually or does it issue 1 API call to run them all? | 08:31 |
shadower | I've noticed something in the way the legacy API handles group validations and I wonder if it's going to be an issue | 08:31 |
*** fzdarsky_ is now known as fzdarsky | 08:32 | |
* jtomasek checks | 08:32 | |
*** dculik has quit IRC | 08:33 | |
jtomasek | shadower: GUI uses /stages/${stageId}/run | 08:33 |
jtomasek | (PUT) | 08:33 |
shadower | jtomasek: right, thanks | 08:34 |
shadower | jtomasek: that would be a problem, but it turns out, the API just runs all the validations individually for that request so we're good | 08:34 |
jtomasek | shadower: ok | 08:34 |
* shadower highfives himself from 6 moths ago | 08:34 | |
shadower | jtomasek: and fwiw it's good that the UI uses that API call -- that's why we put it there | 08:35 |
snecklifter | hi folks, I've managed to get this passing CI, any chance of a review: https://review.openstack.org/#/c/313933/ | 08:35 |
snecklifter | I'm not the owner but have a vague interest in it getting committed, thanks | 08:35 |
jtomasek | shadower: haha | 08:36 |
shardy | snecklifter: sorry, we can't merge that in it's current state, it needs to be refactored into the composable services format, which I think jaosorior already commented on | 08:37 |
*** noshankus has joined #tripleo | 08:40 | |
*** zoli|brb is now known as zoli | 08:42 | |
*** zoli is now known as zoliXXL | 08:42 | |
*** electrofelix has joined #tripleo | 08:43 | |
shardy | snecklifter: let me know if any help is needed, I can probably cut/paste it pretty quickly if someone else is willing to test the resulting patch | 08:45 |
noshankus | Hi guys - I'm trying to run quickstart.sh through a http proxy... it seems to be failing at the setup.py stage while trying to resolve pbr: "Download error on https://pypi.python.org/simple/pbr/: [SSL: UNKNOWN_PROTOCOL] unknown protocol (_ssl.c:765) -- Some packages may not be found!" I have set env vars for http_proxy and https_proxy (both pointing to a _http_ address) - any ideas? | 08:45 |
shardy | noshankus: have you tested other requests which use https_proxy to confirm it's not a proxy issue? | 08:47 |
noshankus | Yep - for example, the git clone beforehand worked: "git clone https://github.com/openstack/tripleo-quickstart.git /root/.quickstart/tripleo-quickstart" | 08:47 |
shardy | noshankus: but how about e.g a pip install, as that appears to be where it's failing? | 08:48 |
noshankus | I can also wget both http and https urls successfully... if I run the command "python setup.py install egg_info --egg-base /root/.quickstart" manually... it seems to work | 08:48 |
*** dculik has joined #tripleo | 08:48 | |
*** radek_ has joined #tripleo | 08:49 | |
*** bootsha has quit IRC | 08:49 | |
noshankus | Yep - pip install works... | 08:49 |
shardy | noshankus: Ok, may have to ask trown|outtypewww or larsks later when they come online, not sure what else to suggest | 08:49 |
*** jistr|training is now known as jistr | 08:49 | |
jaosorior | hey guys, more or less how long does it take for a change in a component (in my case mistral) to be taken into account in the tripleo gates? I need a fix that was merged there to verify if a tripleo patch works | 08:50 |
noshankus | I'm using latest CentOS-7-x86_64-Everything-1511.iso and latest quickstart.sh from master... | 08:50 |
gfidente | shardy, sshnaidm I might have found the issue with the tht change | 08:50 |
noshankus | @shardy - ok thanks :) | 08:51 |
shardy | jaosorior: whenever current-tripleo gets promoted | 08:51 |
shardy | https://dashboards.rdoproject.org/rdo-dev | 08:51 |
shardy | jaosorior: if you need to test it before then, you can post a WIP patch to tripleo-ci that adds the package to the delorean-current whitelist in tripleo.sh then use a Depends-On | 08:52 |
sshnaidm | gfidente, cool! what is it? | 08:52 |
shardy | gfidente: nice! | 08:52 |
gfidente | :( | 08:52 |
gfidente | it's not nice | 08:52 |
gfidente | you'll want to kill me | 08:52 |
shardy | lol | 08:52 |
gfidente | let me update | 08:52 |
jaosorior | shardy: Alright, will check that out. Thanks man | 08:52 |
* shardy waits to see the diff | 08:52 | |
jaosorior | jistr: Hey dude, if you get time, could you check out the certmonger patch for the undercloud? https://review.openstack.org/#/c/332774/ this other commit is actually testing it in CI https://review.openstack.org/#/c/349606/ | 08:54 |
openstackgerrit | Giulio Fidente proposed openstack/python-tripleoclient: Generate CephX keys and FSID regardless of CephStorageCount https://review.openstack.org/350460 | 08:55 |
sshnaidm | are any DIB people here? | 08:55 |
jistr | jaosorior: ack will review that | 08:55 |
openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates: We don't need to set a default for the CephX keys and cluster FSID https://review.openstack.org/348900 | 08:56 |
sshnaidm | gfidente, so it didn't work, when no ceph storage? | 08:57 |
shardy | d0ugal: Hey, what's the deal with the deployment/template mistral patches? | 08:57 |
gfidente | sshnaidm, yeah because client wasn't generating the key in that case | 08:57 |
snecklifter | shardy: thanks for comments, reviewing now | 08:57 |
shardy | I expected to see confirmation of my debugging/fix and a new series posted? | 08:57 |
shardy | did you and rbrady confirm my findings? | 08:57 |
d0ugal | shardy: Yeah, I did. | 08:59 |
d0ugal | shardy: been working on the CLI, I'll push your fix now... | 08:59 |
shardy | d0ugal: ack, cool - I just wanted to ensure I'm testing the latest stuff locally | 08:59 |
openstackgerrit | Dougal Matthews proposed openstack/tripleo-common: Adds parameters actions https://review.openstack.org/298682 | 08:59 |
openstackgerrit | Dougal Matthews proposed openstack/tripleo-common: Adds Deployment Action & Workflow https://review.openstack.org/298732 | 08:59 |
openstackgerrit | Dougal Matthews proposed openstack/tripleo-common: Adds action for template processing https://review.openstack.org/337615 | 08:59 |
*** oneswig has joined #tripleo | 08:59 | |
d0ugal | shardy: ^ | 08:59 |
shardy | d0ugal: thanks, ping me when there's a new CLI patch and I'll pull/test/review | 09:00 |
shardy | looks like we're super-close now :) | 09:00 |
d0ugal | shardy: Yup, will do! | 09:00 |
jaosorior | shardy: can you check out this revert to make swift the glance backend again? https://review.openstack.org/#/c/349843/ | 09:02 |
*** mgould|afk is now known as mgould | 09:02 | |
*** shardy is now known as shardy_mtg | 09:02 | |
shardy_mtg | jaosorior: will do | 09:03 |
*** dculik has quit IRC | 09:05 | |
openstackgerrit | Martin Mágr proposed openstack/tripleo-specs: Availability monitoring support https://review.openstack.org/347399 | 09:08 |
sshnaidm | does anybody know when can I find DIB related folks? | 09:08 |
*** TSCHAK has quit IRC | 09:11 | |
*** dculik has joined #tripleo | 09:18 | |
*** akrivoka has joined #tripleo | 09:20 | |
*** kbyrne has joined #tripleo | 09:21 | |
*** dtantsur|afk is now known as dtantsur | 09:24 | |
marios | shardy_mtg: o/ fyi haven't forgotten about my AI to look into the upgrades blueprint... it is on my todo list today (just got sidetracked with a support thing but tdb how much time that will take). i will ping you later once i've looked | 09:28 |
openstackgerrit | Christian Schwede proposed openstack/tripleo-common: Install the upload-* scripts with tripleo-common https://review.openstack.org/350474 | 09:31 |
*** electrofelix has quit IRC | 09:40 | |
*** electrofelix has joined #tripleo | 09:40 | |
*** shardy_mtg is now known as shardy | 09:43 | |
shardy | marios: ack, thanks! | 09:43 |
*** zoliXXL is now known as zoli|lunch | 09:47 | |
*** bootsha has joined #tripleo | 09:49 | |
openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack/tripleo-quickstart: [WIP] undercloud reboot https://review.openstack.org/350489 | 09:53 |
*** bootsha has quit IRC | 09:55 | |
*** yamahata has joined #tripleo | 09:56 | |
*** tremble has quit IRC | 09:56 | |
*** Jaison has joined #tripleo | 09:58 | |
*** links has quit IRC | 09:59 | |
*** tremble has joined #tripleo | 09:59 | |
*** tremble has quit IRC | 09:59 | |
*** tremble has joined #tripleo | 09:59 | |
*** bootsha has joined #tripleo | 09:59 | |
openstackgerrit | Carlos Camacho proposed openstack/tripleo-heat-templates: Disable zaqar services by default. https://review.openstack.org/350497 | 10:01 |
openstackgerrit | Carlos Camacho proposed openstack/tripleo-heat-templates: Disable zaqar services by default. https://review.openstack.org/350497 | 10:02 |
openstackgerrit | Julie Pichon proposed openstack/python-tripleoclient: Add 'openstack overcloud plan list' command https://review.openstack.org/350498 | 10:03 |
openstackgerrit | Dougal Matthews proposed openstack/tripleo-common: Generate reference documentation for Workbooks https://review.openstack.org/341572 | 10:03 |
d0ugal | jpich: Any idea what happens with your create workflow if the plan already exists? | 10:03 |
jpich | d0ugal: "A container with this name already exists" error, I think | 10:04 |
d0ugal | jpich: okay, thanks. I think I am going to have to use this in the deploy command | 10:05 |
*** radek_ has quit IRC | 10:05 | |
jpich | d0ugal: How so? | 10:06 |
d0ugal | jpich: The deploy command accepts templates as an arg, so I need to create a plan on the fly | 10:06 |
shardy | what happens if you do a deploy, then delete the stack? | 10:08 |
shardy | we don't have an overcloud delete command | 10:08 |
shardy | sounds like we may need one, to delete the stack, then delete (or at least rename) the plan | 10:08 |
*** Goneri has quit IRC | 10:08 | |
shardy | I would probably prefer we used a timestamp or something as a suffix, or can we use container versioning? | 10:09 |
d0ugal | shardy: You can have an undeployed plan, so it would just become that I guess? | 10:09 |
shardy | d0ugal: I'm just thinking if I do overcloud deploy --templates foo | 10:09 |
*** fragatina has quit IRC | 10:09 | |
shardy | then we'll try to create a new container called "overcloud" | 10:09 |
shardy | but if I do that twice, there will already be a container with that name | 10:09 |
d0ugal | shardy: The container name matches the stack name | 10:10 |
shardy | which always defaults to overcloud | 10:10 |
d0ugal | That's how the workflows link them at the moment | 10:10 |
*** chem has joined #tripleo | 10:10 | |
d0ugal | shardy: Sure, it will default to that. | 10:10 |
*** yamahata has quit IRC | 10:11 | |
d0ugal | shardy: so I am now at the point of getting a CREATE_FAILED and the CLI handles that correctly! | 10:11 |
d0ugal | Trying to figure out what went wrong now. | 10:11 |
shardy | \o/ | 10:11 |
shardy | openstack overcloud stack failures list | 10:12 |
jpich | "overcloud delete" sounds like a scary operation | 10:12 |
shardy | jpich: why? Folks do openstack stack delete atm | 10:12 |
shardy | only that will only delete the heat stack, not anything in swift | 10:13 |
d0ugal | jpich: heat stack-delete overcloud - much the same thing. | 10:13 |
d0ugal | shardy: is that a real command? | 10:13 |
shardy | d0ugal: yes | 10:13 |
d0ugal | I don't seem to have it :/ | 10:13 |
jpich | shardy, d0ugal: If everything worked fine, does it remove the stuff that got properly deployed? | 10:13 |
shardy | d0ugal: pull the latest heatclient, maybe we've not yet released it | 10:13 |
d0ugal | jpich: Yup | 10:14 |
* shardy will chase tagging one | 10:14 | |
jpich | d0ugal: Scary ;) | 10:14 |
d0ugal | shardy: oh, it's in heat client. Works when I remove "overcloud" :) | 10:14 |
*** tremble has quit IRC | 10:14 | |
shardy | cool | 10:14 |
*** sanjayu|AFK has quit IRC | 10:14 | |
*** sanjayu|AFK has joined #tripleo | 10:15 | |
d0ugal | (or rather, move it to the end) | 10:15 |
shardy | d0ugal: I think we should add that same thing to tripleoclient, so we basically run that by default whenver the overcloud deploy fails | 10:15 |
d0ugal | +1 | 10:15 |
d0ugal | Okay, seems I am missing keystone_signing_certificate. /me digs | 10:15 |
*** tremble has joined #tripleo | 10:16 | |
*** tdasilva has quit IRC | 10:16 | |
d0ugal | We really need the keystone certs to be created in the workflow | 10:19 |
d0ugal | I guess for now I'll hack it in and we can move it over | 10:19 |
d0ugal | I have no idea how the UI will do this otherwise | 10:19 |
d0ugal | hrm, there is actually no good way for me to do this from the CLI | 10:19 |
d0ugal | since I can't pass the deploy workflow a env file | 10:20 |
d0ugal | shardy: ^ | 10:20 |
*** radek_ has joined #tripleo | 10:20 | |
shardy | d0ugal: well, we'll have to fix that ;) | 10:20 |
d0ugal | aye | 10:20 |
shardy | agree the cert stuff needs to move into an action (as does the os-cloud-config endpoint stuff) | 10:20 |
d0ugal | I'll see if I can hack it in for now to find out what else is broken | 10:21 |
d0ugal | then I can fix bits like this individually | 10:21 |
shardy | d0ugal: do you need any help or are you happy pushing on this? | 10:21 |
*** spredzy has quit IRC | 10:22 | |
d0ugal | shardy: I'm happy pushing it, but help would be good. | 10:22 |
shardy | seems like we need two new actions, an adjustment to the workflow, and an interface for user provided environments to the templates workflow | 10:22 |
d0ugal | +1 | 10:22 |
d0ugal | and we need to figure out what else is broken :) | 10:22 |
rook | EmilienM: https://review.openstack.org/#/c/350219/ passed CI minus the non-voting centos-7-undercloud, which seems to have failed very early... | 10:23 |
d0ugal | shardy: Maybe the idea is that env files are added during the plan creation. | 10:23 |
jaosorior | jistr: thanks for your comments on the certmonger patch dude. So I could actually work on getting HAProxy to always be there as follow up work. I just left it the way it was before cause that's what the previous behaviour was. | 10:23 |
*** spredzy has joined #tripleo | 10:23 | |
*** Goneri has joined #tripleo | 10:24 | |
*** tosky has joined #tripleo | 10:25 | |
shardy | d0ugal: I suppose it could go into either the plan generation or the templates processing | 10:26 |
d0ugal | shardy: for flexibility we might need both. | 10:26 |
shardy | probably depends if we want to persist the ssl stuff in swift | 10:26 |
d0ugal | True | 10:26 |
d0ugal | We could really do with Barbican for some of this... | 10:27 |
rook | shardy hey! long time --- curious if you have time today to give me the state of composable roles in OOO? (or maybe there is a email/etherpad I should be following) | 10:27 |
shardy | well lets pick one to start with - want me to hack a new action into the plan generation? | 10:27 |
jaosorior | jistr: By the way, note that we still support passing the certificate. In that case, certmonger wouldn't be tracking anything. | 10:28 |
shardy | don't want to overlap with your efforts but happy to help as I need this working asap | 10:28 |
d0ugal | shardy: Sure, that would be great. So it will just create the certs etc. when the plan is created? | 10:28 |
jaosorior | so certmonger only comes into play if we set the "generate_service_certificate" flag | 10:28 |
jaosorior | in the undercloud | 10:28 |
* d0ugal isn't sure what the "etc" part is :) | 10:28 | |
shardy | d0ugal: Yeah I'll drop a yaml file into the plan, then we just need to include it | 10:28 |
shardy | rook: Hi! | 10:28 |
*** limao_ has quit IRC | 10:29 | |
shardy | rook: sure, basically we've landed nearly all of the composable services now | 10:29 |
rook | Most excellent! | 10:29 |
d0ugal | shardy: Great. | 10:29 |
shardy | rook: we don't yet have landed support for custom roles, but I'm working on it | 10:29 |
shardy | this stuff d0ugal and I are discussing is the next step towards it | 10:29 |
d0ugal | shardy: btw, the deploy workflow ends before the stack operation is finished - it doesn't wait. that might be an issue for the endpoint work. | 10:29 |
rook | composable roles vs custom roles? | 10:29 |
shardy | rook: composable services means you can move services between existing roles, custom roles is when you want e.g a completely new group of nodes, e.g "ContrailController" or something | 10:30 |
*** rodrigods has quit IRC | 10:30 | |
shardy | d0ugal: yeah, that needs fixing too | 10:30 |
rook | ah ha! | 10:30 |
*** rodrigods has joined #tripleo | 10:30 | |
shardy | rook: I've been planning a blog post about it, will try to do it later today | 10:30 |
rook | shardy++ | 10:31 |
rook | no karma bot fail | 10:31 |
shardy | https://etherpad.openstack.org/p/tripleo-deep-dive-topics | 10:31 |
shardy | rook: You can also see the most recent deep dive, which was about puppet but touched on the composable stuff too | 10:31 |
rook | shardy thanks! We are getting Mitaka numbers now (at scale) so we will have Liberty-> Mitaka-> Newton numbers (mem usage, time, etc) | 10:32 |
rook | Should be pretty cool. | 10:32 |
openstackgerrit | Dougal Matthews proposed openstack/python-tripleoclient: Migrate the deploy command to use the Mistral workflows https://review.openstack.org/342261 | 10:33 |
shardy | rook: nice! How is liberty-mitaka looking? | 10:33 |
shardy | I would assume roughly the same? | 10:33 |
d0ugal | shardy: ^ still far from complete, but that updated patch will now at least not error out (well, it might if the stack create works!) | 10:33 |
shardy | d0ugal: thanks, will pull it now | 10:33 |
rook | shardy so far, mitaka looks worse for some odd reason. | 10:34 |
openstackgerrit | Dougal Matthews proposed openstack/python-tripleoclient: Migrate the deploy command to use the Mistral workflows https://review.openstack.org/342261 | 10:34 |
shardy | rook: Hmm, Ok, well be happy to help look into that if you can provide access to data | 10:34 |
rook | shardy however, I haven't personally looked at it. | 10:34 |
rook | shardy this is someone on my team - once we have the data, ill be sure to send it off your way for sure. | 10:35 |
shardy | I expect the newton numbers to look quite a lot better, but very keen to get confirmation on that | 10:35 |
rook | shardy++ | 10:35 |
rook | me too | 10:35 |
rook | shardy I will personally work on that when it lands in RDO. | 10:35 |
shardy | rook: nice, thanks! | 10:35 |
*** social has quit IRC | 10:36 | |
d0ugal | shardy: We probably need a version of this that doesn't write to a file. https://github.com/openstack/os-cloud-config/blob/master/os_cloud_config/keystone_pki.py#L140 | 10:36 |
rook | Now will we have a custom role for OpenShift :P | 10:37 |
shardy | d0ugal: Yeah, for now I was going to pass it a temporary file | 10:37 |
shardy | we can fix it later to return a map | 10:37 |
*** tdasilva has joined #tripleo | 10:38 | |
*** tosky has quit IRC | 10:39 | |
*** tosky has joined #tripleo | 10:39 | |
*** tosky has quit IRC | 10:39 | |
*** tosky has joined #tripleo | 10:40 | |
*** tosky has quit IRC | 10:41 | |
*** social has joined #tripleo | 10:41 | |
*** tosky has joined #tripleo | 10:43 | |
openstackgerrit | Juan Antonio Osorio Robles proposed openstack/instack-undercloud: Use certmonger for automatic cert generation https://review.openstack.org/332774 | 10:44 |
jaosorior | jistr: addressed your comment of using reload instead of restart ^^ | 10:44 |
*** milan has quit IRC | 10:48 | |
*** Jaison has quit IRC | 10:49 | |
*** tosky has quit IRC | 10:49 | |
marios | gfidente: jaosorior can you please revisit https://review.openstack.org/#/c/350046/1 | 10:50 |
jaosorior | marios: I did | 10:51 |
marios | gfidente: jaosorior shardy https://review.openstack.org/#/c/188137 when you get a chance please. we have a green run at https://review.openstack.org/#/c/342961/ wdyt? | 10:51 |
jaosorior | nice! | 10:51 |
marios | jaosorior: thanks | 10:51 |
gfidente | so | 10:52 |
marios | jaosorior: i haven't forgotten your reviews man, i got pulled into some other thing. which is ongoing but quiet for now, iso i'll try do some reviews | 10:52 |
gfidente | this can be uses against you in the future | 10:52 |
gfidente | I will :) | 10:52 |
gfidente | I was more concerned about the manila_config cleanup though | 10:53 |
marios | gfidente: there are a few things that are given in life, and one must learn to deal with them. You taking advantage of this situation is a give. I was waiting after you typed 'so' i was just expecting you to say 'NO" and torment me for a while | 10:53 |
*** links has joined #tripleo | 10:53 | |
marios | gfidente: i am alsmost disappointed that you didn't | 10:53 |
openstackgerrit | Dougal Matthews proposed openstack/os-cloud-config: Extract the keystone cert generation into a function https://review.openstack.org/350515 | 10:54 |
*** tosky has joined #tripleo | 10:54 | |
marios | i .. i can't bring myself to say it... | 10:54 |
marios | "thanks gfidente" | 10:55 |
gfidente | marios, ahahah | 10:55 |
d0ugal | :-o | 10:55 |
*** bvandenh has joined #tripleo | 10:56 | |
openstackgerrit | Merged openstack/puppet-tripleo: Fixup nit in manila pacemaker profile, duplicate variable https://review.openstack.org/350046 | 11:01 |
*** oneswig has quit IRC | 11:06 | |
jaosorior | marios: regarding the certmonger patch. I actually changed to using reload in patch #37 https://review.openstack.org/#/c/332774/37 | 11:07 |
jaosorior | that's pretty much the only change | 11:07 |
marios | jaosorior: ack yeah man i had seen it but already wrote the comment so posted it whilst reviewing v37 agan 'properly' | 11:07 |
marios | lgtm, comments there | 11:08 |
*** kbyrne has quit IRC | 11:11 | |
openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci: Reinstall python-requests in bootstrap script https://review.openstack.org/350521 | 11:11 |
*** kbyrne has joined #tripleo | 11:12 | |
jistr | jaosorior: thanks for the replies, +2'd | 11:13 |
jaosorior | marios, jistr: I'm trying again an update from old style auto generate to the certmonger approach | 11:15 |
marios | jaosorior: nice thanks | 11:16 |
gfidente | shardy, so it's green now https://review.openstack.org/#/c/348900/ , it just needed https://review.openstack.org/#/c/350460/ :( | 11:20 |
openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci: Don't assume an eth1 https://review.openstack.org/350524 | 11:20 |
*** lucasagomes is now known as lucas-hungry | 11:21 | |
shardy | gfidente: ack, thanks, approved! | 11:22 |
gfidente | shardy sorry ;( | 11:22 |
shardy | no worries! :) | 11:22 |
*** rhallisey has joined #tripleo | 11:25 | |
*** tremble has quit IRC | 11:25 | |
*** yolanda has quit IRC | 11:27 | |
*** tremble has joined #tripleo | 11:28 | |
*** yolanda has joined #tripleo | 11:31 | |
jaosorior | jistr, marios: crap... so with the addition of the anchor I did; doing an upgrade from old-style auto generate to the certmonger approach can fail (certmonger executes some stuff pretty randomly). For some reason puppet disregards the notify I had in puppet-tripleo that should execute the CA code before the haproxy code. So it starts doing keystone commands before tha CA is trusted, and thus the SSL | 11:32 |
jaosorior | verification fails. This isn't an issue with FreeIPA, or if it's a new deployment. But yeah, this case is an issue. I need to figure out a way for puppet to respect that ordering. I marked the commit as WIP again until I figure this out. Sorry about it. | 11:32 |
marios | jaosorior: ack thanks for checking good thing jistr brought it up then | 11:33 |
jistr | jaosorior: no worries thanks for verifying :) | 11:33 |
jaosorior | had tested it, but it was before I added the anchor. I'll let you guys know when I come up with a solution and have tested the full matrix again (new deployment, update, and FreeIPA) | 11:34 |
openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci: Don't assume an eth1 https://review.openstack.org/350524 | 11:34 |
thrash | zaneb: ping... got this when trying to scale up with your patch for the image id thing: Resource CREATE failed: BadRequest: resources.NovaCompute: Invalid imageRef provided. (HTTP 400) | 11:35 |
*** bootsha has quit IRC | 11:35 | |
marios | jaosorior: nobody likes your excuses jaosorior i mean look at gfidente. fully joking obviously man thanks for checking and the fact you did and -1d it is why i was happy to say and stand by " I also think jaosorior is sane enough" | 11:36 |
marios | for what *thats* worth anyway | 11:36 |
*** aviramb has joined #tripleo | 11:37 | |
*** bootsha has joined #tripleo | 11:38 | |
gfidente | marios, right I noticed on manila you were already at patchset #91 and didn't want you to hit 100 | 11:38 |
marios | :( | 11:38 |
marios | gfidente: so in my defence it was already weeel into double digits by the time i came to it | 11:39 |
gfidente | you started | 11:39 |
gfidente | I just replied | 11:39 |
marios | gfidente: what did i ever do to you gfidente :( i will have to report you to hr | 11:40 |
gfidente | let me see if I can find a picture | 11:40 |
*** dculik has quit IRC | 11:44 | |
*** sshnaidm is now known as sshnaidm|afk | 11:44 | |
*** coolsvap_ is now known as coolsvap | 11:45 | |
*** dprince has joined #tripleo | 11:45 | |
gfidente | https://makeameme.org/meme/people-told-me-wj6h3r | 11:46 |
gfidente | or better https://media.makeameme.org/created/people-told-me-wj6h3r.jpg | 11:46 |
marios | :( | 11:47 |
*** limao has joined #tripleo | 11:47 | |
*** morazi has joined #tripleo | 11:50 | |
openstackgerrit | Christopher Brown proposed openstack/tripleo-heat-templates: Enable Neutron LBaaS Integration https://review.openstack.org/313933 | 11:51 |
*** fultonj has joined #tripleo | 11:51 | |
*** bfournie has quit IRC | 11:52 | |
jaosorior | marios, jistr: actually, I just noticed that the try I did did not include the latest puppet-tripleo | 11:54 |
jaosorior | so the notify I did wasn't in the package I used for puppet-tripleo, for some reason | 11:54 |
jistr | ah ok, so there's still a chance that everything's fine | 11:55 |
jaosorior | jistr: Yes | 11:55 |
jaosorior | I'm trying to build the puppet-tripleo package with what's in master | 11:55 |
jaosorior | but it's the package build is failing using tripleo.sh | 11:56 |
*** tzumainn has joined #tripleo | 11:56 | |
jistr | perhaps you could try the swift artifacts .tar.gz approach | 11:56 |
jaosorior | jistr: Can that work with testing the undercloud? | 11:57 |
jaosorior | thought it was an overcloud thing | 11:57 |
jistr | jaosorior: of course it can't, i'm confused :)) | 11:57 |
jistr | jaosorior: so i think in this case you could just update it in-place | 11:57 |
jistr | if the RPM building gives you lemons | 11:58 |
marios | jaosorior: ack, theres a bunch of things that review depended on and that's just the last few revisions since i looked so i'm not surprised but yeah take your time and verify would be good | 11:58 |
openstackgerrit | Attila Darazs proposed openstack/tripleo-quickstart: Add ceph_pacemaker config https://review.openstack.org/350536 | 11:58 |
jaosorior | marios: Yep, verifying again | 11:59 |
*** limao has quit IRC | 11:59 | |
openstackgerrit | Brad P. Crochet proposed openstack/tripleo-common: Change image names to match what the rest of the system expects https://review.openstack.org/336642 | 11:59 |
openstackgerrit | Brad P. Crochet proposed openstack/tripleo-common: Remove the delorean-repo element from CentOS images https://review.openstack.org/336524 | 11:59 |
openstackgerrit | Brad P. Crochet proposed openstack/tripleo-common: Remove epel element from base yaml ramdisk agent image https://review.openstack.org/336551 | 11:59 |
*** limao_ has joined #tripleo | 12:00 | |
*** zoli|lunch is now known as zoli | 12:01 | |
EmilienM | hi | 12:01 |
*** zoli is now known as zoliXXL | 12:01 | |
panda | there's something during undercloud install that modifies /etc/sudoers ? | 12:02 |
openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci: Make $TRIPLEO_ROOT overrideable https://review.openstack.org/350539 | 12:03 |
jaosorior | EmilienM: sup dude | 12:03 |
*** radek_ has quit IRC | 12:04 | |
*** milan has joined #tripleo | 12:05 | |
*** weshay has joined #tripleo | 12:07 | |
openstackgerrit | Gael Chamoulaud proposed openstack/tripleo-quickstart: Run dstat on the virthost and the undercloud https://review.openstack.org/349591 | 12:11 |
*** ccamacho is now known as ccamacho|lunch | 12:11 | |
openstackgerrit | Steven Hardy proposed openstack/os-cloud-config: Add the option to dump keystone pki data to a string not a file https://review.openstack.org/350548 | 12:13 |
*** dculik has joined #tripleo | 12:13 | |
openstackgerrit | Gael Chamoulaud proposed openstack/tripleo-quickstart: Run dstat on the virthost and the undercloud https://review.openstack.org/349591 | 12:14 |
openstackgerrit | Steven Hardy proposed openstack/tripleo-common: Add action to generate PKI certs in plan https://review.openstack.org/350549 | 12:15 |
shardy | d0ugal: sorry for the delay, I think that'll do it | 12:15 |
shardy | I only added it to the default plan creation workflow for now | 12:15 |
jaosorior | shardy: I thought we weren't using keystone PKI tokens anymore | 12:15 |
jaosorior | And the plan was to move to fernet | 12:16 |
shardy | we still generate the certs and pass them in, but I actually think the default is still uuid | 12:16 |
d0ugal | shardy: np, I'll check it out shortly. Just going to grab some lunch now. | 12:16 |
jaosorior | shardy: I think the recommendation is not to use PKI tokens | 12:16 |
shardy | jaosorior: that is just moving some existing logic from tripleoclient to mistral | 12:16 |
*** limao_ has quit IRC | 12:17 | |
shardy | jaosorior: feel free to investigate, it may be that we need to adjust defaults and deprecate that stuff | 12:17 |
shardy | jaosorior: for now, the deployment fails without it | 12:17 |
*** limao has joined #tripleo | 12:17 | |
jaosorior | shardy: I think we should be deprecating it. But yeah, we can ask ayoung when he's online | 12:17 |
d0ugal | that would be even better :) | 12:17 |
jaosorior | d0ugal, what? | 12:17 |
*** chem has quit IRC | 12:18 | |
d0ugal | jaosorior: deprecating it :) | 12:18 |
*** ramishra has quit IRC | 12:18 | |
shardy | rip it all out and make my last hours work pointless :D | 12:18 |
d0ugal | lol | 12:18 |
*** jpena is now known as jpena|lunch | 12:18 | |
*** chem has joined #tripleo | 12:18 | |
* d0ugal will bbiab | 12:18 | |
jaosorior | shardy lol, sorry about that dude | 12:19 |
*** oshvartz has quit IRC | 12:19 | |
EmilienM | slagle: why did you do a recheck on https://review.openstack.org/#/c/350266/ ? | 12:19 |
EmilienM | slagle: the patch failed, and it was expected | 12:19 |
EmilienM | also see https://review.openstack.org/#/c/350265/ | 12:19 |
shardy | jaosorior: see KeystoneCACertificate in ./puppet/services/keystone.yaml | 12:19 |
EmilienM | slagle: it demonstrates well that puppet modules are installed from source when needed | 12:19 |
shardy | I'm fine with deprecating it if that's the reccomended approach, was only joking | 12:19 |
EmilienM | slagle: http://logs.openstack.org/66/350266/1/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha/69bf25c/logs/postci.txt.gz#_2016-08-02_22_41_36_000 | 12:20 |
shardy | it's not actually PKI is it | 12:20 |
shardy | jaosorior: we generate KeystoneCACertificate and KeystoneSigningKey | 12:21 |
jaosorior | shardy: shardy, no worries, lets just consult ayoung when he's online. He's the keystone guru | 12:21 |
slagle | EmilienM: the multinode job failed b/c of a different error and never made it to OC deploy | 12:22 |
slagle | but, yea, i see that the uc job did fail on the right thing | 12:22 |
EmilienM | slagle: no, oc | 12:22 |
EmilienM | slagle: but yeah I'll let the jobs finish to make sure it works everywhere | 12:22 |
EmilienM | shardy, slagle, bnemec: fyi, I got an undercloud upgrade working in CI from Mitaka to newton | 12:23 |
slagle | EmilienM: ok, i'm confused | 12:23 |
EmilienM | upgrade: https://review.openstack.org/#/c/346995/ | 12:23 |
slagle | EmilienM: first, the console log you linked me is from an ovb ha job | 12:23 |
*** adarazs is now known as adarazs_brb | 12:23 | |
EmilienM | slagle: right, so we're good, we see the Puppet failure, which was wanted | 12:23 |
slagle | EmilienM: the point of this though is to make sure the multinode job fails | 12:24 |
slagle | on the puppet failure | 12:24 |
EmilienM | slagle: ok | 12:24 |
openstackgerrit | Saravanan KR proposed openstack/os-net-config: Add support for OVS DPDK Bridge and Port https://review.openstack.org/338694 | 12:24 |
slagle | and it failed on a different thing, which is the bug i filed and fixed and then rechecked | 12:24 |
EmilienM | slagle: sorry I was confused why you rechecked; I'll make sure it works in multi node too :-) | 12:24 |
slagle | assuming the recheck gets it on to a different cloud (rax) | 12:25 |
openstackgerrit | Merged openstack/python-tripleoclient: Generate CephX keys and FSID regardless of CephStorageCount https://review.openstack.org/350460 | 12:25 |
EmilienM | slagle: it doesn't matter | 12:25 |
jaosorior | shardy: By the way, if we are defaulting to UUID tokens, what are we using the keystoneCA and signing keys for? | 12:25 |
*** tremble has quit IRC | 12:25 | |
*** bfournie has joined #tripleo | 12:26 | |
*** tremble has joined #tripleo | 12:27 | |
slagle | EmilienM: it does matter | 12:27 |
slagle | EmilienM: due to the other unrelated bug | 12:27 |
*** jayg|g0n3 is now known as jayg | 12:28 | |
slagle | EmilienM: everything will fail on ovh and osic clouds right now | 12:28 |
shardy | jaosorior: we don't actually specify the token format in services/keystone AFAICT, so it's probably from when keystone or puppet-keystone had pki as a default? | 12:28 |
shardy | If we can remove them now, lets do that | 12:28 |
slagle | EmilienM: https://review.openstack.org/#/c/350524/ | 12:28 |
hewbrocca | I know keystone guys would be really happy if we got rid of UUID tokens | 12:29 |
EmilienM | slagle: damn | 12:29 |
openstackgerrit | Emilien Macchi proposed openstack/python-tripleoclient: Stop all services before upgrading undercloud https://review.openstack.org/331804 | 12:29 |
EmilienM | bnemec: I took over, and I tested and upgraded mitaka to newton successfuly ^ | 12:30 |
*** dculik has quit IRC | 12:30 | |
shardy | hewbrocca: AFAICS we're just using the keystone default | 12:30 |
*** limao has quit IRC | 12:31 | |
hewbrocca | Cool | 12:32 |
*** adarazs_brb is now known as adarazs | 12:34 | |
jaosorior | hewbrocca: which is UUID AFAIK | 12:34 |
*** sshnaidm|afk is now known as sshnaidm | 12:34 | |
shardy | https://github.com/openstack/keystone/blob/master/etc/keystone.conf.sample | 12:35 |
shardy | according to that all the cert stuff is deprecated | 12:35 |
dprince | shardy: this is what I'm using for my signalling fix | 12:35 |
dprince | shardy: https://review.openstack.org/350553 A hook which invokes os-apply-config. | 12:35 |
shardy | so I wonder if we can reasonably remove it | 12:35 |
shardy | given that we don't actually offer any parameter that enables operators to choose pki | 12:35 |
shardy | they might get broken if they're forcing the token format via hieradata | 12:35 |
shardy | but that's true for almost any hiera override I think | 12:35 |
shardy | dprince: nice, will check it out! | 12:36 |
openstackgerrit | Attila Darazs proposed openstack/tripleo-quickstart: Add new pacemaker configs https://review.openstack.org/350536 | 12:36 |
jaosorior | jistr, marios: Fuck yeaaaah! it does work with ugprades! Just tried it again on a freshl deployment, making sure the puppet-tripleo dependency was there. | 12:37 |
marios | rock | 12:37 |
jistr | jaosorior: neat. So i think we can un-wip and land | 12:37 |
*** radek_ has joined #tripleo | 12:37 | |
jistr | once CI results come in of course | 12:37 |
jaosorior | jistr, marios: sure. Either way, I also re-ran the non-ha, so it should again show that it works on the first run. | 12:38 |
openstackgerrit | Steven Hardy proposed openstack/puppet-tripleo: Remove keystone PKI cert generation https://review.openstack.org/350554 | 12:38 |
marios | jaosorior: yeah maybe add some info as a comment on what/how you tested | 12:38 |
jaosorior | marios: will do | 12:39 |
openstackgerrit | Steven Hardy proposed openstack/tripleo-heat-templates: Remove keystone PKI related parameters https://review.openstack.org/350556 | 12:41 |
shardy | d0ugal: Ok, that's the other way to fix it ;) | 12:41 |
*** rlandy has joined #tripleo | 12:41 | |
*** lucas-hungry is now known as lucasagomes | 12:43 | |
shardy | dprince: the only weird thing about that is o-a-c isn't operating on input data from the deployment? | 12:44 |
dprince | shardy: yep, we don't need it I think though | 12:44 |
shardy | Sure, for this case, but it makes it kind of broken as a generic heat-config hook | 12:44 |
dprince | shardy: it is weird. The point was this solves signalling issues for good here | 12:44 |
shardy | dprince: yeah, I'd like to see us deploy hiera via a hook too | 12:45 |
dprince | shardy: would you rather tie in signalling to 99-refresh-completed which is already quite a mess? | 12:45 |
dprince | shardy: I was about to do that... but this seemed easier since it is all plummed and ready to go | 12:46 |
shardy | dprince: No, I'd really like to see 99-refresh-completed die in a fire ;) | 12:46 |
dprince | shardy: right, so this is a step towards that | 12:46 |
dprince | shardy: except that we do still need os-refresh-config | 12:46 |
openstackgerrit | Gael Chamoulaud proposed openstack/tripleo-quickstart: Run dstat on the virthost and the undercloud https://review.openstack.org/349591 | 12:47 |
dprince | the os-refresh-config element that is, which is where 99-refresh-config lives | 12:47 |
*** julim has quit IRC | 12:48 | |
dprince | shardy: So right now this would send 2 signals I think, unless you removed that file | 12:48 |
shardy | dprince: Yeah, that's the part I'm unsure on - if you have a group: os-apply-config SoftwareConfig, this hook will run, but what deploys the data passed to the config? | 12:48 |
dprince | shardy: the data is deployed by os-collect-config | 12:49 |
dprince | shardy: os-collect-config writes metadata. All of it | 12:49 |
dprince | shardy: and os-apply-config gets at it by using the /var/lib/os-collect-config/os_config_files.json list | 12:49 |
shardy | yeah, I meant we have no way of knowing which deployment acts on which piece of metadata | 12:49 |
dprince | shardy: we already have that with os-apply-config | 12:50 |
dprince | shardy: to me the point here is signalling | 12:50 |
shardy | Yeah, but it's not what any other heat hooks do | 12:50 |
dprince | shardy: it isn't | 12:50 |
dprince | shardy: but group: os-apply-config in our heat templates is the only case that doesn't abide by proper signal handling IMO | 12:51 |
dprince | shardy: and this would fix that | 12:51 |
openstackgerrit | Saravanan KR proposed openstack/os-net-config: Add support for OVS DPDK Bond https://review.openstack.org/349183 | 12:51 |
shardy | dprince: well, it wouldn't really fix it, because if you run 10 group: os-apply-config concurrently and one of them fails you have zero idea which one was actually broken | 12:52 |
shardy | one will just randomly fail | 12:52 |
shardy | it fixes the transport though, that's true | 12:52 |
gfidente | breaking news, we are now gating on ceph/jewel in master and ceph/hammer in stable/liberty | 12:52 |
shardy | dprince: FWIW I envisaged a hook like this which *only* worked with the data passed to the deployment | 12:52 |
shardy | so we can break the oac global merging of configs thing | 12:53 |
shardy | we'd have to rename it e.g group: heat-os-apply-config I guess | 12:53 |
dprince | shardy: sure, so I think that isn't called os-apply-config | 12:53 |
dprince | shardy: but we could switch to it, sure | 12:53 |
*** tosky has quit IRC | 12:53 | |
shardy | dprince: Ok, cool, just throwing some ideas around, I'm OK with this as an interim measure if it's needed | 12:54 |
shardy | ideally I'd like us to stop using os-apply-config completely ;) | 12:54 |
dprince | shardy: well, this gives me signals | 12:55 |
*** tosky has joined #tripleo | 12:55 | |
dprince | shardy: its an idea. If people don't like it we can go a different direction | 12:55 |
shardy | dprince: sure, I think it's better than what we have currently as it fixes some transports, and gives us an error path | 12:55 |
dprince | shardy: but, the short term would likely be adding more to 99-refresh-completed | 12:56 |
shardy | dprince: yeah, I guess if we land this we can then disable the signalling in there | 12:56 |
dprince | shardy: I wasn't sure on that actually. People may will still depend on that | 12:57 |
dprince | shardy: like Magnum or something | 12:57 |
dprince | shardy: lots of groups use os-apply-config I think as an element | 12:57 |
shardy | dprince: we could have it check the installed hooks and not signal if the o-a-c one is installed | 12:57 |
dprince | shardy: so maybe we keep both elements, and one "provides" the os-apply-config requirement | 12:58 |
dprince | shardy: yep, I thought of that too | 12:58 |
shardy | the duplicate signal will be ignored, but they generate confusing events | 12:58 |
dprince | shardy: have the user just choose one over the other seems cleaner. DIB already has a "provides" feature so I think we could use it to satisfy the requirement | 12:58 |
*** rhallisey has quit IRC | 13:00 | |
openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: Implement undercloud upgrade job - Mitaka -> Newton https://review.openstack.org/346995 | 13:00 |
EmilienM | slagle: when you have time, could you give feedback on ^ please? | 13:01 |
slagle | EmilienM: yep will do | 13:04 |
EmilienM | slagle: thanks, also bnemec & shardy when you have time | 13:04 |
*** akrivoka has quit IRC | 13:05 | |
*** rhallisey has joined #tripleo | 13:05 | |
EmilienM | slagle: when we have the upstream undercloud upgrade CI job in place, I would like to continue with overcloud. Did you already start something on this side? | 13:07 |
slagle | EmilienM: not for upgrades | 13:08 |
slagle | EmilienM: i'm doing some testing of HA locally | 13:08 |
EmilienM | slagle: do you mind if I start looking at it? | 13:08 |
slagle | EmilienM: at what? upgrades? | 13:08 |
EmilienM | yes | 13:09 |
slagle | no. go4it | 13:09 |
EmilienM | ok | 13:09 |
openstackgerrit | Saravanan KR proposed openstack/os-net-config: Add support for OVS DPDK Bond https://review.openstack.org/349183 | 13:09 |
openstackgerrit | Saravanan KR proposed openstack/os-net-config: Add support for OVS DPDK Bridge and Port https://review.openstack.org/338694 | 13:09 |
EmilienM | slagle: I am a bit confused about current status of baremetal jobs, are we supposed to have the old upgrade job back soon? | 13:09 |
slagle | what baremetal jobs? | 13:10 |
EmilienM | slagle: I plan to start small and upgrade a single node overcloud | 13:10 |
slagle | or do you just mean ovb? | 13:10 |
EmilienM | slagle: you know, the old upgrade job we had the last months | 13:10 |
EmilienM | which was more an "update" job | 13:10 |
slagle | we should be able to add all the old jobs back once we bring back full capacity on rh1 | 13:10 |
EmilienM | ok, when is it scheduled? | 13:10 |
slagle | EmilienM: when do you want to schedule it? :) | 13:11 |
EmilienM | tonight ! | 13:11 |
slagle | EmilienM: if you can check the jobs that have been running on rh1 and see if they are stable | 13:11 |
slagle | if they look good, then propos a patch to increase the workers on rh1 in project-config | 13:11 |
*** ramishra has joined #tripleo | 13:11 | |
slagle | then look at adding the jobs back | 13:11 |
*** oshvartz has joined #tripleo | 13:12 | |
*** jcoufal has joined #tripleo | 13:12 | |
slagle | EmilienM: these are the patches: | 13:14 |
slagle | https://review.openstack.org/#/c/348884/ | 13:14 |
slagle | https://review.openstack.org/#/c/348880/ | 13:14 |
slagle | https://review.openstack.org/#/c/348879/ | 13:14 |
slagle | EmilienM: can you push on those? | 13:15 |
slagle | if we can try and move those forward... | 13:15 |
slagle | that is the road to getting rh1 back | 13:15 |
*** jpena|lunch is now known as jpena | 13:16 | |
EmilienM | slagle: sure thing, /me looks now | 13:16 |
*** ccamacho|lunch is now known as ccamacho | 13:17 | |
openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: Enable net-iso for ovb jobs https://review.openstack.org/343017 | 13:17 |
EmilienM | this is a blocker iiuc ^ | 13:18 |
*** akrivoka has joined #tripleo | 13:19 | |
EmilienM | slagle: can you look https://review.openstack.org/350580 ? name is ok? | 13:21 |
*** jdob has quit IRC | 13:22 | |
*** jdob has joined #tripleo | 13:23 | |
dtantsur | folks, do you know if ooo-quickstart tears down the previous undercloud by default? | 13:23 |
openstackgerrit | Attila Darazs proposed openstack/tripleo-quickstart: Add minimal_pacemaker config https://review.openstack.org/350536 | 13:23 |
*** pgadiya has quit IRC | 13:25 | |
openstackgerrit | Gabriele Cerami proposed openstack/diskimage-builder: secure_path in sudoers: deal with possible quotes https://review.openstack.org/350582 | 13:25 |
*** fragatina has joined #tripleo | 13:28 | |
marios | shardy: and magic https://bugs.launchpad.net/tripleo/+milestone/newton-3 | 13:29 |
openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack/tripleo-quickstart: [WIP] undercloud reboot https://review.openstack.org/350489 | 13:31 |
panda | this blocked one of my undercloud installs, don't know why that option is surrounded by quotes now. ^ | 13:31 |
shardy | marios: thanks! :) | 13:32 |
openstackgerrit | Karthik S proposed openstack/tripleo-heat-templates: Configure the pci_passthrough_whitelist via THT https://review.openstack.org/329415 | 13:33 |
*** noshankus has quit IRC | 13:33 | |
openstackgerrit | Jiri Tomasek proposed openstack/tripleo-ui: Environments Listing and Updating using Mistral Actions https://review.openstack.org/350586 | 13:34 |
openstackgerrit | Gael Chamoulaud proposed openstack/tripleo-quickstart: Run dstat on the virthost and the undercloud https://review.openstack.org/349591 | 13:36 |
*** jdob has quit IRC | 13:36 | |
*** jdob has joined #tripleo | 13:37 | |
*** lblanchard has joined #tripleo | 13:40 | |
jaosorior | dtantsur: I don't know. What I do is actually do a destroy and then undefine of the undercloud domain. And then I run the quickstart.sh again | 13:40 |
dtantsur | hmm, not very handy | 13:40 |
jaosorior | dtantsur: not really. If you figure it out let me know. Would save me some time too | 13:41 |
openstackgerrit | Gael Chamoulaud proposed openstack/tripleo-quickstart: Run dstat on the virthost and the undercloud https://review.openstack.org/349591 | 13:41 |
*** radek_ has quit IRC | 13:47 | |
openstackgerrit | Jeff Peeler proposed openstack/tripleo-common: Change input for UpdateCapabilitiesAction https://review.openstack.org/349732 | 13:49 |
*** fragatina has quit IRC | 13:51 | |
*** ebalduf has joined #tripleo | 13:53 | |
*** akshai has joined #tripleo | 13:54 | |
jaosorior | dprince: Does zaqar use some sort of api-paste configuration? | 13:56 |
dprince | jaosorior: I don't think so. Don't rememver | 13:58 |
jaosorior | dprince: Also, I tried doing a simple curl to zaqar to see how the version json-home behaves and I got an error straight: http://paste.openstack.org/show/547772/ | 13:58 |
jaosorior | dprince: This is in a clean deployment with the latest undercloud | 13:59 |
jaosorior | I haven't put anything in front of zaqar yet | 13:59 |
jaosorior | I'm working on adding TLS for it in the undercloud | 13:59 |
dprince | jaosorior: Now I feel like I've seen this before | 13:59 |
dprince | jaosorior: I think it may have been from a too recent autobahn library or something | 14:00 |
dprince | jaosorior: using the packages? | 14:00 |
*** fragatina has joined #tripleo | 14:00 | |
*** krot_sickleave is now known as krotscheck | 14:00 | |
*** akshai has quit IRC | 14:00 | |
dprince | jaosorior: when I hit this the package didn't work at all | 14:00 |
openstackgerrit | Merged openstack-infra/tripleo-ci: Don't assume an eth1 https://review.openstack.org/350524 | 14:00 |
openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack/tripleo-quickstart: [WIP] undercloud reboot https://review.openstack.org/350489 | 14:00 |
dprince | jaosorior: I had to go and add policy.json, and request that RDO build the autobahn dep. | 14:01 |
jaosorior | dprince: Yes, using packages | 14:01 |
dprince | jaosorior: so, I've seen it but I can't remember what exactly I did to solve it | 14:01 |
jaosorior | dprince: policy.json is present in /etc/zaqar | 14:01 |
dprince | jaosorior: other than I ended up not needing to fix anything | 14:01 |
dprince | jaosorior: right policy.json isn't related. Just saying I used to pip install it because it was broken otherwise | 14:02 |
dprince | jaosorior: and that was when I hit this error | 14:02 |
*** akshai has joined #tripleo | 14:02 | |
openstackgerrit | Steven Hardy proposed openstack/tripleo-heat-templates: Replace hard-coded regionOne with parameter references https://review.openstack.org/350599 | 14:02 |
jaosorior | dprince: Anybody I could ask about this? | 14:03 |
hewbrocca | Hey folks, just a little workflow love for paramite spec and ops-tools will be off to the races | 14:06 |
hewbrocca | https://review.openstack.org/#/c/347399/ | 14:06 |
openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: Implement undercloud upgrade job - Mitaka -> Newton https://review.openstack.org/346995 | 14:06 |
hewbrocca | already got a lovely +2 from mr slagle | 14:07 |
dprince | jaosorior: it has to do with falcon | 14:07 |
jaosorior | seems so | 14:07 |
dprince | jaosorior: there was a revert for this, sec | 14:07 |
slagle | EmilienM: looks like both your test CI patches failed at the right place this time | 14:07 |
dprince | jaosorior: 9cfa5bc8ca1b31845848a0d3b5db0ee068854739 | 14:08 |
dprince | jaosorior: I filed a bug for this and it was a dup | 14:08 |
dprince | jaosorior: memory is coming back to me now :) | 14:08 |
dprince | jaosorior: looking for links... | 14:08 |
EmilienM | slagle: ok, I'll double check and abandon them, so we know we're good now :) | 14:08 |
*** tbonds has joined #tripleo | 14:08 | |
dprince | jaosorior: https://bugs.launchpad.net/zaqar/+bug/1583358 | 14:10 |
openstack | Launchpad bug 1583358 in zaqar "Zaqar tempest test failed" [High,Fix released] - Assigned to Fei Long Wang (flwang) | 14:10 |
pabelanger | EmilienM: slagle: how is tripleo-ci this morning? Any problems stopping us from increasing capacity on -rh1? | 14:10 |
dprince | jaosorior: and this was the commit that fixed it e6d975fd1009c70a1d2b3fbe699895345220562e | 14:10 |
slagle | pabelanger: EmilienM was going to check | 14:11 |
dprince | jaosorior: double check that... and perhaps if you need more help reach out to wanghao who seems familiar in this territory | 14:11 |
openstackgerrit | Christopher Brown proposed openstack/tripleo-heat-templates: Enable Neutron LBaaS Integration https://review.openstack.org/313933 | 14:12 |
EmilienM | pabelanger: we have some blockers apparently | 14:12 |
EmilienM | pabelanger: AFIK we need https://review.openstack.org/#/c/343017/ first | 14:12 |
openstackgerrit | Merged openstack/tripleo-ui: Add prepublish script to bundle dependencies https://review.openstack.org/350019 | 14:12 |
*** ayoung has joined #tripleo | 14:12 | |
slagle | EmilienM: why is that a blocker? | 14:13 |
jaosorior | dprince: Actually it seems that I have the proposed fix as part of the package | 14:13 |
EmilienM | slagle: required for https://review.openstack.org/#/c/348884/ | 14:13 |
dprince | jaosorior: link? | 14:13 |
ayoung | jaosorior, once you are done with this, can we discuss TLS/SSL? | 14:13 |
jaosorior | dprince: No link, just checked the source code | 14:14 |
jaosorior | ayoung: if it's fast. I gotta go soon :/ | 14:14 |
dprince | jaosorior: sorry, what I mean is look to see if there have been any recent changes there | 14:14 |
ayoung | jaosorior, OK | 14:14 |
dprince | jaosorior: when I saw issues with this that was the fix... | 14:14 |
jaosorior | ayoung: what's up? | 14:14 |
jaosorior | dprince: It does seem to me that the packaged version of falcon is 0.3.0 | 14:15 |
dprince | jaosorior: this happend 2 months back, thus my memory fails me a bit here | 14:15 |
ayoung | jaosorior, so I need to set up TLS with Hostname support | 14:15 |
ayoung | I was looking at http://docs.openstack.org/developer/tripleo-docs/advanced_deployment/ssl.html and a couple things are not clear | 14:15 |
ayoung | I did a deploy using CloudName: my-overcloud.my-domain.com | 14:15 |
ayoung | DnsServers: 10.0.0.100 | 14:15 |
ayoung | but modified for mine, and the auth URL is still not the FQDN | 14:15 |
slagle | EmilienM: ok. why is https://review.openstack.org/#/c/348884/ a blocker? | 14:15 |
jaosorior | ayoung: you gotta use CLOUDNAME instead of IP_ADDRESS in the enable-tls.yaml | 14:16 |
*** ramishra has quit IRC | 14:16 | |
EmilienM | slagle: you told me so | 14:16 |
EmilienM | 1h ago | 14:16 |
ayoung | jaosorior, ah, so not just in the overall setup? I had not yet done TLS...OK, that makes sense. Next question | 14:16 |
EmilienM | and I'm still figuring why | 14:16 |
slagle | EmilienM: i did not say that was a blocker | 14:16 |
ayoung | if I set things up using self signed, can I swap in differn certs afterwards? | 14:16 |
EmilienM | slagle: ok, good. Thanks. | 14:17 |
slagle | EmilienM: it was one of a few patches | 14:17 |
jaosorior | ayoung: Also remember to have a certificate that has the the fqdn you would use, and not the ip address (I've messed that up too many times) | 14:18 |
jaosorior | dprince: I think it's related to having an old falcon version | 14:18 |
ayoung | jaosorior, yes, that is what I am trying to do. I was hoping to deploy with FQDN without TLS first to test things out, and then to roll in TLS, but looks like I need to go all the way to make it work? | 14:18 |
ayoung | jaosorior, so I set: | 14:18 |
ayoung | CloudName: openstack.ayoung-dell-t1700.test | 14:19 |
ayoung | CloudDomain: ayoung-dell-t1700.test | 14:19 |
ayoung | DnsServers: 10.18.57.26 | 14:19 |
jaosorior | ayoung: You can do it if you override the EndpointMap to use CLOUDNAME instead of IP_ADDRESS, which is the default | 14:19 |
jaosorior | so you need something similar to enable-tls.yaml but without the TLS bits (which is the ports and the https protocols) | 14:19 |
ayoung | jaosorior, I was also hoping to not have to copy the entire template directory, but rather just the templates I need to modify | 14:19 |
d0ugal | shardy: With https://review.openstack.org/#/c/350549/ do we not need something in the deployment workflow to make it include the environment? | 14:20 |
jaosorior | ayoung: You only need one environment file | 14:20 |
jaosorior | to override that value | 14:20 |
*** jeckersb is now known as jeckersb_gone | 14:20 | |
ayoung | jaosorior, and that needs to just have the same values as enable-tls.yaml does in the sample...let me try | 14:20 |
ayoung | jaosorior, what is that about a root cert? | 14:21 |
jaosorior | ayoung: Not the same values. If you're not doing TLS, then you don't need any of the certs and CA stuff | 14:21 |
EmilienM | pabelanger: honestly, I'm not familiar enough atm to know the road to RH1, I started to look at the patches this morning | 14:21 |
jaosorior | and you need to switch the ports to the defaults (not the overridden ports that we use for TLS), and also set all the ports to http instead of https | 14:21 |
EmilienM | pabelanger: I would ask bnemec probably | 14:21 |
dtantsur | hmm, now is it normal for ooo-quickstart to get stuck on "Prepare the undercloud for deploy" for an hour? | 14:21 |
ayoung | jaosorior, I do want to do TLS, just step by step | 14:22 |
jaosorior | ooh, alright | 14:23 |
jaosorior | ayoung: Then in that case just copy enable-tls.yaml | 14:23 |
jaosorior | and add the cert and key there | 14:23 |
*** links has quit IRC | 14:23 | |
ayoung | jaosorior, so the part that says | 14:23 |
ayoung | resource_registry: | 14:23 |
ayoung | OS::TripleO::NodeTLSData: ../puppet/extraconfig/tls/tls-cert-inject.yaml | 14:23 |
jaosorior | the root ca you need to add to another environment | 14:23 |
slagle | dtantsur: fyi, i dont know if quickstart people answer in this channel. they have their own channel apparently. you might try there. or encourage them to join here | 14:23 |
ayoung | can I edit that and make it an absolute URL? | 14:23 |
jaosorior | ayoung: Yes | 14:23 |
ayoung | er path | 14:23 |
dtantsur | slagle, oh, really? do you know the channel name? | 14:23 |
jaosorior | the root CA you add here https://github.com/openstack/tripleo-heat-templates/blob/master/environments/inject-trust-anchor.yaml | 14:24 |
jaosorior | that's another environment | 14:24 |
* dtantsur is really surprised, provided that we're pushing for quickstart to become the default entrance to tripleo | 14:24 | |
slagle | dtantsur: oooq maybe? or ooo-q | 14:24 |
ayoung | ok, so I have a bunch of files I am maintainitng in a git repo in /home/stack/deployment. I'll add a copy of enable-tls in there with my values | 14:24 |
slagle | dtantsur: yea, well, not everyone is | 14:24 |
bnemec | EmilienM: pabelanger: tripleo-ci looks to be in good shape today, and I see seven passed jobs on rh1 so I'd say let's go for it. | 14:24 |
slagle | dtantsur: for reasons such as these :) | 14:24 |
ayoung | jaosorior, should be enough to get me rolling. I'll bug you more tomorrow. I still need to make it work with the CA cert issued by IPA. | 14:24 |
pabelanger | bnemec: EmilienM: okay, let me get a review up in openstack-infra | 14:25 |
bnemec | When I looked yesterday the failed jobs on rh1 all appeared to be legit, either due to problems with the patch or the ceph bug. | 14:25 |
jaosorior | ayoung: awesome! hope it goes well :) | 14:25 |
jaosorior | I gotta go now | 14:25 |
shardy | d0ugal: Yes, but I'm currently testing https://review.openstack.org/#/c/350556/ as an alternative | 14:25 |
jaosorior | have a good day guys! | 14:25 |
*** jaosorior has quit IRC | 14:25 | |
dtantsur | slagle, hmm, I see.. it would be nice to get it sorted.. thanks, #oooq worked | 14:25 |
slagle | dtantsur: it is sorted from my perspective :) | 14:25 |
dtantsur | heh | 14:26 |
EmilienM | pabelanger: https://review.openstack.org/#/c/348879/ | 14:26 |
d0ugal | shardy: oh, I guess I could try just setting them to an empty string for now | 14:26 |
EmilienM | pabelanger: and https://review.openstack.org/#/c/348880/ | 14:26 |
dmsimard | Hey, are specs rendered somewhere ? https://github.com/openstack/tripleo-specs mentions http://docs.openstack.org/developer/tripleo-specs but that's a dead link. | 14:26 |
shardy | d0ugal: Yeah, it looks like we need to remove them, then switch to fernet but I wouldn't get blocked on it | 14:26 |
shardy | AFAICS we don't use them anymore | 14:26 |
slagle | dmsimard: http://specs.openstack.org/openstack/tripleo-specs/ | 14:27 |
dtantsur | slagle, as a person a bit aside of tripleo processes, I'm really confused by these numerous ways of setting up a virtual environment.. | 14:27 |
dmsimard | slagle: okay, I'll submit a patch to fix the link :) | 14:27 |
slagle | dtantsur: we have 1 documented way | 14:27 |
openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates: Add support for deploying RGW on controllers. (Composable) https://review.openstack.org/289027 | 14:27 |
openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates: Add support for deploying Ceph RGW role https://review.openstack.org/289027 | 14:28 |
slagle | dtantsur: i was just working on a patch to update it since it broke when we removed epel repos | 14:28 |
*** ramishra has joined #tripleo | 14:28 | |
*** yamahata has joined #tripleo | 14:28 | |
slagle | dtantsur: tripleo quickstart was supposed to replace that 1 way. but docs were never updated, so afaic it's not replaced | 14:29 |
dtantsur | slagle, I've heard people saying things like "tripleo-docs is hopelessly broken". was it referring to this epel problem? | 14:29 |
shardy | also CI doesn't use it, which is a reason many of did not switch to quickstart | 14:29 |
slagle | dtantsur: no. and i don't subscribe to "hopelessly broken" fud | 14:30 |
slagle | i'm sure there are problems | 14:30 |
slagle | we fix them | 14:30 |
dtantsur | cool. because I've used the docs successfully for quite some time | 14:30 |
slagle | or we can fix bugs that are filed against the docs | 14:30 |
dtantsur | so I was just as surprised | 14:30 |
shardy | dtantsur: if quickstart doesn't work for you, and you want to automate some steps from the docs (and get pretty close to what we run in CI) try this: | 14:30 |
shardy | https://paste.fedoraproject.org/400728/14702346 | 14:30 |
shardy | those are notes of how I set my env up, many folks do similar | 14:30 |
dtantsur | aha, now tripleo.sh :) this is the 3rd way, but I think someone was working on changing docs to use it | 14:31 |
shardy | dtantsur: No, it's just a script we use in CI | 14:31 |
shardy | I was trying to be helpful and get things working for you :\ | 14:31 |
shardy | the steps are supposed to match the docs, but invariably docs rot, so we need to auto-generate them from CI | 14:32 |
dtantsur | thanks, shardy, I'm not blaming you at all, just trying to figure out the whole picture | 14:32 |
slagle | dtantsur: tripleo.sh is copy/paste commands from the docs. that's what it is there for | 14:32 |
jrist | I know everyone is very busy but can we get some eyes on https://review.openstack.org/#/c/299937/ ? | 14:33 |
openstackgerrit | Julie Pichon proposed openstack/python-tripleoclient: Add 'openstack overcloud plan delete' command https://review.openstack.org/350617 | 14:33 |
openstackgerrit | David Moreau Simard proposed openstack/tripleo-specs: Update rendered specs URL https://review.openstack.org/350618 | 14:35 |
openstackgerrit | Giulio Fidente proposed openstack/puppet-tripleo: Add cinder-backup profiles https://review.openstack.org/311218 | 14:36 |
shardy | jrist: the assumption of plan names always matching stack names seems bogus to me, so we'll probably end up having to fix that | 14:37 |
shardy | I can imagine folks wanting to upload multiple plans, then deploy overclouds with the same name, changing plan version while testing | 14:38 |
*** myoung is now known as myoung|mtg | 14:38 | |
shardy | I guess we can land it as-is, but it seems to be an indication we've not really thought through the operator workflow around plans yet? | 14:38 |
jrist | rbrady: ^ | 14:38 |
openstackgerrit | James Slagle proposed openstack/tripleo-incubator: Don't require EPEL https://review.openstack.org/350619 | 14:39 |
shardy | Similar with the hacky introspection of overcloud.yaml in the roles listing | 14:39 |
openstackgerrit | Merged openstack/tripleo-validations: Documentation how validations work https://review.openstack.org/322178 | 14:39 |
shardy | like, it's fine as a first step but it's going to break pretty soon | 14:39 |
shardy | jrist: I'd probably prefer we added a PlanName parameter to overcloud.yaml, then always pass in the name of the actual plan used when creating/updating a stack | 14:40 |
rbrady | shardy, jrist: this has been used int he previous implementation and was included in the spec for discussion. I have no problem changing it, but the concept has had opportunity for discussion but seemed maybe it was missed | 14:41 |
shardy | rbrady: sure, I guess it was missed - I'm just pointing out that as someone who regularly uses TripleO, this won't work for me | 14:42 |
shardy | maybe it's fine as a first revision tho | 14:42 |
shardy | like, already today I've been having to manually delete the overcloud plan then do swift upload of my local templates | 14:43 |
shardy | it's be much nicer to enable multiple plans and just switch between them | 14:43 |
shardy | openstack overcloud deploy --plan overcloud_dev123 | 14:43 |
shardy | kinda similar to what we used to do with Tuskar | 14:43 |
jrist | heh | 14:44 |
rbrady | shardy: is this something you've recently discovered? (the fact that this would not work for you) | 14:44 |
EmilienM | shardy, slagle: do you know why delorean would fail to build tripleoclient when using depends-on? see http://logs.openstack.org/95/346995/26/experimental/gate-tripleo-ci-centos-7-undercloud-upgrades-nv/a2046f8/console.html#_2016-08-03_14_32_10_939863 | 14:44 |
shardy | rbrady: Yes, since I've been using the WIP mistral based deployment patches | 14:44 |
*** ramishra has quit IRC | 14:44 | |
shardy | they've only been semi-working since yesterday | 14:44 |
shardy | jrist: what's the UI flow around this, does it assume only ever having one plan and one overcloud? | 14:45 |
jrist | jtomasek: can you comment on ^ - I believe that is *not* the case | 14:45 |
jrist | especially considering OpsTools | 14:46 |
*** ramishra has joined #tripleo | 14:46 | |
shardy | rbrady: I'm not trying to be difficult btw, it's expected that we'll iterate on these interfaces until folks are comfortable | 14:46 |
jrist | that sounds like a +2 to me? :) | 14:47 |
jrist | shardy: :) | 14:47 |
jrist | jk | 14:47 |
rbrady | shardy: I'm not thinking that you personally are trying to be difficult. I am frustrated that this detail was in 2 previously approved specs and was never discussed | 14:47 |
rbrady | shardy: I could have implemented it the way you're suggesting earlier...being more efficient, etc | 14:48 |
*** fragatina has quit IRC | 14:49 | |
*** myoung|mtg is now known as myoung | 14:49 | |
*** fragatina has joined #tripleo | 14:49 | |
openstackgerrit | Martin André proposed openstack/tripleo-validations: Validation to detect if undercloud disks use 512e https://review.openstack.org/322179 | 14:50 |
shardy | rbrady: To be fair, this isn't directly addressed by the spec | 14:50 |
shardy | "The deletion of a plan will be implemented in a Workflow to ensure there isn't an associated stack before deleting the templates," | 14:50 |
shardy | there's no mention of plan or stack multiplicity | 14:50 |
*** ramishra has quit IRC | 14:50 | |
shardy | rbrady: I'm cool to just land it if it's going to cause frustration - I'll post patches revising it for you consideration afterwards | 14:51 |
* jtomasek reads back | 14:51 | |
shardy | rbrady: FWIW my confusion may stem from the fact that we documented things like listing plans | 14:51 |
shardy | if you can't select different plans, I'm not sure why we'd do that | 14:52 |
rbrady | shardy: in this case, it's removing code. I can take some time away from deployment to change this | 14:53 |
openstackgerrit | Martin André proposed openstack/tripleo-validations: Add network gateway validation https://review.openstack.org/329385 | 14:53 |
*** ramishra has joined #tripleo | 14:53 | |
shardy | rbrady: Don't take any time away from deployment, let's get that working and fix this later | 14:53 |
larsks | hey all. what consumes tripleo-puppet-elements? | 14:53 |
shardy | I'm just trying to have an open discussion having spent the last two days digging into this trying to help | 14:54 |
rbrady | larsks: large purple monsters | 14:54 |
larsks | YUM! | 14:54 |
hewbrocca | The triple owl | 14:54 |
jrist | hahaha | 14:54 |
openstackgerrit | Martin André proposed openstack/tripleo-validations: Port the network environment validation https://review.openstack.org/341586 | 14:54 |
shardy | purple owls? | 14:54 |
larsks | Seriously though, what process actually applies those? | 14:54 |
hewbrocca | d-i-b, right? | 14:54 |
larsks | But what *passes* those to dib? | 14:55 |
openstackgerrit | Lucas Alvares Gomes proposed openstack/instack-undercloud: Ironic: Configure iPXE to work with UEFI https://review.openstack.org/350627 | 14:55 |
rbrady | shardy: I have added blueprints for the remaining work(that I know of) associated with tripleo workflows and actions and linked them to the primary bp: https://blueprints.launchpad.net/tripleo/+spec/mistral-deployment-library | 14:55 |
shardy | larsks: we use the puppet-modules one during image building to include puppet modules in the overcloud images | 14:55 |
shardy | larsks: the hiera one is used on the overcloud nodes to deploy hieradata | 14:56 |
shardy | so, again, it's built into the image via DiB | 14:56 |
*** Ryjedo has joined #tripleo | 14:56 | |
larsks | shardy: where does the image building happen? I am trying to track down an issue with the images used by tripleo-quickstart, and it's not clear to me whether those images use these elements... | 14:57 |
shardy | https://github.com/openstack/python-tripleoclient/blob/master/tripleoclient/v1/overcloud_image.py#L182 | 14:57 |
shardy | larsks: tripleoclient calls DiB when you do openstack overcloud image build | 14:57 |
jtomasek | shardy, rbrady: isn't the plan only something that acts as a name for a set of templates uploaded to swift? I thought we support the usecase shardy specified | 14:57 |
larsks | shardy: thanks. | 14:57 |
shardy | larsks: there is another tripleo-build-images tool too, but we're not using it upstream yet | 14:58 |
jtomasek | shardy, rbrady I see now, so the heat stack name matches plan name... | 14:58 |
rbrady | jtomasek: the plan consists of the swift container and all of its contents and the mistral environment which contains deployment specific data | 14:58 |
jtomasek | is taht what the problem is? | 14:58 |
shardy | jtomasek: the only problem is the plan delete workflow assumes a stack called "overcloud" can only ever be deployed from a container called "overcloud" | 14:59 |
jtomasek | shadower: I see, hmm | 14:59 |
*** pradk_ has joined #tripleo | 14:59 | |
jtomasek | shardy: is 'overcloud' stak name somewhat mandatory from the TripleO deployment standpoint? | 15:00 |
shardy | if we refine that slightly, we'll make it cleaner for folks wishing to do stuff like dev testing or staging workflows switching between different plans | 15:00 |
shardy | jtomasek: No, it's completely configurable | 15:00 |
shardy | Anyway, I didn't mean to make a big issue, just wanted to share some ideas | 15:00 |
jtomasek | shardy: ok, yeah, your point makes sense | 15:01 |
*** fragatina has quit IRC | 15:02 | |
pabelanger | okay, so http://tarballs.openstack.org/tripleo-ci/images/ is a thing now. I have a patch up to start using tarballs.o.o for our cached images | 15:04 |
pabelanger | running in the CI now | 15:04 |
*** paramite has quit IRC | 15:06 | |
pabelanger | larsks: I think you're tripleo-quickstart-core, have a few minutes to review some patches? https://review.openstack.org/#/q/topic:quickstart-linters | 15:09 |
*** akshai has quit IRC | 15:09 | |
larsks | pabelanger: sure. | 15:09 |
openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci: WIP: DONT MERGE TESTING https://review.openstack.org/316436 | 15:09 |
larsks | pabelanger: just need to finish a call first... | 15:09 |
pabelanger | larsks: great, thanks | 15:09 |
*** dprince has quit IRC | 15:10 | |
openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates: Add cinder-backup service as role https://review.openstack.org/304563 | 15:12 |
*** ebarrera has quit IRC | 15:15 | |
openstackgerrit | Emilien Macchi proposed openstack/python-tripleoclient: undercloud/upgrade: yum update of instack dependencies https://review.openstack.org/331804 | 15:15 |
EmilienM | bnemec: I took over the tripleoclient patch, let me know if it's a problem and I can revert my change and do a separacted patch. | 15:15 |
*** julim has joined #tripleo | 15:16 | |
EmilienM | bnemec: I'm doing good progress | 15:16 |
openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci: Retry stack-delete in case of failure https://review.openstack.org/350640 | 15:16 |
openstackgerrit | Paul Belanger proposed openstack/tripleo-quickstart: [WIP] Stop using data_files https://review.openstack.org/350271 | 15:16 |
*** bootsha has quit IRC | 15:17 | |
*** akshai has joined #tripleo | 15:18 | |
openstackgerrit | Martin André proposed openstack/tripleo-validations: Add a dynamic ansible inventory script https://review.openstack.org/348305 | 15:20 |
dtantsur | folks, I'm seeing a strange output of "introspection bulk start", something might be slightly wrong about the new workflows: http://paste.openstack.org/show/547803/ | 15:20 |
* dtantsur is looking whom to ping about it | 15:20 | |
openstackgerrit | Paul Belanger proposed openstack/tripleo-quickstart: [WIP] Stop using data_files https://review.openstack.org/350271 | 15:20 |
*** hewbrocca is now known as hewbrocca-afk | 15:23 | |
openstackgerrit | Paul Belanger proposed openstack/tripleo-quickstart: [WIP] Stop using data_files https://review.openstack.org/350271 | 15:24 |
*** bana_k has joined #tripleo | 15:26 | |
openstackgerrit | Paul Belanger proposed openstack/tripleo-quickstart: [WIP] Stop using data_files https://review.openstack.org/350271 | 15:28 |
openstackgerrit | Carlos Camacho proposed openstack/puppet-tripleo: Remove unused parameter in sahara https://review.openstack.org/350649 | 15:33 |
*** leanderthal is now known as leanderthal|afk | 15:37 | |
panda | how do I access the overcloud nodes if the ssh key is not working ? | 15:39 |
openstackgerrit | Carlos Camacho proposed openstack/puppet-tripleo: Remove unused parameter in sahara https://review.openstack.org/350649 | 15:41 |
*** jpeeler has quit IRC | 15:43 | |
openstackgerrit | Pradeep Kilambi proposed openstack/tripleo-heat-templates: Add Aodh composable roles https://review.openstack.org/333556 | 15:43 |
*** zoliXXL is now known as zoli|gone | 15:45 | |
openstackgerrit | Emilien Macchi proposed openstack/instack-undercloud: Implement pre_upgrade hook when deploying undercloud https://review.openstack.org/350657 | 15:45 |
marios | shardy: do you think you're ready to reinstate your +2 ? https://review.openstack.org/#/c/188137/ green ci and green run at https://review.openstack.org/#/c/342961/ - fyi when/if you have a moment thanks! | 15:50 |
*** zoli|gone is now known as zoli_gone-proxy | 15:50 | |
openstackgerrit | Ryan Brady proposed openstack/tripleo-common: Adds Plan List Action https://review.openstack.org/300221 | 15:50 |
*** aufi has quit IRC | 15:54 | |
EmilienM | bnemec, shardy: I just sent an email to ML about undercloud upgrade CI job status | 15:58 |
*** akshai has quit IRC | 15:59 | |
*** ramishra has quit IRC | 15:59 | |
EmilienM | bnemec, slagle: you disagree on where to run systemctl stop openstack-* | 16:00 |
slagle | EmilienM: i don't like that logic living in tripleoclient | 16:00 |
openstackgerrit | Ryan Brady proposed openstack/tripleo-common: Adds Plan List Action https://review.openstack.org/300221 | 16:00 |
EmilienM | slagle suggested me to move it to instack-undercloud but it looks like bnemec wants to keep it in tripleoclient | 16:00 |
slagle | EmilienM: tripleoclient should not be the source of truth on how to "upgrade an undercloud" | 16:00 |
EmilienM | I agree we should use isntack-undercloud | 16:00 |
bnemec | slagle: Doing it in instack-undercloud means instack-undercloud has to know how to upgrade itself. | 16:01 |
bnemec | Which it can't because it's already running at that point. | 16:01 |
bnemec | Unless we want to play the re-exec game. | 16:01 |
bnemec | Which we do not. | 16:01 |
slagle | bnemec: no.... | 16:01 |
slagle | we still need the yum update i-u in tripleoclient | 16:02 |
slagle | but the systemctl stop openstack* should be in i-u | 16:02 |
slagle | would that work? | 16:02 |
bnemec | slagle: But why? Now we've split the upgrade logic. | 16:02 |
slagle | i don't consider upgrading i-u itself part of the logic to upgrade an undercloud | 16:03 |
slagle | it's a precondition | 16:03 |
slagle | "you need the latest versin of this package installed" | 16:03 |
slagle | that package contains the logic to upgrade the undercloud | 16:03 |
slagle | consider what would happen when we need to systemctl stop something else | 16:04 |
slagle | how would they get the latest tripeloclient to do that if it was in tripleoclient? | 16:04 |
slagle | they would have to have the latest | 16:05 |
slagle | so just make all the "logic" in i-u and have tripleoclient make sure you have the latest of i-u | 16:05 |
*** jistr is now known as jistr|biab | 16:06 | |
EmilienM | I tend to agree with slagle's proposal, and that's what I'm currently implementing | 16:07 |
*** Ryjedo has quit IRC | 16:08 | |
EmilienM | but we need to find a concensus | 16:08 |
bnemec | TBH, I want to go back to PS 2 of my client patch. | 16:08 |
bnemec | Stop all, upgrade all, install undercloud | 16:08 |
openstackgerrit | Emilien Macchi proposed openstack/instack-undercloud: Implement pre_upgrade hook when deploying undercloud https://review.openstack.org/350657 | 16:08 |
EmilienM | bnemec: we CANT upgrade all | 16:09 |
EmilienM | I already explained why... | 16:09 |
EmilienM | because puppet already upgrade all later | 16:09 |
bnemec | EmilienM: Where? | 16:09 |
shardy | marios: -1! | 16:09 |
bnemec | What does that have to do with it? | 16:09 |
EmilienM | but puppet needs to upgrade OpenStack packages first | 16:09 |
shardy | only joking, approved :) | 16:09 |
EmilienM | so it can trigger things in catalog | 16:10 |
EmilienM | like dbsync at the right time, etc | 16:10 |
*** Ryjedo has joined #tripleo | 16:10 | |
EmilienM | if you run yum update -y before puppet run, I suspect some future issues with Puppet catalog | 16:10 |
EmilienM | bnemec: but we can try it, if you think it's better | 16:10 |
EmilienM | the question is, where? in tripleoclient? | 16:11 |
bnemec | The db sync is fixed, and we need to sanely handle packages being updated outside puppet anyway. | 16:12 |
bnemec | Because people can and will do that. | 16:12 |
EmilienM | ok, let's say we update all before | 16:12 |
EmilienM | I'm ok with it | 16:12 |
EmilienM | I just need a concensus on where to do it | 16:12 |
EmilienM | i-u or tc | 16:12 |
*** saneax is now known as saneax_AFK | 16:12 | |
bnemec | This would have to be the client. | 16:13 |
marios | shardy: /me will be telling his grandkids about the day manila landed | 16:13 |
slagle | so hardcoding a list of service names in the client? | 16:13 |
openstackgerrit | Paul Belanger proposed openstack/tripleo-quickstart: [WIP] Stop using data_files https://review.openstack.org/350271 | 16:13 |
bnemec | lol | 16:13 |
marios | shardy: thanks! | 16:13 |
EmilienM | marios: lol | 16:13 |
bnemec | slagle: No, all of openstack. Plus neutron because they're a grandfathered in snowflake. | 16:13 |
openstackgerrit | OpenStack Proposal Bot proposed openstack/os-cloud-config: Updated from global requirements https://review.openstack.org/350676 | 16:14 |
EmilienM | we can start by this proposal, i'm fine with it | 16:14 |
EmilienM | we need to start somewhere anyway | 16:14 |
slagle | bnemec: and rabbit? haproxy? httpd? :) | 16:14 |
EmilienM | slagle: puppet should take care of it later | 16:14 |
slagle | mariadb? | 16:14 |
EmilienM | bnemec: did you figure WHY we need those "stop" ? | 16:14 |
EmilienM | why Puppet catalog hangs? | 16:14 |
bnemec | EmilienM: It's because the OpenStack service conf files aren't forward compatible. | 16:14 |
*** Ryjedo has quit IRC | 16:15 | |
EmilienM | slagle: no mariadb is the same in mitaka & newton | 16:15 |
EmilienM | slagle: 10.x | 16:15 |
bnemec | slagle: Shit, that's a good point. We could hit the same problem in another service. :-( | 16:15 |
openstackgerrit | OpenStack Proposal Bot proposed openstack/python-tripleoclient: Updated from global requirements https://review.openstack.org/350678 | 16:15 |
*** dtantsur is now known as dtantsur|afk | 16:15 | |
EmilienM | we can start by openstack-only services now.. | 16:16 |
openstackgerrit | Merged openstack/tripleo-heat-templates: Enable Manila integration - as a composable controller service https://review.openstack.org/188137 | 16:16 |
openstackgerrit | OpenStack Proposal Bot proposed openstack/tripleo-common: Updated from global requirements https://review.openstack.org/348695 | 16:16 |
EmilienM | bnemec, slagle: or better, I could even solve it in Puppet catalog | 16:17 |
EmilienM | in the same catalog, stop all services before anything else in catalog | 16:17 |
*** Goneri has quit IRC | 16:18 | |
EmilienM | but we have to consider that everytime a user would run puppet on the undercloud, services would stop/start | 16:18 |
EmilienM | is it fine? | 16:18 |
snecklifter | rhefner: hi, I've sort of hijacked https://review.openstack.org/#/c/313933/ | 16:18 |
bnemec | EmilienM: I think that's fine. The services will start and stop anyway, right? | 16:18 |
EmilienM | yeah | 16:19 |
EmilienM | bnemec, slagle: in my proposal, we keep https://review.openstack.org/#/c/331804/ and I need to patch undercloud puppet manifest to stop services before. Is it fine? | 16:19 |
bnemec | Yeah, I'm convinced that this is right now. | 16:20 |
*** anshul has quit IRC | 16:20 | |
EmilienM | slagle: ? | 16:21 |
rhefner | snecklifter: hi, I saw some of your commits - thanks! I was pulled off of that for a little while and it's on my plate to resume later this week | 16:21 |
*** jistr|biab is now known as jistr | 16:22 | |
snecklifter | rhefner: ah ok, looks like latest build has stalled but hopefully its not far off :) | 16:22 |
rhefner | snecklifter: perfect! | 16:23 |
*** devvesa has quit IRC | 16:23 | |
*** Ryjedo has joined #tripleo | 16:23 | |
*** Ryjedo has quit IRC | 16:23 | |
openstackgerrit | Carlos Camacho proposed openstack/puppet-tripleo: Implement aodh profiles https://review.openstack.org/332854 | 16:24 |
openstackgerrit | Carlos Camacho proposed openstack/tripleo-heat-templates: Add Aodh composable roles https://review.openstack.org/333556 | 16:26 |
*** ccamacho is now known as ccamacho|out | 16:26 | |
*** fultonj has quit IRC | 16:26 | |
*** Ryjedo has joined #tripleo | 16:27 | |
*** tremble has quit IRC | 16:27 | |
openstackgerrit | Carlos Camacho proposed openstack/puppet-tripleo: Fix mistral profile steps https://review.openstack.org/343346 | 16:28 |
*** fultonj has joined #tripleo | 16:28 | |
*** ohamada has quit IRC | 16:29 | |
slagle | EmilienM: i really don't like that | 16:29 |
jpich | shardy, rbrady: Once I have a better understanding of how it needs to change I'm happy to try and give a hand with iterating over the plan deletion etc actions, if that helps. I just created a couple of patches to call these from the client today so I have a moderate understanding of what they do now, if nothing else! Just let me know. | 16:29 |
slagle | why the reinstall? | 16:29 |
slagle | EmilienM: we keep making everything slower and slower | 16:29 |
slagle | EmilienM: i probably need to just review the spec | 16:29 |
shardy | jpich: thanks! I've taken the bug for now, but will happily pass it on if you've got bandwidth to do it | 16:29 |
*** akshai has joined #tripleo | 16:30 | |
shardy | jpich: lets get the basic workflow landed, then we can iterate on it | 16:30 |
EmilienM | slagle: yes | 16:30 |
bnemec | slagle: The spec probably needs an update now though. | 16:30 |
slagle | EmilienM: that's going to probably be 100+ packages to reinstall | 16:30 |
*** jpena is now known as jpena|off | 16:30 | |
slagle | definitely not a fan of that | 16:30 |
bnemec | It was written based on my assumption that we could do a mass update, which isn't true. | 16:30 |
EmilienM | slagle: why 100+? where do you see that? | 16:30 |
slagle | EmilienM: on an uc, how many get removed when you run that yum remove? | 16:31 |
slagle | i won't guess, you can tell me :) | 16:31 |
EmilienM | slagle: the tripleo packages? | 16:31 |
EmilienM | os-, tripleo-* etc , | 16:31 |
EmilienM | ? | 16:31 |
EmilienM | around 20 | 16:31 |
EmilienM | I have logs | 16:31 |
EmilienM | one sec | 16:31 |
openstackgerrit | Merged openstack/tripleo-quickstart: Add minimal_pacemaker config https://review.openstack.org/350536 | 16:31 |
*** oshvartz has quit IRC | 16:31 | |
bnemec | EmilienM: I'm also concerned that we aren't properly updating the dependencies doing it this way. | 16:32 |
bnemec | I left a comment on the client review. | 16:32 |
*** akshai_ has joined #tripleo | 16:32 | |
EmilienM | slagle: ok more but really fast http://logs.openstack.org/95/346995/24/experimental/gate-tripleo-ci-centos-7-undercloud-upgrades-nv/5276290/console.html#_2016-08-02_22_16_50_454176 | 16:32 |
*** tremble has joined #tripleo | 16:32 | |
slagle | sure, in CI with a local mirror | 16:32 |
slagle | i'll just review the patch | 16:32 |
EmilienM | slagle: less than ONE minute to remove/install tripleo deps | 16:33 |
shardy | weren't we talking about just stopping services, not reinstalling packages? | 16:33 |
*** dprince has joined #tripleo | 16:33 | |
EmilienM | I'm about to drop off for lunch, feel free to review and give insights. we need to find concensus here | 16:33 |
*** snecklifter has quit IRC | 16:33 | |
*** Ryjedo has quit IRC | 16:34 | |
slagle | EmilienM: which is why we have a spec right? | 16:34 |
jpich | shardy: Sure! Subscribed to https://bugs.launchpad.net/tripleo/+bug/1609454 , I'll keep an eye on it | 16:34 |
openstack | Launchpad bug 1609454 in tripleo "Workflows Assuming a Single Plan is Associated with a Single Stack" [Medium,Triaged] - Assigned to Steven Hardy (shardy) | 16:34 |
slagle | EmilienM: we might consider letting that play out. | 16:34 |
EmilienM | sure, I'll let the spec make progress | 16:35 |
* EmilienM afk | 16:35 | |
*** akshai has quit IRC | 16:35 | |
openstackgerrit | Carlos Camacho proposed openstack/tripleo-heat-templates: Composable Mistral services https://review.openstack.org/323436 | 16:36 |
*** rcernin has quit IRC | 16:38 | |
*** dsariel has quit IRC | 16:39 | |
*** ramishra has joined #tripleo | 16:43 | |
*** lucasagomes is now known as lucas-dinner | 16:45 | |
*** devvesa has joined #tripleo | 16:46 | |
*** devvesa has quit IRC | 16:47 | |
openstackgerrit | Steven Hardy proposed openstack/puppet-tripleo: Remove keystone PKI cert generation https://review.openstack.org/350554 | 16:50 |
*** Ryjedo has joined #tripleo | 16:50 | |
*** aviramb has quit IRC | 16:52 | |
*** oshvartz has joined #tripleo | 16:52 | |
shardy | d0ugal: Hey, are you looking at adding support for user environments to the deploy workflow? | 16:52 |
*** krotscheck is now known as kro_focused | 16:53 | |
shardy | d0ugal: I've hit a barrier with testing because .tripleo/environments/puppet-modules-url.yaml no longer works | 16:53 |
*** Ryjedo has quit IRC | 16:55 | |
d0ugal | shardy: I am trying, but not having much luck figuring out how to fit it in | 16:57 |
*** Ryjedo has joined #tripleo | 16:58 | |
d0ugal | rbrady: Had you thought how users would provide extra environment files? | 16:58 |
shardy | seems like we either need to push them to the swift container, then reference them in the environment list defined in the mistral environment, or put the whole content in the environment | 16:59 |
shardy | are there limits on the size of mistral environments? | 16:59 |
d0ugal | shardy: Not that I can see, it is just a text field. | 16:59 |
*** pcaruana has quit IRC | 17:00 | |
shardy | it still might be cleaner to push them to swift I guess, then all template stuff is in one place | 17:00 |
d0ugal | Yeah, I think so | 17:00 |
d0ugal | we really need all the crud operations for plans | 17:00 |
shardy | ProcessTemplatesAction looks like it will do the right thing if we add the swift urls to the environment "environments" list | 17:01 |
d0ugal | shardy: right, so add it to swift and then add it to the list of envs to be processed | 17:01 |
shardy | I'm not sure how we differentiate between environments that are already in the plan data, vs ones that are defined elsewhere | 17:02 |
shardy | maybe just compare the dirname | 17:02 |
d0ugal | shardy: yeah, it gets tricky as users at the moment can give us a path to an environment anywhere | 17:02 |
d0ugal | we might need to do something to give them a unique name | 17:03 |
shardy | openstack overcloud deploy --templates /tmp/tht/ -e /tmp/tht/environments/enable_foo.yaml -e ./override_foo.yaml | 17:03 |
*** jbadiapa has quit IRC | 17:03 | |
*** tesseract- has quit IRC | 17:03 | |
shardy | d0ugal: probably just creating a "user_environments" subdir in the container would be enough? | 17:03 |
d0ugal | shardy: if they are both called enable_foo.yaml it becomes more fun :) | 17:03 |
d0ugal | user_environments/enable_foo.yaml? | 17:04 |
shardy | d0ugal: the environment path needs to be fully qualified and match either a swift URL or something in the files map | 17:04 |
shardy | d0ugal: Yeah, it'd be something like push user_environments/enable_foo.yaml | 17:04 |
shardy | to swift | 17:04 |
d0ugal | Why do I feel like this is going to be the source of weird edge cases :) | 17:05 |
shardy | then add path: http:<swifturl prefix>/user_environments/enable_foo.yaml to the mistral env | 17:05 |
*** openstackgerrit_ has joined #tripleo | 17:06 | |
shardy | d0ugal: actually, do mistral environment-get overcloud | 17:07 |
shardy | there are no prefixes | 17:07 |
shardy | so they must be added elsewhere, because we do download overcloud-resource-registry-puppet.yaml from swift (ref bug breaking deployment yesterday) | 17:07 |
*** openstackgerrit_ has quit IRC | 17:08 | |
shardy | ah, yeah, we join them in the ProcessTemplatesAction | 17:08 |
d0ugal | right | 17:08 |
d0ugal | shardy: I'm going to have to call it for today. | 17:10 |
d0ugal | I think I can get something together quickly for tomorrow | 17:10 |
shardy | d0ugal: WHAT? | 17:10 |
shardy | Only joking, have a great evening :) | 17:10 |
* shardy needs to do likewise soon | 17:10 | |
d0ugal | shardy: I am not a force of nature like you :) | 17:10 |
*** mgould is now known as mgould|afk | 17:11 | |
*** akshai_ has quit IRC | 17:11 | |
*** Ryjedo has quit IRC | 17:12 | |
*** yamahata has quit IRC | 17:15 | |
*** Ryjedo has joined #tripleo | 17:22 | |
openstackgerrit | Pradeep Kilambi proposed openstack/tripleo-heat-templates: Set Coordination url for central agent https://review.openstack.org/350702 | 17:25 |
*** bootsha has joined #tripleo | 17:26 | |
openstackgerrit | Paul Belanger proposed openstack/tripleo-quickstart: [WIP] Stop using data_files https://review.openstack.org/350271 | 17:27 |
*** Ryjedo has quit IRC | 17:27 | |
*** fzdarsky is now known as fzdarsky|afk | 17:32 | |
*** abehl has joined #tripleo | 17:33 | |
* bnemec has a fruit fly problem in his office | 17:33 | |
bnemec | Which I don't understand. I'm very careful not to leave open food lying around for exactly that reason. | 17:34 |
*** abehl has quit IRC | 17:34 | |
openstackgerrit | Merged openstack/tripleo-quickstart: Fix E226 missing whitespace around arithmetic operator https://review.openstack.org/348999 | 17:34 |
openstackgerrit | Merged openstack/tripleo-quickstart: Use ansible 2.1 importing of boilerplate code https://review.openstack.org/349000 | 17:35 |
openstackgerrit | Merged openstack/tripleo-quickstart: Fix H102 Apache 2.0 license header not found https://review.openstack.org/349001 | 17:35 |
openstackgerrit | Paul Belanger proposed openstack/tripleo-quickstart: [WIP] Stop using data_files https://review.openstack.org/350271 | 17:35 |
openstackgerrit | James Slagle proposed openstack/tripleo-docs: Document additional NODE vars https://review.openstack.org/350708 | 17:40 |
*** shardy has quit IRC | 17:44 | |
*** bana_k has quit IRC | 17:45 | |
openstackgerrit | Athlan-Guyot sofer proposed openstack/tripleo-quickstart: Fix wrong naming of image's name in release. https://review.openstack.org/350712 | 17:46 |
rbrady | shardy, d0ugal: I think one thing to think about is how environments are chosen (UpdateCapabilities). We need to update that code to ensure it provides any additional envs from either swift or stored in the mistral environment.variables['environments'] list | 17:48 |
rbrady | d0ugal: I had been thinking of adding an action to support adding environment data to a plan, but it has not been requested as a feature that I know of yet | 17:49 |
openstackgerrit | James Slagle proposed openstack/tripleo-docs: Document additional NODE vars https://review.openstack.org/350708 | 17:49 |
EmilienM | slagle: what is your suggestion about https://review.openstack.org/#/c/331804/4/tripleoclient/v1/undercloud.py ? I'm not sure to understand | 17:52 |
*** milan has quit IRC | 17:53 | |
*** sarath has joined #tripleo | 17:53 | |
social | EmilienM: why it's part of tripleo client? | 17:53 |
*** yamahata has joined #tripleo | 17:53 | |
*** julim has quit IRC | 17:53 | |
slagle | here we go :) | 17:54 |
EmilienM | social: that's what we are discussing right now | 17:54 |
EmilienM | slagle: so should we remove the UpgradeUndercloud function? | 17:54 |
sarath | dprince, bnemec: could you kindly review the os-net-config PR at https://review.openstack.org/#/c/345599/ | 17:54 |
EmilienM | err, class | 17:54 |
slagle | EmilienM: i'd like to know why | 17:54 |
slagle | EmilienM: i asked in the review | 17:54 |
slagle | why the remove/install | 17:54 |
slagle | i admit i probably do not fully understand all the issues, as I've not yet read the spec | 17:55 |
slagle | however, my initial thinking is to upgrade the packages that provide whatever code would already be running/loaded when instack-install-undercloud is executed | 17:55 |
slagle | essentially, what i-u can't upgrade itself w/o re-exec'ing | 17:56 |
social | EmilienM: shouldn't it at max take care of installing/upgrading instack-undercloud and puppet,opm&deps upgrade should be present in instack-undercloud? | 17:56 |
slagle | and then let i-u upgrade everything else | 17:56 |
social | EmilienM: and rest update in puppet run | 17:56 |
*** julim has joined #tripleo | 17:56 | |
slagle | EmilienM: and i-u can upgrade "everything else" however it wants. via puppet, etc | 17:57 |
*** bana_k has joined #tripleo | 17:58 | |
slagle | now...the other use case that we've said we want to support is if someone externally runs "yum update", and then rerun openstack undercloud install | 17:59 |
slagle | it should work | 17:59 |
slagle | i do not thing we necessarily have to immediately solve both of those in the same patch | 17:59 |
slagle | *think | 17:59 |
EmilienM | I have to admit I'm really confused right now | 18:00 |
social | slagle: you don't resolve external yum update that's all | 18:00 |
slagle | i dont understand what you're saying | 18:02 |
slagle | it doesn't work now? it will never work? | 18:03 |
openstackgerrit | Ben Nemec proposed openstack/tripleo-specs: Undercloud Upgrade https://review.openstack.org/349737 | 18:05 |
*** tosky has quit IRC | 18:09 | |
*** akrivoka has quit IRC | 18:10 | |
openstackgerrit | Julie Pichon proposed openstack/python-tripleoclient: Add 'openstack overcloud plan list' command https://review.openstack.org/350498 | 18:10 |
openstackgerrit | Julie Pichon proposed openstack/python-tripleoclient: Add 'openstack overcloud plan delete' command https://review.openstack.org/350617 | 18:10 |
*** jdob has quit IRC | 18:11 | |
*** jdob has joined #tripleo | 18:11 | |
openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: WIP - Implement undercloud upgrade job - Mitaka -> Newton https://review.openstack.org/346995 | 18:14 |
EmilienM | bnemec, slagle: ok: I pushed over ^ to come back to my PoC without tripleoclient or i-u changes. | 18:14 |
EmilienM | so in the code, we know what we need to and decide where to put it later | 18:14 |
*** Ryjedo has joined #tripleo | 18:15 | |
*** yamahata has quit IRC | 18:18 | |
*** yamahata has joined #tripleo | 18:19 | |
*** mcornea has quit IRC | 18:19 | |
*** jdob has quit IRC | 18:20 | |
*** fzdarsky|afk is now known as fzdarsky | 18:20 | |
*** jdob has joined #tripleo | 18:20 | |
*** bootsha has quit IRC | 18:22 | |
*** jpich has quit IRC | 18:22 | |
*** electrofelix has quit IRC | 18:22 | |
*** Ryjedo has quit IRC | 18:23 | |
EmilienM | http://logs.openstack.org/50/347050/3/check/gate-tripleo-ci-centos-7-nonha-multinode-nv/00b3455/console.html#_2016-08-03_13_52_19_305958 | 18:25 |
EmilienM | Device "eth1" does not exist. | 18:25 |
EmilienM | on internap cloud | 18:25 |
EmilienM | https://review.openstack.org/#/c/347050/ | 18:25 |
EmilienM | slagle: do we still have this bug in master CI? I thought it was fixed in instack-undercloud | 18:25 |
*** bootsha has joined #tripleo | 18:26 | |
ccamacho|out | EmilienM, slagle mind to review this? https://review.openstack.org/#/c/350649 its about to remove an unused param, a harmless submission. | 18:27 |
*** rwsu has quit IRC | 18:27 | |
*** oshvartz has quit IRC | 18:28 | |
EmilienM | ccamacho|out: +2 | 18:28 |
ccamacho|out | thanks! | 18:29 |
*** jaosorior has joined #tripleo | 18:29 | |
*** jaosorior has quit IRC | 18:30 | |
openstackgerrit | Dan Sneddon proposed openstack/tripleo-specs: Spec: undercloud-ntp-server - Add NTP services to Undercloud https://review.openstack.org/350729 | 18:31 |
EmilienM | I see the eth1 issue on ovh too | 18:32 |
gfidente | ccamacho|out, sounds like we don't enable sahara by default | 18:32 |
gfidente | is this intended? | 18:32 |
gfidente | thrash, do you know anything about ^^ ? | 18:37 |
gfidente | we had in the default setup in mitaka | 18:37 |
ccamacho|out | gfidente, not sure (::sahara::service::api::enable its defaulted to true)but from what I remember they were all enabled by default, which I | 18:37 |
gfidente | ccamacho|out, oh I meant we don't enable the role on the controllers | 18:37 |
ccamacho|out | think at the end we should provide a naked overcloud | 18:38 |
openstackgerrit | Merged openstack-infra/tripleo-ci: Add opstools candidate repo https://review.openstack.org/325095 | 18:38 |
*** ayoung has quit IRC | 18:38 | |
*** ccamacho|out is now known as ccamacho | 18:39 | |
ccamacho | gfidente, you are right there.. its not enabled | 18:39 |
ccamacho | mmmm need to get the submission to see if it was tested. | 18:39 |
ccamacho | ill take notes | 18:39 |
rook | EmilienM re the glance-api patch for # of workers. Leaving a empty string will make glance-api use the default # of cores. | 18:41 |
rook | EmilienM: is THT doing something eles if workers= ? | 18:42 |
*** oshvartz has joined #tripleo | 18:42 | |
ccamacho | gfidente, https://review.openstack.org/#/c/318840 nope it wasnt enabled, /me taking notes and Ill test it tomorrow | 18:42 |
EmilienM | rook: look the logs, you'll see workers = 1 | 18:43 |
ccamacho | good catch | 18:43 |
thrash | gfidente: it's supposed to be | 18:43 |
EmilienM | rook: it sounds like it take the default in puppet-glance | 18:43 |
EmilienM | pradk: hey about https://review.openstack.org/#/c/350702/1/puppet/services/pacemaker/ceilometer-agent-central.yaml are you sure about the format? | 18:43 |
thrash | gfidente: I thought we had a patch to fix that tho | 18:43 |
EmilienM | redis://password@host:port ? | 18:43 |
thrash | gfidente: https://review.openstack.org/#/c/336119/15 | 18:44 |
gfidente | thrash, oh right I remember that | 18:44 |
gfidente | so it needs rebase and such | 18:45 |
thrash | gfidente: yep | 18:45 |
thrash | I'll work on that. | 18:45 |
ccamacho | thrash++ nice, Ill add it to my queue for tomorrow then | 18:45 |
pradk | EmilienM, yes format is correct.. my local install in progress.. i can confirm if coordination works once thats done | 18:45 |
rook | EmilienM: i see nothing in puppet-glance that would set it to one, unless the # of vCPUs is 1 | 18:46 |
*** julim has quit IRC | 18:46 | |
rook | EmilienM puppet-glance defaults to # of cores. | 18:46 |
gfidente | tty tomorrow guys | 18:46 |
rook | just like every other service. | 18:47 |
rook | or most other services* | 18:47 |
EmilienM | pradk: ack | 18:47 |
EmilienM | rook: the default value in puppet-glance is the number of processors | 18:47 |
EmilienM | rook: right ! | 18:47 |
EmilienM | so I guess it takes it | 18:47 |
rook | EmilienM: does our CI deploy things with > 1vCPU? if so, then you are right, this patch isn't good. | 18:48 |
rook | however, if they deploy with 1vcpu then we are fine. | 18:48 |
*** bootsha has quit IRC | 18:48 | |
*** bootsha has joined #tripleo | 18:49 | |
openstackgerrit | Brad P. Crochet proposed openstack/tripleo-heat-templates: Split Sahara pacemaker roles into separate services https://review.openstack.org/327722 | 18:50 |
openstackgerrit | Brad P. Crochet proposed openstack/tripleo-heat-templates: Add Sahara services to ControllerServices list https://review.openstack.org/336119 | 18:50 |
EmilienM | rook: /me reading logs | 18:55 |
EmilienM | rook: I confirm, 1 | 18:55 |
*** cwolferh has quit IRC | 18:56 | |
sarath | dsneddon: os-net-config IVS cleanup patch as you had requested: https://review.openstack.org/#/c/346161/ | 18:57 |
openstackgerrit | Merged openstack/instack-undercloud: Use certmonger for automatic cert generation https://review.openstack.org/332774 | 18:57 |
rook | EmilienM: looking at the nova logs, it they only started a single worker too | 18:57 |
rook | ok | 18:57 |
*** paramite has joined #tripleo | 18:59 | |
gfidente | thrash, I think the dependency goes in the other way around no? | 19:01 |
gfidente | it's https://review.openstack.org/#/c/336119/16 which depends on https://review.openstack.org/#/c/327722/11 | 19:02 |
openstackgerrit | Merged openstack/tripleo-heat-templates: We don't need to set a default for the CephX keys and cluster FSID https://review.openstack.org/348900 | 19:02 |
*** cwolferh has joined #tripleo | 19:02 | |
thrash | gfidente: huh? | 19:04 |
thrash | gfidente: nope | 19:04 |
gfidente | well we only enable sahara in https://review.openstack.org/#/c/336119/16 | 19:04 |
gfidente | but we deploy it in pacemaker | 19:05 |
thrash | gfidente: 336119 has some bug fixes | 19:05 |
thrash | gfidente: are you mixing up the puppet submission with the THT submission for the split? | 19:05 |
gfidente | yeah but fixes for the non-pacemaker scenario we don't test | 19:05 |
thrash | gfidente: I don't follow... | 19:06 |
gfidente | no my point is that https://review.openstack.org/#/c/336119/16 will enable sahara, but gate is doing pacemaker scenario, so it'll try to run sahara without the fix in https://review.openstack.org/#/c/327722/11 | 19:06 |
gfidente | and it will probably fail because of that | 19:06 |
*** fultonj has quit IRC | 19:08 | |
thrash | we can squash them. | 19:08 |
thrash | gfidente: if that helps. :) | 19:08 |
gfidente | squash or invert the order | 19:09 |
thrash | gfidente: I know I can't invert the order. 327722 will not work without the fixes in 336119 | 19:09 |
gfidente | at which point https://review.openstack.org/#/c/327722/11 will always pass because we don't test it | 19:09 |
thrash | which is also why we inverted it in the first place. | 19:10 |
thrash | didn't want to merge something that we knew wasn't tested. | 19:10 |
gfidente | right but it'll be tested by https://review.openstack.org/#/c/336119/16 | 19:10 |
thrash | gfidente: I'll just squash it. It will make it easier. | 19:12 |
openstackgerrit | Brad P. Crochet proposed openstack/tripleo-heat-templates: Add Sahara services to ControllerServices list https://review.openstack.org/336119 | 19:18 |
thrash | gfidente: there you go. | 19:18 |
gfidente | ack | 19:18 |
gfidente | let's merge it :) | 19:18 |
thrash | haha | 19:18 |
*** tbarron has joined #tripleo | 19:20 | |
pabelanger | bnemec: EmilienM: slagle: tripleo-test-cloud-rh1 online now, jobs stopping on tripleo-test-cloud-rh2. 43 jobs currently coming online | 19:20 |
*** radek_ has joined #tripleo | 19:21 | |
EmilienM | woot | 19:21 |
*** maeca1 has joined #tripleo | 19:28 | |
bswartz | http://paste.openstack.org/show/547837/ | 19:28 |
bswartz | ^ I'm trying to run tripleo-quickstart and it's failing with permission denied creating a directory | 19:28 |
tbarron | ^^ marios if you are there we're trying to get off the ground for the netapp/manila plugin now that https://review.openstack.org/#/c/188137/ merged (w00t!) | 19:29 |
*** tesseract- has joined #tripleo | 19:31 | |
rook | EmilienM: so we should be good to push that lil patch | 19:34 |
*** jcoufal has quit IRC | 19:34 | |
pabelanger | slagle: EmilienM: bnemec: still waiting for new nodes to launch in tripleo-test-cloud-rh1. Been 20mins in building start, anybody able to see what is going on ? | 19:36 |
*** tesseract- has quit IRC | 19:37 | |
bnemec | pabelanger: I see environments spinning up. A lot of slaves in the BUILD status. | 19:38 |
pabelanger | bnemec: 36, been about 24mins now | 19:38 |
*** dsneddon has quit IRC | 19:39 | |
*** bootsha has quit IRC | 19:39 | |
bnemec | pabelanger: Yeah, it seems a bit slow. It's possible the controller is freaking out a bit about trying to spin up so much at once. | 19:39 |
*** dsneddon has joined #tripleo | 19:39 | |
*** rwsu has joined #tripleo | 19:40 | |
pabelanger | okay, we'll see what happens | 19:40 |
bnemec | pabelanger: I'm seeing quite a bit of glance and swift activity on the controller. We may be bottlenecked getting the images cached on all the compute nodes. | 19:42 |
pabelanger | bnemec: ya, that is likely the issue. Our image is 5GB | 19:43 |
openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates: Add cinder-backup service as role https://review.openstack.org/304563 | 19:44 |
*** ayoung has joined #tripleo | 19:45 | |
openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: Add undercloud job in cistatus https://review.openstack.org/350763 | 19:50 |
*** Goneri has joined #tripleo | 19:51 | |
dsneddon | sarath, Thank you very much! | 19:58 |
*** dprince has quit IRC | 19:58 | |
*** maeca2 has joined #tripleo | 20:01 | |
pabelanger | metrics_to_graphite "23.253.94.71" #Dan's temp graphite server | 20:01 |
pabelanger | Who is Dan in that comment? | 20:01 |
pabelanger | that is from tripleo-ci? | 20:01 |
pabelanger | dprince I suspect | 20:01 |
*** maeca1 has quit IRC | 20:04 | |
bnemec | pabelanger: Yeah, dprince | 20:06 |
bnemec | I'm seeing a bunch of 404's on the bmc and ipxe images in the rh1 glance logs. | 20:06 |
bnemec | Which I don't understand because some of the envs stood up fine. | 20:06 |
d0ugal | rbrady: every CLI user provides environments when deploying I think - so it should be requested :) | 20:08 |
pabelanger | bnemec: thanks, I'll ping him tomorrow about it. I'd like to understand what its use-case is and how to replace it for something in openstack-infra | 20:08 |
bnemec | pabelanger: It's just for tracking how long various parts of the CI deployments take over time. If there's an infra place we can do that instead I think everyone would be fine with it. | 20:09 |
*** Goneri has quit IRC | 20:10 | |
pabelanger | bnemec: right, talking in openstack-infra about it, it seems subunit2sql with stackviz could provide the same info | 20:10 |
*** coolsvap has quit IRC | 20:11 | |
slagle | pabelanger: is there a concern with keeping rh2 in nodepool longer? | 20:17 |
EmilienM | pabelanger: we use it in puppet CI | 20:17 |
slagle | pabelanger: i don't feel that the pressure to get it removed is coming from the tripleo side | 20:17 |
bnemec | Hey, the rh1 slaves have come up. | 20:17 |
EmilienM | pabelanger: http://logs.openstack.org/65/349765/3/check/gate-puppet-openstack-integration-3-scenario002-tempest-centos-7/732ec15/logs/stackviz/#/testrepository.subunit/timeline | 20:17 |
slagle | pabelanger: we have the rack for as long as we reasonably need accourding to weshay | 20:18 |
weshay | I'm a nice guy.. what can I say ;) | 20:19 |
rbrady | d0ugal: do you want a register a blueprint for it and document in the summary how the CLI provides this to the operator so we can duplicate it with an action or workflow? | 20:20 |
pabelanger | slagle: that is a question for derekh, as I understood it the hardware was getting reclaimed | 20:20 |
pabelanger | 31 nodes in tripleo-test-cloud-rh1 ATM | 20:21 |
*** tremble has quit IRC | 20:21 | |
weshay | pabelanger, slagle I suspect when the rdocloud comes up we can get it back | 20:21 |
pabelanger | EmilienM: right, tripleo-ci should be using that too over graphite | 20:21 |
d0ugal | rbrady: I can look into doing that tomorrow, but if you want it now, just look at the -e arg for `openstack overcloud deploy` | 20:21 |
pabelanger | weshay: I can ping ggilies about that tonight | 20:22 |
slagle | ok. well i guess if rh1 is coming up fine, we can keep it shut off | 20:22 |
slagle | but i think we'll need a couple of days with things running at normal capacity to confirm that | 20:23 |
EmilienM | and since couple of days means Friday, I suggest to wait Monday. | 20:23 |
*** sarath has quit IRC | 20:23 | |
pabelanger | yes we can do that | 20:24 |
openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart: add temp new release to verify centos7-new https://review.openstack.org/350772 | 20:24 |
pabelanger | slagle: infact, all jobs in check-tripleo are running right now | 20:24 |
EmilienM | pabelanger: I have a new experimental job request: https://review.openstack.org/#/c/350580/ | 20:25 |
*** bootsha has joined #tripleo | 20:28 | |
openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates: Import CI THT environments from tripleo-ci https://review.openstack.org/350775 | 20:28 |
openstackgerrit | James Slagle proposed openstack/tripleo-docs: Add all puppet modules to includepkgs https://review.openstack.org/350776 | 20:32 |
openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates: Import CI THT environments from tripleo-ci https://review.openstack.org/350775 | 20:33 |
openstackgerrit | Thomas Herve proposed openstack/tripleo-common: Ignore changes to images in nova servers https://review.openstack.org/350778 | 20:36 |
pabelanger | how is overcloud-full.qcow2 build today? | 20:36 |
pabelanger | actually, where does the build happen | 20:37 |
openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: Remove test-environments from tripleo-ci https://review.openstack.org/350779 | 20:37 |
slagle | pabelanger: it's built on the undercloud | 20:38 |
EmilienM | bnemec, slagle: you're probably going to -2 me but this is a start: https://review.openstack.org/#/q/topic:ci/envs | 20:38 |
*** jayg is now known as jayg|g0n3 | 20:38 | |
EmilienM | feedback is welcome | 20:38 |
slagle | pabelanger: during the CI job. if we determine we can't use the cache | 20:38 |
*** paramite has quit IRC | 20:38 | |
slagle | i think the periodic job always builds as well | 20:38 |
bnemec | Actually they're all building images right now because of the switch over from source puppet to packaged puppet. | 20:40 |
bnemec | If we updated the cache since that change was made we can try undoing that though. | 20:41 |
openstackgerrit | Merged openstack/puppet-tripleo: Remove unused parameter in sahara https://review.openstack.org/350649 | 20:43 |
ayoung | bnemec, slagle I'm trying to do SSL as per http://docs.openstack.org/developer/tripleo-docs/advanced_deployment/ssl.html and something seems to be missing, but I can fix it by hand. However, it leaves the overcloud deploy in a 1/2 done state. How do I say "continue deploying from where it was broken?" | 20:43 |
slagle | i dont think you can, for various reasons | 20:44 |
slagle | it's not going to do the post deployment endpoint creation if it's already failed | 20:45 |
slagle | and it's also likely to blow away your manual fix, unless you made that fix on the template side | 20:45 |
bnemec | Actually, there's a CLI switch to force postconfig now: https://github.com/openstack/python-tripleoclient/blob/master/tripleoclient/v1/overcloud_deploy.py#L878 | 20:45 |
slagle | i'll be a hot dog | 20:46 |
bnemec | I wanted it on by default. | 20:46 |
bnemec | But I got shouted down as usual. | 20:46 |
bnemec | Not that I'm bitter or anything. :-P | 20:47 |
openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart: add temp new release to verify centos7-new https://review.openstack.org/350772 | 20:48 |
slagle | where is the patch to move all this to puppet anyway? | 20:49 |
EmilienM | can we enable again the tripleo upgrade job? | 20:49 |
EmilienM | now we have more nodes | 20:49 |
bnemec | I'm betting it doesn't work. | 20:49 |
bnemec | net-iso is broken | 20:49 |
bnemec | *sigh* | 20:49 |
ayoung | slagle, so, what is happening is that the cert is not getting added to the list of trusted certs on the controller. I can add that by hand, and I would not expect anything to undo that | 20:49 |
bnemec | And I think net-iso is used in the upgrade job. | 20:50 |
EmilienM | bnemec: why is it broken? what is broken? | 20:50 |
bnemec | ayoung: You should not have to do that. | 20:50 |
ayoung | should I just re-run openstack overcloud deploy | 20:50 |
slagle | ayoung: ok | 20:50 |
ayoung | bnemec, I know, but Juan is not back until morning | 20:50 |
bnemec | EmilienM: I don't know, but Ceph refuses to start on my net-iso re-enablement patch. | 20:50 |
ayoung | and he and I will work through it then. I think the rest of the deploy is OK, would like to be able to keep working on it | 20:50 |
slagle | ayoung: yes, but you need to pass --force-postconfig as well | 20:50 |
ayoung | slagle, ++ | 20:50 |
slagle | when you rerun | 20:51 |
ayoung | I can do that...I have that technology! | 20:51 |
slagle | since what I told you yesterday worked, i would expect that today this will blow up | 20:51 |
EmilienM | bnemec: I don't mind if it fails in CI at least we can see the failure online and not on your devenv | 20:51 |
ayoung | it fails very late in the process. THe funny thing is that the remote systems are OK with the cert, it is just controller calling itself via https that fails, which is a strange path | 20:51 |
bnemec | EmilienM: Have at: https://review.openstack.org/#/c/343017/ | 20:51 |
bnemec | I'm actually in the process of trying to reproduce it locally so I can debug. | 20:52 |
EmilienM | ok | 20:52 |
EmilienM | nice | 20:52 |
EmilienM | bnemec: do you remember what disabled the upgrade job? was it in project-config? | 20:52 |
bnemec | EmilienM: I would assume so. | 20:52 |
EmilienM | ok so let's get net-iso back first, and then we'll enable upgrade job I guess | 20:53 |
EmilienM | bnemec: I'm looking at logs | 20:53 |
EmilienM | Error: /Stage[main]/Ceph::Osds/Ceph::Osd[/srv/data]/Exec[ceph-osd-activate-/srv/data]/returns: change from notrun to 0 failed: Command exceeded timeout\ | 20:54 |
EmilienM | I thought we were running Jewel | 20:54 |
EmilienM | we need to change that path on Jewel, otherwise we'll have SElinux alerts | 20:54 |
EmilienM | not sure it's related to the patch though | 20:54 |
EmilienM | bnemec: wow you're adding a lot of templates.. | 20:55 |
EmilienM | while I'm moving them away from tripleo CI | 20:55 |
bnemec | EmilienM: I have no idea, but I'm pretty sure they're all failing basically the same way. | 20:55 |
gfidente | EmilienM, which job is printing that? | 20:55 |
EmilienM | I thought we agree to move them | 20:55 |
EmilienM | gfidente: https://review.openstack.org/#/c/343017/ | 20:55 |
EmilienM | bnemec: https://review.openstack.org/#/c/350779/1 | 20:55 |
gfidente | cause jewel is running now from an entire day | 20:55 |
bnemec | EmilienM: I just -1'd the t-h-t patch. :-P | 20:56 |
gfidente | so I am surprised it would pop up only now | 20:56 |
bnemec | gfidente: I'm assuming it's because that patch turns back on net-iso. | 20:56 |
gfidente | bnemec, but the error seems unrelated though | 20:56 |
gfidente | looking | 20:56 |
EmilienM | bnemec: how do you want to branch the templates without moving them? | 20:57 |
EmilienM | the worker-config-mitaka-and-below.yaml is a proof that we need to move them | 20:57 |
EmilienM | have bash conditionals for every release is a terrible idea | 20:57 |
EmilienM | we have stable branches | 20:57 |
pabelanger | slagle: is it python-tripleoclient that builds it? | 20:57 |
gfidente | EmilienM, bnemec oh command exceeded timeout | 20:58 |
gfidente | then yes looks like OSDs couldn't reach the MONs | 20:58 |
gfidente | it's not the selinux thing | 20:58 |
bnemec | EmilienM: These are templates that users write. If we're breaking them then we should absolutely be feeling that pain. | 20:58 |
bnemec | Because all of our users are too. | 20:58 |
gfidente | EmilienM, in ci centos runs with selinux in permissive mode anyway | 20:59 |
gfidente | EmilienM, which is why we aren't blocked on the selinux/path problem | 20:59 |
EmilienM | bnemec: I don't think we can maintain backward compatibily in templates for life | 20:59 |
EmilienM | bnemec: worker-config-mitaka-and-below.yaml is a proof, again. | 20:59 |
bnemec | EmilienM: That's fine. When we break compatibility, we'll appreciate that we've just caused our users pain because we'll have to deal with it too. | 20:59 |
bnemec | This is a feature, not a bug IMHO. | 20:59 |
pabelanger | slagle: looks like this is the code that does it: http://git.openstack.org/cgit/openstack/python-tripleoclient/tree/tripleoclient/v1/overcloud_image.py | 21:00 |
EmilienM | bnemec: can you look my comments on https://review.openstack.org/#/c/350775/2 and tell me if I can go ahead? | 21:04 |
gfidente | weshay, do we have to do anything in quickstart to cover https://github.com/openstack-infra/tripleo-ci/commit/c08d803ab1b91e82e9c986a3efccef72ebe397f2 ? | 21:04 |
weshay | gfidente, last I heard this morning ceph deployments were failing this morning | 21:07 |
weshay | gfidente, sec | 21:07 |
bnemec | EmilienM: That's fine. I thought that's what we had agreed on in the ML discussion. | 21:07 |
gfidente | weshay, in upstream ci ? | 21:07 |
EmilienM | bnemec: cool | 21:07 |
weshay | rdo | 21:07 |
*** liverpooler has quit IRC | 21:08 | |
gfidente | weshay, and is rdo using quickstart? | 21:10 |
weshay | yes | 21:10 |
gfidente | so yes they will fail until we got ^^ those repos | 21:10 |
*** fzdarsky has quit IRC | 21:10 | |
weshay | gfidente, I can answer in a bit | 21:10 |
gfidente | into quickstart | 21:10 |
*** lblanchard has quit IRC | 21:12 | |
openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates: Import CI THT environments from tripleo-ci https://review.openstack.org/350775 | 21:13 |
openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: Remove test-environments from tripleo-ci https://review.openstack.org/350779 | 21:13 |
EmilienM | bnemec: done ^ | 21:13 |
EmilienM | we should really converge the tools here | 21:14 |
* bnemec points out that he already wrote, tested, and proposed that tool: https://github.com/cybertron/dlrn-repo | 21:15 | |
bnemec | And got a resounding "meh" in response. | 21:15 |
EmilienM | meh! | 21:17 |
openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: WIP - Implement undercloud upgrade job - Mitaka -> Newton https://review.openstack.org/346995 | 21:18 |
bnemec | Although now that we're adding a bunch of other repos, it may have to be renamed tripleo-repo. | 21:19 |
EmilienM | bnemec: the tool looks cool, why not using it in our CI? | 21:20 |
* EmilienM off for a few hours from now | 21:23 | |
*** Ryjedo has joined #tripleo | 21:24 | |
openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart: add temp new release to verify centos7-new https://review.openstack.org/350772 | 21:24 |
openstackgerrit | Bob Fournier proposed openstack/tripleo-heat-templates: Add management network to list of networks to ping https://review.openstack.org/350787 | 21:27 |
bnemec | EmilienM: There's a bunch more work required to import it to gerrit, package it, and integrate it into the flow. I didn't want to do all of that without some indication that it was something people thought we should do. | 21:27 |
*** radek_ has quit IRC | 21:29 | |
Slower | I think the "Enable latest RDO Trunk Delorean repository only for the TripleO packages" step is wrong now in the triple.org install docs | 21:30 |
Slower | the includepkgs list does not include all the puppet packages | 21:30 |
pabelanger | any suggestions on my error with python-tripleoclient? | 21:31 |
pabelanger | ERROR: Element 'overcloud-full' not found in '/usr/share/tripleo-puppet-elements:/usr/share/instack-undercloud:/usr/share/tripleo-image-elements:/usr/share/openstack-heat-templates/software-config/elements:/usr/bin/../share/diskimage-builder/elements' | 21:31 |
pabelanger | I am not sure which project provides overcloud-full | 21:31 |
*** rhallisey has quit IRC | 21:32 | |
openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart: add temp new release to verify centos7-new https://review.openstack.org/350772 | 21:34 |
*** bootsha has quit IRC | 21:35 | |
bnemec | pabelanger: It's in instack-undercloud: https://github.com/openstack/instack-undercloud/tree/master/elements/overcloud-full | 21:36 |
openstackgerrit | Dan Sneddon proposed openstack/os-net-config: Disable IPv6 Autoconf When use_dhcpv6=False https://review.openstack.org/350794 | 21:37 |
*** chem has quit IRC | 21:39 | |
rlandy | bnemec: Running tripleo-quickstart install on OVB from master, I get "Connection timed out (http://ipxe.org/xxxxx)" when getting agent.kernel. Introspection gets stuck - repeating the error message over. Running a mitaka install on the same env works. Any thoughts on what's going wrong with master? | 21:40 |
*** rwsu has quit IRC | 21:40 | |
*** dbecker has quit IRC | 21:42 | |
*** tbonds has quit IRC | 21:43 | |
weshay | gfidente, sorry back | 21:44 |
bnemec | rlandy: Not really. It looks like it's working in CI on master. I don't tend to run it locally because I rarely need introspection data. | 21:44 |
rlandy | bnemec: thanks - I am wondering if I need bigger instances for master than mitaka - master passed on real baremetal hardware | 21:46 |
bnemec | rlandy: Introspection shouldn't require much in the way of resources. It's just collecting data, not actually deploying anything. | 21:47 |
EmilienM | bnemec: for https://review.openstack.org/#/c/350779/ we are fucked, because the templates are not in THT stable branches | 21:48 |
*** bootsha has joined #tripleo | 21:49 | |
*** limao has joined #tripleo | 21:49 | |
*** limao has quit IRC | 21:50 | |
*** limao_ has joined #tripleo | 21:50 | |
gfidente | weshay, so I think we need to bring those repo changes into quickstart | 21:51 |
weshay | gfidente, k.. the centos ceph hammer is in.. at least it is in our rdo tests internal.. thanks I'll double check at the ci.centos level | 21:52 |
weshay | gfidente, thanks! | 21:53 |
pabelanger | bnemec: thanks, took 20mins but it is working | 21:53 |
pabelanger | $ openstack overcloud image build --type overcloud-full --elements-path="/home/centos/instack-undercloud/elements:/home/centos/tripleo-puppet-elements/elements:/home/centos/tripleo-image-elements/elements:/home/centos/heat-templates/hot/software-config/elements" | 21:53 |
gfidente | weshay, I am not sure where to add those changes | 21:54 |
weshay | image building.. np | 21:54 |
weshay | I got it | 21:54 |
gfidente | ok unfortunately centos/jewel isn't finished yet but that is what we need on master | 21:54 |
weshay | https://github.com/redhat-openstack/ansible-role-tripleo-image-build | 21:54 |
gfidente | add me on the reviews | 21:54 |
pabelanger | bnemec: EmilienM: Who should I be talking to about gate jobs for python-tripleoclient? | 21:54 |
weshay | gfidente, ack | 21:55 |
*** morazi has quit IRC | 21:56 | |
*** rlandy has quit IRC | 21:59 | |
bnemec | pabelanger: I'm not sure there's anyone specific. What are you looking to add? | 22:00 |
*** rwsu has joined #tripleo | 22:03 | |
*** florianf has quit IRC | 22:03 | |
pabelanger | bnemec: I think we could add a job to build overcloud-full for example | 22:06 |
bnemec | pabelanger: We could, but it still has to be run through CI to make sure it works before we publish it. | 22:07 |
*** ebalduf has quit IRC | 22:08 | |
pabelanger | bnemec: I think there is a few use cases. 1st being, can we build an overcloud-full properly with all the dependency. Which I think could happen in python-tripleoclient. Then the 2nd would be, can tripleo-ci use that image properly | 22:09 |
pabelanger | the 2nd would be for the purpose of using cached artifacts for CI jobs, to make things faster | 22:10 |
bnemec | pabelanger: We already do all of that though. | 22:11 |
bnemec | tripleoclient changes get tripleo-ci run on them, which rebuilds the image. | 22:12 |
bnemec | Patches that can't affect the image (like tripleo-heat-templates) use a cached image. | 22:12 |
bnemec | (well, they did before we shut it off while we transition to packaged puppet modules) | 22:12 |
*** rwsu has quit IRC | 22:13 | |
pabelanger | Right, I am not sure how some of this works in tripleo-ci, but up until recently it was only running on tripleo-test-cloud-rh1. My thinking is, if we reorg some of the things we do in tripleo-ci, into external jobs. | 22:14 |
pabelanger | it also means we don't need to setup devstack everytime to test a patch to python-tripleoclient in the gate | 22:15 |
pabelanger | as an example: | 22:16 |
pabelanger | openstack overcloud image build --type overcloud-full --elements-path="/home/centos/instack-undercloud/elements:/home/centos/tripleo-puppet-elements/elements:/home/centos/tripleo-image-elements/elements:/home/centos/heat-templates/hot/software-config/elements" --debug | 22:16 |
pabelanger | fails for me locally | 22:16 |
pabelanger | and I don't know why | 22:16 |
*** fultonj has joined #tripleo | 22:18 | |
*** Ryjedo has quit IRC | 22:23 | |
*** rwsu has joined #tripleo | 22:24 | |
*** Ryjedo has joined #tripleo | 22:24 | |
*** Ryjedo has quit IRC | 22:25 | |
*** Ryjedo has joined #tripleo | 22:25 | |
*** Ryjedo has quit IRC | 22:25 | |
*** Ryjedo has joined #tripleo | 22:26 | |
*** Ryjedo has quit IRC | 22:26 | |
*** Ryjedo has joined #tripleo | 22:27 | |
*** Ryjedo has quit IRC | 22:27 | |
*** Ryjedo has joined #tripleo | 22:28 | |
*** Ryjedo has quit IRC | 22:28 | |
*** Ryjedo has joined #tripleo | 22:29 | |
*** julim has joined #tripleo | 22:37 | |
*** bfournie has quit IRC | 22:38 | |
*** rwsu has quit IRC | 22:40 | |
*** Ryjedo_ has joined #tripleo | 22:40 | |
*** Ryjedo has quit IRC | 22:40 | |
*** Ryjedo_ is now known as Ryjedo | 22:40 | |
*** rwsu has joined #tripleo | 22:45 | |
*** limao has joined #tripleo | 22:48 | |
*** limao_ has quit IRC | 22:51 | |
*** limao has quit IRC | 22:51 | |
*** limao has joined #tripleo | 22:51 | |
*** limao_ has joined #tripleo | 23:12 | |
*** limao has quit IRC | 23:16 | |
*** limao_ has quit IRC | 23:26 | |
*** tobias_fiberdata has joined #tripleo | 23:30 | |
slagle | pabelanger: before we publish images, we want to make sure they work. which is why that happens as part of tripleo-ci | 23:31 |
slagle | and we want to gate changes to tripleoclient on the condition that the built image has been used successfully by tripleo-ci | 23:32 |
slagle | i guess you're saying just move the publishing part as part of the publish job on tripleoclient? | 23:33 |
*** Ryjedo_ has joined #tripleo | 23:33 | |
slagle | i'm not sure how those jobs work, but as long as we're publishing the same qcow2 that was used in tripleo-ci, it wfm | 23:34 |
*** Ryjedo has quit IRC | 23:35 | |
*** gfidente has quit IRC | 23:37 | |
tobias_fiberdata | Nice, you have updated the tripleo.org documentation with mitaka repo! Very nice | 23:37 |
tobias_fiberdata | Now i'll go sleep :p | 23:37 |
*** tobias_fiberdata has quit IRC | 23:37 | |
pabelanger | slagle: Ya, I'll have some code tomorrow and possible experimental job | 23:38 |
slagle | cool | 23:38 |
slagle | the last time i looked at the publish jobs...i couldn't see a way to tie it in | 23:38 |
pabelanger | slagle: the biggest benefit, it could become voting once stable | 23:38 |
*** Ryjedo_ has quit IRC | 23:38 | |
*** gfidente has joined #tripleo | 23:38 | |
pabelanger | which is something you couldn't do running them under tripleo-test-cloud-rh1 | 23:38 |
*** bootsha has quit IRC | 23:38 | |
*** gfidente has quit IRC | 23:38 | |
*** gfidente has joined #tripleo | 23:38 | |
slagle | trove for instance, just publishes an image on a commit to their elements | 23:39 |
slagle | but i dont know what tests that said image actually works | 23:39 |
pabelanger | ya, a few projects are building images on commit | 23:39 |
pabelanger | and then publishing in the post pipeline | 23:39 |
pabelanger | right now, I'd just like to figure out the buildimage part | 23:40 |
pabelanger | then have the discussion on how to publish them, if needed | 23:40 |
pabelanger | now to find some food | 23:41 |
openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: WIP - Implement undercloud upgrade job - Mitaka -> Newton https://review.openstack.org/346995 | 23:44 |
*** tobias_fiberdata has joined #tripleo | 23:45 | |
*** jrist has quit IRC | 23:49 | |
*** maeca2 has left #tripleo | 23:55 | |
*** tobias_fiberdata has quit IRC | 23:59 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!