| *** dsneddon has joined #tripleo | 00:03 | |
| *** Penick has quit IRC | 00:15 | |
| *** gfidente has quit IRC | 00:16 | |
| *** rcarrillocruz has quit IRC | 00:35 | |
| *** rcarrillocruz has joined #tripleo | 00:39 | |
| *** noslzzp has joined #tripleo | 00:43 | |
| *** dkehnx1 has quit IRC | 00:48 | |
| openstackgerrit | A change was merged to openstack/tripleo-heat-templates: Make target to validate a template https://review.openstack.org/105325 | 01:03 |
|---|---|---|
| *** julim has quit IRC | 01:03 | |
| jerryz | lifeless: deployment of undercloud hangs at "booting from hard disk", any idea why? | 01:05 |
| openstackgerrit | A change was merged to openstack/os-cloud-config: Generate keystone PKI into JSON file https://review.openstack.org/104528 | 01:08 |
| greghaynes | w00t ^ | 01:08 |
| greghaynes | now we need to release os-cloud-config | 01:08 |
| greghaynes | lifeless: How do we get that to happen ^ | 01:08 |
| StevenK | greghaynes: We will probably talk about releasing the world at the meeting | 01:09 |
| tchaypo | I'd suggest putting it on the agenda but like | 01:11 |
| tchaypo | it's a standard part of the agenda. | 01:11 |
| tchaypo | so. | 01:11 |
| vinsh | PUSH ALL THE BUTTONS! | 01:13 |
| SpamapS | FIRE ZE MISSILES | 01:20 |
| lifeless | greghaynes: we cut a release | 01:32 |
| lifeless | greghaynes: let me do that now | 01:33 |
| lifeless | greghaynes: its documented and all but basically, check for criticals that might block it, then release. | 01:33 |
| lifeless | jerryz: means it didn't PXE boot | 01:34 |
| lifeless | jerryz: could be networking, wrong MAC registered with Ironic, bios not set to PXE boot... | 01:34 |
| jerryz | lifeless: but it didn't go any further | 01:34 |
| jerryz | lifeless: it already finished pxe boot | 01:34 |
| lifeless | jerryz: booting from hard disk means it failed to pxe boot | 01:35 |
| openstackgerrit | Adam Gandelman proposed a change to openstack/tripleo-image-elements: Centralize all db migrations to boot-stack element https://review.openstack.org/105640 | 01:36 |
| lifeless | greghaynes: release pushed; should be on pypi shortly | 01:37 |
| jerryz | lifeless: here is the last screen i saw before it rebooted to boot from hard disk http://picpaste.com/Capture-CckvgNqd.JPG | 01:37 |
| lifeless | jerryz: it shouldn't boot from hard disk ever. | 01:38 |
| *** eghobo has quit IRC | 01:38 | |
| lifeless | jerryz: tht looks like the deployment ramdisk output which is as expected. | 01:38 |
| lifeless | jerryz: the next boot should be PXE again. | 01:38 |
| lifeless | grabbing food, brb | 01:39 |
| *** killer_prince has quit IRC | 01:41 | |
| openstackgerrit | James Polley proposed a change to openstack-infra/tripleo-ci: Add a vlan check https://review.openstack.org/100078 | 01:43 |
| *** nosnos has joined #tripleo | 01:44 | |
| jerryz | lifeless: try changing the boot device from hd to network in virsh domain config file, i didn't use the the vm config files generated by devtest_testenv.sh. it seems to work this time. | 01:48 |
| openstackgerrit | James Polley proposed a change to openstack/tripleo-image-elements: Support new virtual_interfaces definition for VIPs https://review.openstack.org/104368 | 01:51 |
| lifeless | greghaynes: its up now | 01:51 |
| lifeless | jerryz: ah well, so yeah, if you're not using the automation, expect issues :) | 01:52 |
| lifeless | hmm,no ssl on this bring-up, stil, one step at a time | 01:54 |
| jerryz | lifeless: yeah, that's bitterly true. but just wanted to take the rough route(breaking the scripts into steps) to get an idea how it works. | 01:55 |
| lifeless | cool | 01:57 |
| jerryz | lifeless: but apologize to bring up those silly issue to bother you | 01:57 |
| lifeless | its fine | 01:59 |
| *** ramishra has joined #tripleo | 02:15 | |
| *** eghobo has joined #tripleo | 02:16 | |
| *** tzumainn has quit IRC | 02:16 | |
| *** killer_prince has joined #tripleo | 02:20 | |
| *** killer_prince is now known as lazy_prince | 02:20 | |
| lifeless | greghaynes: around? | 02:23 |
| *** ccrouch has joined #tripleo | 02:28 | |
| *** noslzzp has quit IRC | 02:38 | |
| tchaypo | lifeless: I'm looking at the test failrue on 101438 (t-h-t) | 02:40 |
| tchaypo | the cause is that NeutronPublicInterfaceTag "was not provided". | 02:40 |
| tchaypo | I'm thinking maybe we need to default that to the empty string rather than null? | 02:41 |
| lifeless | tchaypo: ugh | 02:46 |
| lifeless | so its an int | 02:46 |
| lifeless | empty strings are not ints | 02:46 |
| tchaypo | good point. | 02:46 |
| lifeless | this is a heat bug I think | 02:46 |
| tchaypo | ho hum. | 02:46 |
| lifeless | in that its internally treating None as 'not supplied' | 02:47 |
| lifeless | but a default is always supplied, even if its None | 02:47 |
| lifeless | stevebaker: plausible? ^ | 02:47 |
| lifeless | tchaypo: for now yes we may need to make it a string not an int and go lalalalala to type safety | 02:49 |
| lifeless | I'll do that in a sec | 02:49 |
| tchaypo | I can do it if you want... | 02:51 |
| lifeless | I'm editing the patch to add overcloud support | 02:51 |
| tchaypo | ah, I was wondering about that | 02:51 |
| * tchaypo reviews ther things in the meantime | 02:53 | |
| tchaypo | *other | 02:54 |
| greghaynes | lifeless: ohai | 02:54 |
| greghaynes | lifeless: just rechecked the -incubator patch | 02:55 |
| lifeless | so I'm hoping, based on my vrrp readin | 02:55 |
| lifeless | to not need an IP address on the vlan for each control plane node | 02:56 |
| lifeless | but I think I should go lalalala and focus on the minimum to get up and running | 02:56 |
| lifeless | because if we can get the test for this running | 02:56 |
| lifeless | in the gate | 02:56 |
| lifeless | before HA is turned on by default | 02:56 |
| lifeless | it becomes someone elses issue ;) | 02:56 |
| greghaynes | :( | 02:57 |
| greghaynes | well, itll be a close race | 02:57 |
| *** untriaged-bot has joined #tripleo | 03:00 | |
| untriaged-bot | Untriaged bugs so far: | 03:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1339047 | 03:00 |
| uvirtbot | Launchpad bug 1339047 in tripleo "db sync must be run once from the bootstrap node" [Undecided,In progress] | 03:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1336115 | 03:00 |
| uvirtbot | Launchpad bug 1336115 in tripleo "instances no longer running post-upgrade" [Undecided,In progress] | 03:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1335105 | 03:00 |
| uvirtbot | Launchpad bug 1335105 in tuskar-ui "Devtest init script: sourcing tripleorc fails" [Undecided,New] | 03:00 |
| *** untriaged-bot has quit IRC | 03:00 | |
| *** dsneddon is now known as dsneddon_away | 03:00 | |
| *** shardy has quit IRC | 03:01 | |
| greghaynes | lifeless: Can you not just list multiple things in virtual_ip_addresses under the same vrrp_instance | 03:04 |
| greghaynes | and specify dev ethX for each one | 03:05 |
| tchaypo | yes, that sounds feasible. | 03:05 |
| * greghaynes hasnt actually done this before, bewarned | 03:05 | |
| greghaynes | In different news, you can put literally everything in tmpfs with 64gb of ram and do a full devtest run | 03:06 |
| *** lazy_prince is now known as killer_prince | 03:09 | |
| lifeless | greghaynes: so | 03:09 |
| lifeless | greghaynes: imagine we have not-a-vlan | 03:10 |
| lifeless | greghaynes: eth0 and eth1 | 03:10 |
| lifeless | greghaynes: what vrrp instance should we be driving to select a winner on eth1 | 03:10 |
| greghaynes | Is this with the caveat that you want to only have ip's on one of the interfaces?} | 03:11 |
| greghaynes | aside from the vip | 03:12 |
| tchaypo | greghaynes: that's lovely, iffing you have 64gb of ram :p | 03:14 |
| tchaypo | lifeless: I would imagine that's topology-dependent. | 03:14 |
| openstackgerrit | lifeless proposed a change to openstack/tripleo-heat-templates: Expose public_interface_tag support. https://review.openstack.org/103454 | 03:14 |
| tchaypo | if eth1 is public but the hosts in the node use eth0 as the controlplane, I'd say it could make sense to be using the vrrp instance on eth0 to control the vip on eth1 | 03:14 |
| lifeless | greghaynes: no such caveat | 03:15 |
| lifeless | greghaynes: we want controlip on eth0 and publicip on eth1 | 03:15 |
| greghaynes | then just define two vrrp_instances methinks | 03:15 |
| tchaypo | one for each interface? | 03:15 |
| greghaynes | yes | 03:15 |
| lifeless | greghaynes: but if node A has won on eth0 and its eth1 is broken, you don't want to be down. | 03:15 |
| greghaynes | ah, now I see | 03:16 |
| openstackgerrit | James Polley proposed a change to openstack/tripleo-heat-templates: Define and bind a public virtual IP https://review.openstack.org/101438 | 03:16 |
| greghaynes | and you also want the inverse true, yes? | 03:16 |
| lifeless | greghaynes: now you've circled back to my point which is that I would like to avoid having any IP address other than the VIP on eth1 | 03:16 |
| lifeless | greghaynes: well we're not routing, so I think we want them to be independent | 03:16 |
| lifeless | there is a complication though | 03:17 |
| lifeless | say we have A,B.C | 03:17 |
| lifeless | A,B,C | 03:17 |
| lifeless | eth1 is up on all three | 03:17 |
| *** eghobo has quit IRC | 03:17 | |
| lifeless | eth0 is down on C | 03:17 |
| lifeless | should C be permitted to own the VIP on eth1 ? | 03:17 |
| lifeless | it can't actually offer any services since it doesn't have internal connectivity. | 03:18 |
| * lifeless has head explode | 03:18 | |
| greghaynes | eth1 is internal in this case? | 03:18 |
| lifeless | greghaynes: sorry, eth0=ctlplane, eth1=public | 03:18 |
| tchaypo | this wouldn't be a problem if it was doing vrrp on eth0 | 03:19 |
| lifeless | so we actually want the winner to win on both eth0 and eth1 | 03:19 |
| greghaynes | I would think if eth1 is down for you, you can still become master on eth0. If eth0 is down for you then you cannot become master on eth1 | 03:19 |
| greghaynes | As long as all our services hit each other via controlplane | 03:20 |
| *** nosnos has quit IRC | 03:20 | |
| lifeless | greghaynes: can we do that while still requiring that you cannot become master on eth1 unless you're part of the quorum set on eth0 ? | 03:20 |
| greghaynes | heh | 03:20 |
| greghaynes | We might need a more powerful tool for this job | 03:21 |
| tchaypo | hrm. | 03:21 |
| tchaypo | greghaynes: I'm thinking that if we just have one vrrp instance on eth0, it's not going to be checking eth1, right? | 03:21 |
| tchaypo | so my idea is *ahem* less than perfect. | 03:21 |
| *** killer_prince is now known as lazy_prince | 03:21 | |
| lifeless | I think a vrrp instance that checks two interfaces will do it | 03:22 |
| lifeless | but it means 'winner wins on both eth0 and eth1' AFAICT | 03:22 |
| *** nati_ueno has quit IRC | 03:22 | |
| lifeless | via | 03:23 |
| lifeless | vrrp_sync_group | 03:23 |
| lifeless | and two vrrp instances | 03:23 |
| tchaypo | I think it's probably good enough, for now, to say that if either interface is down it can't be a master | 03:23 |
| lifeless | vrrp_sync_group is the win then | 03:24 |
| tchaypo | it gives less-than-maximum availablity but it's still better than controlscale=1 | 03:24 |
| greghaynes | lifeless: nice find | 03:24 |
| greghaynes | good enough for me | 03:24 |
| lifeless | tchaypo: https://review.openstack.org/#/c/103454/ updated FWIW, ircbot fail :) | 03:28 |
| tchaypo | I know, I already rebased https://review.openstack.org/#/c/101438/ :) | 03:30 |
| openstackgerrit | lifeless proposed a change to openstack/tripleo-incubator: Add public VLAN support. https://review.openstack.org/103757 | 03:37 |
| lifeless | ok incubator patch upwards flowing | 03:37 |
| tchaypo | 103757? | 03:38 |
| tchaypo | but it was like only two days ago that greghaynes hit 100000 | 03:38 |
| greghaynes | :) | 03:38 |
| greghaynes | and its almost time to un-WIP that one | 03:38 |
| tchaypo | is "please stop I can't keep up" an option? | 03:38 |
| *** mestery has quit IRC | 03:40 | |
| tchaypo | so | 03:45 |
| tchaypo | infra have abandoned pypi-mirror | 03:45 |
| lifeless | no wheels? | 03:45 |
| tchaypo | do we want to (A) update our docs/elements to deprecate it, (B) take on care-and-feeding, (C) something else | 03:45 |
| tchaypo | I think it's more that they found it not useful | 03:45 |
| lifeless | its a good question | 03:46 |
| tchaypo | infra have a full bandersnatch mirror; having the additional local mirror probably doesn't help a lot | 03:46 |
| greghaynes | I really like it | 03:46 |
| lifeless | I think it should be asked to a broader audience than the 0346UTC IRC groupies | 03:46 |
| greghaynes | pypi-mirror that is | 03:46 |
| StevenK | I think we want A, but A-1 should be add a bandersnatch element | 03:46 |
| lifeless | tchaypo: actually wheels are a huge win | 03:46 |
| greghaynes | yes | 03:46 |
| * tchaypo done emails | 03:46 | |
| StevenK | tchaypo: Where did you see pypi-mirror == unloved ? | 03:47 |
| tchaypo | https://bugs.launchpad.net/openstack-ci/+bug/1294381 | 03:49 |
| uvirtbot | Launchpad bug 1294381 in openstack-ci "pypi-mirror does not pickup all necessary python packages" [Medium,Won't fix] | 03:49 |
| tchaypo | "The openstack infra team is no longer using this tool." | 03:50 |
| tchaypo | also clarkb confirmed in infra | 03:50 |
| tchaypo | email sented | 03:54 |
| openstackgerrit | A change was merged to openstack/diskimage-builder: Really handle Ubuntu mirror cache skew, https://review.openstack.org/105578 | 04:00 |
| StevenK | lifeless: ^ *thank you* | 04:02 |
| StevenK | I saw that yesterday, and was wondering if I was going mad | 04:02 |
| StevenK | Because I kept circling back to "But Ng fixed it ..." | 04:03 |
| *** marun has joined #tripleo | 04:03 | |
| *** yamahata has joined #tripleo | 04:03 | |
| lifeless | stevebaker: so - we can either wait 2-3 weeks+, or put an HTTP server up instead of purely file-local repos. | 04:04 |
| lifeless | stevebaker: apt-get install apache2-mpm-worker FTW> | 04:04 |
| SpamapS | lifeless: is this the mod_speling thing? | 04:06 |
| SpamapS | (best module name evar) | 04:06 |
| lifeless | nope | 04:07 |
| lifeless | just avoiding broken 404 handling | 04:07 |
| SpamapS | thats... the same thing? | 04:08 |
| lifeless | nope | 04:08 |
| SpamapS | as in, pip expects SQLAlchemy and sqlalchemy to be the same thing | 04:08 |
| lifeless | so no | 04:08 |
| lifeless | we've not hit that at all AFAICT | 04:09 |
| SpamapS | we have | 04:09 |
| lifeless | its all been about 404 handling | 04:09 |
| SpamapS | but we fixed it | 04:09 |
| lifeless | sigh | 04:09 |
| lifeless | we fixed the 404 handling | 04:09 |
| lifeless | which fixes the thing that looks like its a case insensitivity issue | 04:09 |
| lifeless | but isn't | 04:09 |
| SpamapS | There were several projects which had mixed case requirements that failed on file:/// mirrors but not http:// mirrors that were case-insensitive. | 04:09 |
| SpamapS | ok | 04:09 |
| SpamapS | ok | 04:09 |
| lifeless | they also worked on http:// mirrors that *ARE* case sensitive. | 04:10 |
| * SpamapS steps away and reads new context things arriving in inbox | 04:10 | |
| lifeless | the case folding handling is done by doing a directory listing (inefficient, but true), but the file:/// exception thrown on the missing directory preventing the listing code ever being triggered. | 04:10 |
| lifeless | if I recall the exact details correctly. | 04:11 |
| lifeless | The exception has other side effects too, which is why I'm saying its really 404 (with file:///) handling | 04:11 |
| SpamapS | I believe you, and yield! | 04:13 |
| *** nosnos has joined #tripleo | 04:13 | |
| greghaynes | ok, just pused a patch to fix that wontfix bug | 04:15 |
| lifeless | cool | 04:16 |
| greghaynes | so, I think *technically* its not unsupported | 04:16 |
| greghaynes | :p | 04:16 |
| clarkb | its not like its hard. no one would do it after I spelled out how to do it | 04:17 |
| clarkb | I could've done it but time was better spent getting away from the tool :) | 04:17 |
| greghaynes | fair enough | 04:18 |
| StevenK | Right, I have the beginnings of a pypi mirror | 04:18 |
| tchaypo | StevenK: did you see jogo's email | 04:21 |
| StevenK | tchaypo: Yup | 04:21 |
| StevenK | tchaypo: I've wanted a full pypi mirror locally to match my Ubuntu mirror for a while | 04:22 |
| tchaypo | lifeless: that's correct - http indexes throw a 404 which gets caught; file indexes thought something like os.pathnotfound which just bubbles all the way up and aborts. All I did was catch the latter and turn it into a fake-404 | 04:22 |
| tchaypo | which is hardly what you'd call a "correct" fix but it leads to the desired result | 04:23 |
| clarkb | devpi won't build wheels either | 04:24 |
| clarkb | fwiw | 04:24 |
| *** nosnos has quit IRC | 04:24 | |
| jogo | tchaypo StevenK: devpi-server can download packages in the background | 04:25 |
| jogo | so when you run 'tox -r' for the first time in a month you don't have to wait to pull thigns down | 04:26 |
| jogo | that and I have a wheelhouse | 04:26 |
| * tchaypo prepares to relocate | 04:26 | |
| lifeless | so actual omg it might work test running up now | 04:27 |
| lifeless | jogo: so yeah bandersnatch | 04:29 |
| lifeless | jogo: but the wheelhouse is the interesting thing | 04:29 |
| lifeless | jogo: we want one that looks like a pypi index | 04:29 |
| lifeless | tchaypo: can you run the meeting tonight? | 04:30 |
| greghaynes | wheelhouse? | 04:30 |
| StevenK | greghaynes: A place for wheels to live | 04:31 |
| greghaynes | heh | 04:31 |
| StevenK | Or a sweet spot on a baseballer | 04:31 |
| greghaynes | Do you just make that with pip wheel or is there an actual tool to do the transitive dependencies? | 04:31 |
| jogo | lifeless: I haven't looked into bandersnatch yet but it sounds cool | 04:31 |
| jogo | the wheelhouse stuff is fun no gcc runs anymore | 04:32 |
| greghaynes | yes, this is why I use pypi-mirror ;) | 04:32 |
| greghaynes | I thought bandersnatch didnt wheel;? | 04:32 |
| * StevenK is currently checking out if he can safely cron bandersnatch runs on his file server | 04:32 | |
| StevenK | /dev/mapper/sys-srv 4.5T 2.8T 1.8T 62% /srv | 04:32 |
| jogo | at$ cat ~/.pip/pip.conf | 04:32 |
| jogo | [global] | 04:32 |
| jogo | index-url = http://localhost:3141/root/pypi/+simple/ | 04:32 |
| jogo | wheel-dir = /home/jogo/.pip/wheelhouse | 04:32 |
| jogo | find-links = /home/jogo/.pip/wheelhouse | 04:32 |
| *** amitpp has joined #tripleo | 04:32 | |
| StevenK | I may have ~90GiB to spare for a pypi mirror | 04:32 |
| jogo | pip wheel -r global-reqs | 04:33 |
| lifeless | StevenK: /dev/mapper/lifelesswks--vg-lifelesswks--lvs 15T 7.5T 6.8T 53% / | 04:33 |
| clarkb | greghaynes: bandersnatch will grab any wheels that are on pypi | 04:33 |
| StevenK | lifeless: Hah, nice | 04:33 |
| greghaynes | clarkb: yea :/ | 04:33 |
| clarkb | greghaynes: it won't build them for your arch if they don't exist. this was a pypi-mirror special | 04:33 |
| lifeless | jogo: so the thing is that we're not running locally... its chroot image builds etc | 04:33 |
| clarkb | lifeless: right so you can shove that wheelhouse behind mod autoindex | 04:34 |
| StevenK | 2014-07-09 14:33:51,879 INFO: 45824 packages to sync. | 04:34 |
| clarkb | its basiaclly what mordred was sugesting earlier | 04:34 |
| *** amitpp has quit IRC | 04:34 | |
| StevenK | But it doesn't say how *much* :-( | 04:34 |
| mordred | I suggest all things | 04:34 |
| StevenK | apt-mirror gave me: 1.0 GiB will be downloaded into archive. | 04:36 |
| greghaynes | StevenK: bandersnatch is afraid you might stop using it if they told you | 04:37 |
| StevenK | I wonder if the mirror todo for bandersnatch caculates files | 04:37 |
| StevenK | *file sizes | 04:37 |
| StevenK | steven@thrashed:~$ head -n 2 /srv/pypi/todo | tail -n 1 | 04:38 |
| StevenK | openslides-csv-export 1111916 | 04:38 |
| *** weshay has quit IRC | 04:38 | |
| StevenK | That's a serial, and not a file size, right? | 04:38 |
| *** amitpp has joined #tripleo | 04:38 | |
| tchaypo | Yes, I can run tonight | 04:38 |
| tchaypo | Lifeless: ^^ | 04:38 |
| *** ci-overcloud has joined #tripleo | 04:38 | |
| ci-overcloud | ************** ci-overcloud complete status=1 ************ | 04:38 |
| *** ci-overcloud has quit IRC | 04:38 | |
| greghaynes | WHOA | 04:38 |
| greghaynes | havent seen that in a while | 04:39 |
| *** sseago__ has quit IRC | 04:40 | |
| *** eghobo has joined #tripleo | 04:42 | |
| *** akuznetsov has quit IRC | 04:46 | |
| *** akuznetsov has joined #tripleo | 04:53 | |
| lifeless | + echo 'Error: Heat environment cache "/home/shared/overcloud-env.json" not set to permissions of 0600.' | 04:56 |
| StevenK | lifeless: greghaynes did it | 04:57 |
| greghaynes | http://www.sadtrombone.com/ | 04:57 |
| *** ci-overcloud has joined #tripleo | 04:57 | |
| ci-overcloud | ************** ci-overcloud complete status=1 ************ | 04:57 |
| *** ci-overcloud has quit IRC | 04:57 | |
| lifeless | hmm ERROR: Property error : controllerConfig: config The specified reference "NeutronPublicInterfaceIPTag" (in unknown) is incorrect. | 05:02 |
| lifeless | will get back to this after this call | 05:02 |
| *** rushiagr_ has joined #tripleo | 05:06 | |
| *** vinsh has quit IRC | 05:10 | |
| *** rushiagr_ has quit IRC | 05:11 | |
| greghaynes | wooo failed undercloud and undercloud ironic but passed overcloud https://review.openstack.org/#/c/95101/ | 05:12 |
| *** akuznetsov has quit IRC | 05:12 | |
| openstackgerrit | lifeless proposed a change to openstack/tripleo-heat-templates: Expose public_interface_tag support. https://review.openstack.org/103454 | 05:13 |
| *** bmahalakshmi has joined #tripleo | 05:15 | |
| *** ci-overcloud has joined #tripleo | 05:16 | |
| ci-overcloud | ************** ci-overcloud complete status=1 ************ | 05:16 |
| *** ci-overcloud has quit IRC | 05:16 | |
| *** morganfainberg_Z is now known as morganfainberg | 05:18 | |
| *** ramishra has quit IRC | 05:19 | |
| *** rcarrill` has joined #tripleo | 05:20 | |
| jogo | have an old nova bug that you guys may be able to help re-triage https://bugs.launchpad.net/nova/+bug/1069917 | 05:21 |
| uvirtbot | Launchpad bug 1069917 in nova "Setting InnoDB for tables breaks mysqlcluster/ndb replication" [Undecided,In progress] | 05:21 |
| *** rcarrillocruz has quit IRC | 05:23 | |
| *** akuznetsov has joined #tripleo | 05:26 | |
| *** dshulyak_ has joined #tripleo | 05:27 | |
| *** bmahalakshmi has quit IRC | 05:29 | |
| *** bmahalakshmi has joined #tripleo | 05:30 | |
| *** dshulyak_ has quit IRC | 05:31 | |
| *** lazy_prince is now known as killer_prince | 05:32 | |
| *** rakesh_hs has joined #tripleo | 05:36 | |
| openstackgerrit | A change was merged to openstack/tripleo-image-elements: ensure-bridge: bring up bridges first on Debian https://review.openstack.org/104625 | 05:40 |
| *** chuckC has quit IRC | 05:41 | |
| *** rameshg87_afk is now known as rameshg87 | 05:50 | |
| *** ramishra has joined #tripleo | 05:54 | |
| openstackgerrit | Ghe Rivero proposed a change to openstack/tripleo-image-elements: Rename os-config-applier to os-apply-config https://review.openstack.org/104355 | 06:00 |
| *** rushiagr has joined #tripleo | 06:05 | |
| *** bmahalakshmi has quit IRC | 06:07 | |
| *** bmahalakshmi has joined #tripleo | 06:08 | |
| *** rdopieralski has joined #tripleo | 06:08 | |
| *** bmahalakshmi has quit IRC | 06:15 | |
| *** bmahalakshmi has joined #tripleo | 06:15 | |
| *** morganfainberg is now known as morganfainberg_Z | 06:23 | |
| *** marun has quit IRC | 06:27 | |
| *** kshantha_ has joined #tripleo | 06:31 | |
| *** kshantha_ has quit IRC | 06:31 | |
| *** shausy has joined #tripleo | 06:36 | |
| *** shausy has quit IRC | 06:41 | |
| *** mrunge has joined #tripleo | 06:44 | |
| *** shausy has joined #tripleo | 06:44 | |
| *** mkerrin has joined #tripleo | 06:49 | |
| tchaypo | meeting in 7 minutes, woo | 06:54 |
| *** jprovazn has joined #tripleo | 06:55 | |
| *** jcoufal has joined #tripleo | 06:55 | |
| marios | meeting? | 07:01 |
| *** jistr has joined #tripleo | 07:01 | |
| openstackgerrit | Ghe Rivero proposed a change to openstack/tripleo-incubator: Wait for stats to be updated https://review.openstack.org/105662 | 07:05 |
| shadower | stevebaker: still around by any chance? | 07:05 |
| *** jml has quit IRC | 07:10 | |
| *** jtomasek has joined #tripleo | 07:12 | |
| *** jml has joined #tripleo | 07:14 | |
| *** eghobo has quit IRC | 07:17 | |
| *** eghobo has joined #tripleo | 07:18 | |
| *** regebro has joined #tripleo | 07:24 | |
| *** nibalizer has quit IRC | 07:27 | |
| d0ugal | shadower: Just spotted this review - https://review.openstack.org/#/c/105592/ | 07:35 |
| d0ugal | shadower: "I'm landing my own revert. The patch was approved without proper testing, and this _WILL_ break TripleO if it ever gets into a release of os-collect-config." | 07:36 |
| d0ugal | shadower: thought I'd point it out since your releasing | 07:36 |
| shadower | d0ugal: thanks that's really good to know | 07:37 |
| d0ugal | np | 07:38 |
| greghaynes | huh, our testing tripleo for those patches is pretty useless | 07:38 |
| greghaynes | since we install from latest release | 07:38 |
| greghaynes | (os-c-c) | 07:38 |
| d0ugal | So its never fully tested until its released? | 07:39 |
| d0ugal | Yikes :) | 07:39 |
| greghaynes | Yes... that seems like a good thing to fix in CI | 07:39 |
| StevenK | greghaynes: Same for os-cloud-config. I have a WIP change for os-a-c and os-cloud-config | 07:39 |
| greghaynes | StevenK: oh, sweet | 07:39 |
| *** gfidente has joined #tripleo | 07:39 | |
| d0ugal | Nice | 07:39 |
| StevenK | os-collect-config is installed via tie, maybe it should respect ZUUL_* | 07:40 |
| *** eguz has joined #tripleo | 07:41 | |
| *** eguz has quit IRC | 07:43 | |
| lifeless | greghaynes: StevenK: so thats what stevebaker's patch is for | 07:45 |
| lifeless | builds a local pip repo of the ZUUL refs sdisted | 07:45 |
| *** eghobo has quit IRC | 07:45 | |
| lifeless | only catch is it needs pip 1.6 or the index URL to be on http. | 07:45 |
| lifeless | better to fix that than have tie know about zuul .... | 07:45 |
| *** rakesh_hs has quit IRC | 07:47 | |
| *** shardy has joined #tripleo | 07:47 | |
| StevenK | lifeless: Right | 07:47 |
| StevenK | It doesn't help with os-cloud-config, but that just needs me to work out where the os-cloud-config code is checked out on disk for a change so we can pip install from it | 07:48 |
| *** rakesh_hs has joined #tripleo | 07:50 | |
| tchaypo | joFor anyone who missed the meeting | 07:55 |
| tchaypo | @openstack Minutes: http://eavesdrop.openstack.org/meetings/tripleo/2014/tripleo.2014-07-09-07.00.html | 07:55 |
| lifeless | StevenK: why doesn't it help with os-cloud-config ? | 07:56 |
| StevenK | lifeless: Because it's installed via setup-clienttools | 07:57 |
| *** markmc has joined #tripleo | 07:57 | |
| *** rakesh_hs has quit IRC | 07:59 | |
| *** rakesh_hs has joined #tripleo | 08:00 | |
| lifeless | StevenK: still don't see the connection | 08:03 |
| *** pcrews has quit IRC | 08:04 | |
| jprovazn | https://review.openstack.org/#/c/95101/ | 08:04 |
| jprovazn | ^ this is good to go in I believe | 08:04 |
| jprovazn | or anybody disagree? | 08:05 |
| *** jang1 has joined #tripleo | 08:05 | |
| *** martyntaylor has joined #tripleo | 08:05 | |
| lifeless | jprovazn: whats the change to the clienttools stuff for ? | 08:06 |
| lifeless | oh, I see. tats a bit ick but unrelated | 08:07 |
| jprovazn | lifeless: to have the script from os-cloud-config in PATH | 08:07 |
| *** cwolferh has quit IRC | 08:08 | |
| jprovazn | lifeless: thanks for +2, though didn't you want to add +A too? | 08:08 |
| *** lazy_prince has joined #tripleo | 08:09 | |
| lifeless | jprovazn: bah, meant to hit +A only actually :) | 08:09 |
| *** lazy_prince has quit IRC | 08:10 | |
| lifeless | jprovazn: you can do it :> | 08:10 |
| *** shausy has quit IRC | 08:10 | |
| shadower | too late :p | 08:10 |
| jprovazn | ok | 08:10 |
| *** killer_prince is now known as lazy_prince | 08:10 | |
| jprovazn | shadower: cool :) | 08:10 |
| jprovazn | so https://review.openstack.org/#/c/94609/ is unblocked, /me reruns CI tests | 08:11 |
| *** ramishra_ has joined #tripleo | 08:11 | |
| openstackgerrit | A change was merged to openstack/tripleo-incubator: Generate overcloud keystone keys/certs https://review.openstack.org/95101 | 08:11 |
| *** derekh_ has joined #tripleo | 08:11 | |
| shadower | lifeless: will you please add marios to the tripleo-ptl group so he can make releases, too? | 08:11 |
| shadower | I'm the only one able to volunteer from the alt-time meeting bunch | 08:12 |
| shadower | marios@redhat.com | 08:12 |
| proffalken | hey all, the final objection to https://review.openstack.org/#/c/87223/ has been lifted - please can I get a core to review/approve as you see fit? :) | 08:13 |
| shadower | lifeless, marios: actually, never mind, I can do it too apparently | 08:13 |
| shadower | marios: you're in unless someone vetoes it :-) | 08:13 |
| shadower | marios: feel free to look at the other steps in the wiki: https://wiki.openstack.org/wiki/TripleO/ReleaseManagement | 08:14 |
| *** ramishra has quit IRC | 08:14 | |
| shadower | (specifically the gpg key setup) | 08:14 |
| *** cwolferh has joined #tripleo | 08:18 | |
| *** ramishra_ has quit IRC | 08:21 | |
| *** ramishra has joined #tripleo | 08:21 | |
| marios | shadower: thanks mate! and will do | 08:21 |
| shadower | marios: yep, no problem. Wish I realised this sooner | 08:23 |
| marios | shadower: i'll set up so i can have a bash next week (well at least when its on a wednesday again) | 08:25 |
| *** lucasagomes has joined #tripleo | 08:28 | |
| *** cwolferh_ has joined #tripleo | 08:30 | |
| *** cwolferh has quit IRC | 08:32 | |
| *** pblaho has joined #tripleo | 08:32 | |
| openstackgerrit | Ramakrishnan G proposed a change to openstack/diskimage-builder: Add dhcp support for ramdisk element. https://review.openstack.org/97301 | 08:34 |
| jprovazn | another major HA bit which was not addressed yet is neutron (particulary neutron-l3-agent) - https://trello.com/c/DaIs1zxb/82-neutron-ha-redundant-environment | 08:35 |
| jprovazn | as trello card says, the plan is to try out the neutron patch which adds HA support | 08:36 |
| jprovazn | I'm quite skeptic about using unmerged patches in TreipleO - the HA stuff in neutron doesn't seem to be merged anytime soon | 08:37 |
| jprovazn | what would you think about using pacemaker as a temporary solution for now until neutron supports HA? pacemaker setup should be quite easy - something similar to https://review.openstack.org/#/c/105397/ | 08:39 |
| lifeless | derekh_: hi | 08:39 |
| lazy_prince | i am grtting error when building RHEL 6.5 x86_64 images on Ubuntu 13.10. http://paste.openstack.org/show/85742/ | 08:39 |
| lifeless | derekh_: so oc stuff pushed up, there's a local edit to deploy-ci-overcloud | 08:39 |
| lazy_prince | it says no module named argparse | 08:40 |
| lifeless | derekh_: andjust started running it but i need to halt()- in theory its all go | 08:40 |
| lifeless | derekh_: but.. schceduler error on first deploy | 08:40 |
| jprovazn | the other alternative was reassigning routers from failed neutron node (Described here: http://lists.openstack.org/pipermail/openstack/2013-December/004226.html) - this setup seems to be more difficult and little bit more hacky | 08:40 |
| lifeless | derekh_: ironic can see all the nodes power state, so IPMI routing is good | 08:40 |
| jprovazn | though allows a/a to some extent | 08:41 |
| jprovazn | lifeless: greghaynes: marios: gfidente: ^ what do you think? would you go with pacemaker for now? | 08:41 |
| lifeless | jprovazn: hmm, what makes it more hacky? | 08:41 |
| lazy_prince | can someone look at the logs and help me..? | 08:41 |
| lifeless | jprovazn: pacemaker is involved in both cases, no ? | 08:41 |
| derekh_ | lifeless: cool, I'll log in in a big and check up on it, see if it needs any help | 08:42 |
| jprovazn | lifeless: 2 things: you need credential to connect from overcloud inside and you run cronjob on all controller machines | 08:42 |
| jprovazn | lifeless: cron job to reassign routers | 08:42 |
| lifeless | jprovazn: huh, why wouldn't we trigger it from pacemaker ? | 08:42 |
| jprovazn | lifeless: I don't know how to easily use pacemaker for running the reassigning | 08:42 |
| lifeless | ah | 08:43 |
| lifeless | so fabbione might be able to offer us pointers there ;) | 08:43 |
| jprovazn | fabbione: ^ any ideas how to easily use pacemaker to invoke a custom script when once of l3-agents fails? | 08:45 |
| marios | lifeless: its bits of scripts here and there for one | 08:45 |
| marios | thought i haven't had a chance to review the ceilometer solution yet... doing that now | 08:45 |
| lifeless | I need to halt() | 08:45 |
| lifeless | sorry :/ - but look honestly, lets get something up and iterate. Doesn't need to be perfect unless we expect backwards compat issues when we change it. | 08:46 |
| jprovazn | lifeless: agree, and use pacemaker in a/p mode looks like easiest way for the first iteration to me | 08:47 |
| fabbione | jprovazn: uh.... what kind of problem are you trying to solve here? | 08:47 |
| fabbione | btw, i have scheduled time next week to start looking into TripleO+pcmk | 08:48 |
| fabbione | just being swamped all the time doesn't help :) | 08:48 |
| fabbione | and why does it have to be custom? | 08:49 |
| fabbione | pcmk work flow is: | 08:50 |
| fabbione | start -> monitor ... monitor.... | 08:50 |
| fabbione | if a monitor action fails then -> stop | 08:50 |
| fabbione | so if you need to do something "special" you would that in the stop action itself | 08:50 |
| jprovazn | fabbione: neutron-l3-agent - a proposed alternative to run l3 agent in a/p mode was: running multiple l3-agents and make sure that routers from the failed ndoes are reassigned to another l3-agent - this is described here:http://lists.openstack.org/pipermail/openstack/2013-December/004226.html | 08:50 |
| fabbione | jprovazn: did you talk to ajo about it? | 08:50 |
| fabbione | he has been working on solving all those problems in pcmk environment already | 08:51 |
| openstackgerrit | A change was merged to openstack/diskimage-builder: Remove fixfiles from rpm-distro finalize https://review.openstack.org/103451 | 08:51 |
| jprovazn | fabbione: about what? my preference is to use pacemaker in a/p mode for this. lifeless was wondering if it would be possible to use pacemaker for invoking the re-routing script | 08:52 |
| fabbione | jprovazn: yes i just told you ^^^ | 08:52 |
| fabbione | 10:50 < fabbione> if a monitor action fails then -> stop | 08:52 |
| fabbione | 10:50 < fabbione> so if you need to do something "special" you would that in the stop action itself | 08:52 |
| fabbione | you would to the re-routing call only if the local agent is failed right? | 08:52 |
| fabbione | if so you can do it conditionally at stop | 08:52 |
| jprovazn | fabbione: right | 08:52 |
| fabbione | monitor / status -> all ok, keep going | 08:53 |
| gfidente | jprovazn, just for me to understand a little better the problem, do you know if the script linked in the email is something that neutron is supposed to do on its own in some time? | 08:53 |
| fabbione | monitor / status -> fails -> pcmk calls stop | 08:53 |
| fabbione | then it's up to you to have a hook in the stop action to determine if we are doing a clean shutdown or we need to recover something | 08:53 |
| marios | gfidente: no the email says put it into cron job | 08:53 |
| jprovazn | fabbione: thanks, that might work | 08:53 |
| fabbione | jprovazn: if you are mingling with neutron, can you please coordinate with ajo / Miguel from the neutron team? | 08:54 |
| marios | fabbione: i can also help there | 08:54 |
| fabbione | jprovazn: he has done almost all neutron <-> pcmk integration work and he knows pretty well what to do and how | 08:54 |
| gfidente | marios, yeah I can see for now one should run it from cron I just wonder if the approach implemented by the script is something supposed to be 'managed' by neutron at some point or if it is instead just a temporary 'workaround' | 08:54 |
| fabbione | marios: i am sure. but we have most of this stuff already done :) it's a matter of closing a communication gap here | 08:55 |
| marios | gfidente: well... there is an upstream proposal which uses keepalived | 08:55 |
| jprovazn | fabbione: yes, I'll contact him in case I'll get involved in this | 08:55 |
| marios | gfidente: but jprovazn and i agree it won't happen soon | 08:55 |
| fabbione | jprovazn: ok | 08:55 |
| marios | gfidente: sure, just saying... i can help with reviews for example (I have a 3 control node setup waiting for testing) | 08:55 |
| marios | fabbione: ^^^ | 08:56 |
| jprovazn | I'll be on PTO next weeks, so I'll push this forward after then if it will not move itself :) | 08:56 |
| marios | jprovazn: 'praise the sun' | 08:56 |
| fabbione | marios: yeps thanks, appreciated | 08:56 |
| marios | as wfoster says | 08:56 |
| *** lazy_prince is now known as killer_prince | 08:56 | |
| gfidente | marios, jprovazn so with keepalived they would ensure there is a single l3 agent hosting all router at all times, is this correct? | 08:57 |
| marios | fabbione: so is 'this stuff' indepependent of the upstream patches pending review for ha_l3_agent? | 08:57 |
| wfoster | marios: :) | 08:57 |
| jprovazn | fabbione: " but we have most of this stuff already done :)" - but you use pacemaker in a/p mode, correct? | 08:57 |
| jprovazn | for neutron-l3-agent | 08:57 |
| fabbione | marios: this stuff are cleanup init scripts that we added to workaround broken neutron design | 08:58 |
| fabbione | so they would never be accepted upstream | 08:58 |
| fabbione | that's the first problem | 08:58 |
| fabbione | the other is that in the HA world, that broken design, breaks even more | 08:58 |
| gfidente | jprovazn, marios if upstream neutron is kind of using keepalived, I think pacemaker a/p "now" makes the most sense | 08:58 |
| gfidente | (using keepalived in the future) | 08:59 |
| marios | gfidente: well yes. the point is 'is it available today' --> no | 08:59 |
| fabbione | untill you get full recovery mechanism in TO, keepalived is a single point of failure | 08:59 |
| marios | so a/p corosync/pacemaker (for which we have elements already) is available today | 08:59 |
| fabbione | that's one big problem | 08:59 |
| gfidente | marios, indeed it is not today but we'd probably get closer to have a similar behaviour today with pacemaker a/p | 08:59 |
| *** untriaged-bot has joined #tripleo | 09:00 | |
| untriaged-bot | Untriaged bugs so far: | 09:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1339047 | 09:00 |
| uvirtbot | Launchpad bug 1339047 in tripleo "db sync must be run once from the bootstrap node" [Undecided,In progress] | 09:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1336115 | 09:00 |
| gfidente | so I'd +1 the interim pacemaker a/p proposal from jprovazn | 09:00 |
| uvirtbot | Launchpad bug 1336115 in tripleo "instances no longer running post-upgrade" [Undecided,In progress] | 09:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1335105 | 09:00 |
| uvirtbot | Launchpad bug 1335105 in tuskar-ui "Devtest init script: sourcing tripleorc fails" [Undecided,New] | 09:00 |
| *** untriaged-bot has quit IRC | 09:00 | |
| jprovazn | fabbione: so the option of l3-agent in a/a mode reassigning routers with pcmk was already investigated? | 09:03 |
| fabbione | jprovazn: not that specifically, but the problem of "running custom scripts on failure" is a very common thing that needs to be addressed in either your systemd or init script | 09:04 |
| jprovazn | fabbione: and what is your opinion about the option of reassigning failed routers (vs. the standard a/p mode)? | 09:05 |
| gfidente | jprovazn, if in a/a mode, give pacemaker restars the agent, would we really need to balancing script? | 09:05 |
| fabbione | jprovazn: talk to ajo about that design.. i really don't know enough about neutron internals to make a proper call on the subject | 09:05 |
| fabbione | jprovazn: from an infrastructure perspective, i can tell you that it can be done (from pcmk) but if it's wise to do it, that's up to the specialists in the area | 09:06 |
| jprovazn | gfidente: yes, if restart fails after couple of retries you need fencing is usaully done (Which we don't supoprt yet), alternative would be reassigning routers | 09:06 |
| *** rcarrillocruz has joined #tripleo | 09:08 | |
| *** jp_at_hp has joined #tripleo | 09:10 | |
| *** rcarrill` has quit IRC | 09:10 | |
| gfidente | jprovazn, ah yes ... didn't think about all that so overall I'm still in favour of the a/p config | 09:10 |
| *** shausy has joined #tripleo | 09:10 | |
| *** killer_prince is now known as lazy_prince | 09:12 | |
| openstackgerrit | Michael Kerrin proposed a change to openstack/tripleo-image-elements: Only perform db sync once per service on the bootstrap node https://review.openstack.org/105678 | 09:14 |
| openstackgerrit | Ramakrishnan G proposed a change to openstack/diskimage-builder: Add virtual media boot support in ramdisk element. https://review.openstack.org/97143 | 09:18 |
| *** pelix has joined #tripleo | 09:19 | |
| *** rushiagr_ has joined #tripleo | 09:28 | |
| gfidente | ping greghaynes regarding elements/haproxy/os-refresh-config/configure.d/75-haproxy-nonlocal-bind | 09:29 |
| *** jang1 has quit IRC | 09:29 | |
| gfidente | I wonder if at is first start haproxy succeeds despite the systctl key is enabled only after a refresh-config | 09:30 |
| gfidente | *its first start | 09:31 |
| *** rushiagr has quit IRC | 09:32 | |
| *** martyntaylor has quit IRC | 09:37 | |
| *** rakesh_hs has quit IRC | 09:40 | |
| *** rakesh_hs has joined #tripleo | 09:40 | |
| fabbione | gfidente: it depends.. you might be lucky the VIP is there and haproxy will start, but you really want the sysctl key set befoer to be 100% sure | 09:42 |
| gfidente | fabbione, yeah that I suspected indeed | 09:42 |
| fabbione | https://github.com/fabbione/rhos-ha-deploy/blob/master/rhos5-rhel7/mrgcloud-setup/RHOS-RHEL-HA-how-to-mrgcloud-rhos5-on-rhel7-lb-latest.txt | 09:43 |
| fabbione | gfidente: ^^^ | 09:43 |
| *** lazy_prince has quit IRC | 09:44 | |
| fabbione | if you follow the same seuqnce of commands, that's known to work basically | 09:44 |
| fabbione | despite configs/IPs that might be very different of course | 09:44 |
| derekh_ | lifeless: your probably not there but if you are I'm having trouble finding where you kicked off deploy-ci-overcloud | 09:48 |
| *** yamahata has quit IRC | 09:48 | |
| *** martyntaylor has joined #tripleo | 09:50 | |
| openstackgerrit | Ghe Rivero proposed a change to openstack-infra/tripleo-ci: Test the upgrade codepath works as well. https://review.openstack.org/87758 | 09:59 |
| *** ramishra_ has joined #tripleo | 10:01 | |
| *** ramishra has quit IRC | 10:03 | |
| *** killer_prince has joined #tripleo | 10:03 | |
| *** killer_prince is now known as lazy_prince | 10:03 | |
| openstackgerrit | Alexis Lee proposed a change to openstack/tripleo-specs: Control mechanism for os-apply-config https://review.openstack.org/92356 | 10:04 |
| *** jistr has quit IRC | 10:06 | |
| proffalken | SpamapS, lifeless: when you're around, could one of you take a look at https://review.openstack.org/#/c/91324/ please (auto-collect check_mk_checks)? I've got +1, +2 and V, I just need the final steps! :) | 10:08 |
| *** andrearo1 is now known as andrearosa | 10:14 | |
| lifeless | derekh_: you're in luck. | 10:18 |
| lifeless | derekh_: screen -xr | 10:18 |
| lifeless | derekh_: on bastion.hp1.tripleo.org | 10:18 |
| derekh_ | lifeless: thanks, looks like I'm in the right place, overcloud controler seems to be broke, no python processes running so I'll see if I can figure that out first | 10:20 |
| derekh_ | dib-run-parts Wed Jul 9 05:32:14 UTC 2014 Running /opt/stack/os-config-refresh/configure.d/52-mysql-init | 10:20 |
| derekh_ | We are neither cluster initializer or joiner. Refusing to bootstrap mysql cluster until role is known. | 10:20 |
| lifeless | derekh_: ok - deploy failed, there could be lots of issues. Could be the heat template patch for vlans is at fault too - I dunno :). But I did see Ironic fail. | 10:20 |
| *** jerryz has quit IRC | 10:21 | |
| lifeless | derekh_: leaving in your hands :) | 10:21 |
| derekh_ | lifeless: see tab 5, anyways I know I'm looking at the right thing now, so will work on finding the problem | 10:21 |
| derekh_ | lifeless: thanks for popping back, have a good night | 10:21 |
| openstackgerrit | Alexis Lee proposed a change to openstack/tripleo-incubator: Wait longer for BM node registration https://review.openstack.org/102248 | 10:23 |
| *** jistr has joined #tripleo | 10:27 | |
| *** ramishra_ has quit IRC | 10:27 | |
| *** ramishra has joined #tripleo | 10:27 | |
| *** ramishra has quit IRC | 10:28 | |
| *** ramishra has joined #tripleo | 10:31 | |
| openstackgerrit | Jyoti proposed a change to openstack/os-cloud-config: test code https://review.openstack.org/105696 | 10:33 |
| openstackgerrit | Jyoti proposed a change to openstack/os-cloud-config: test case https://review.openstack.org/105697 | 10:38 |
| *** mrunge has quit IRC | 10:38 | |
| *** Shrews has quit IRC | 10:43 | |
| proffalken | 10:45 | |
| shadower | 10:46 | |
| *** Shrews has joined #tripleo | 10:47 | |
| *** yamahata has joined #tripleo | 10:47 | |
| *** mrunge has joined #tripleo | 10:48 | |
| *** rlandy has joined #tripleo | 10:49 | |
| openstackgerrit | A change was merged to openstack/tripleo-image-elements: Templatize keystone ssl keys/certs https://review.openstack.org/94609 | 10:56 |
| *** weshay has joined #tripleo | 10:57 | |
| openstackgerrit | Jan Provaznik proposed a change to openstack/tripleo-image-elements: Fix pacemaker cluster configuration https://review.openstack.org/105396 | 11:00 |
| openstackgerrit | Jan Provaznik proposed a change to openstack/tripleo-image-elements: Use pacemaker for running ceilometer central agent https://review.openstack.org/105397 | 11:00 |
| *** bmahalakshmi has quit IRC | 11:01 | |
| openstackgerrit | Gerry Drudy proposed a change to openstack/tripleo-image-elements: Add proxy-logging to proxy-server pipeline https://review.openstack.org/91533 | 11:02 |
| *** lazy_prince has quit IRC | 11:08 | |
| openstackgerrit | Giulio Fidente proposed a change to openstack/tripleo-heat-templates: add Horizon caches backend definition https://review.openstack.org/105289 | 11:12 |
| *** jcoufal-mobile has joined #tripleo | 11:13 | |
| *** mrunge is now known as mrunge_brb | 11:15 | |
| openstackgerrit | Giulio Fidente proposed a change to openstack/tripleo-heat-templates: add Horizon caches backend definition (memcached) https://review.openstack.org/105289 | 11:15 |
| openstackgerrit | Giulio Fidente proposed a change to openstack/tripleo-image-elements: add support for Horizon's caches backend configuration (memcached) https://review.openstack.org/105295 | 11:20 |
| *** rcarrill` has joined #tripleo | 11:21 | |
| *** rcarrillocruz has quit IRC | 11:23 | |
| *** rushiagr_ has quit IRC | 11:26 | |
| *** rushiagr__ has joined #tripleo | 11:26 | |
| *** derfgtya has joined #tripleo | 11:31 | |
| *** rcarrillocruz has joined #tripleo | 11:33 | |
| *** rcarrill` has quit IRC | 11:34 | |
| *** killer_prince has joined #tripleo | 11:41 | |
| *** killer_prince is now known as lazy_prince | 11:41 | |
| *** rushiagr__ has quit IRC | 11:42 | |
| *** mrunge_brb is now known as mrunge | 11:45 | |
| openstackgerrit | Ghe Rivero proposed a change to openstack-infra/tripleo-ci: Test the upgrade codepath works as well. https://review.openstack.org/87758 | 11:47 |
| *** jcoufal-mobile has quit IRC | 11:49 | |
| *** ci-overcloud has joined #tripleo | 11:50 | |
| ci-overcloud | ************** ci-overcloud complete status=1 ************ | 11:50 |
| *** ci-overcloud has quit IRC | 11:50 | |
| *** morazi has joined #tripleo | 11:52 | |
| *** dprince has joined #tripleo | 11:53 | |
| *** jistr is now known as jistr|english | 11:56 | |
| *** derfgtya has quit IRC | 11:56 | |
| *** shausy has quit IRC | 11:59 | |
| *** derfgtya has joined #tripleo | 12:01 | |
| *** derfgtya has quit IRC | 12:04 | |
| *** tzumainn has joined #tripleo | 12:04 | |
| *** derfgtya has joined #tripleo | 12:05 | |
| *** jistr|mobi has joined #tripleo | 12:07 | |
| *** sseago has joined #tripleo | 12:08 | |
| *** jdob has joined #tripleo | 12:09 | |
| *** lucasagomes is now known as lucas-hungry | 12:11 | |
| *** jistr|mobi has quit IRC | 12:11 | |
| *** jistr|english is now known as jistr | 12:11 | |
| *** ramishra has quit IRC | 12:12 | |
| *** rakesh_hs has quit IRC | 12:17 | |
| *** rcarrill` has joined #tripleo | 12:18 | |
| *** rcarrillocruz has quit IRC | 12:18 | |
| *** rcarrill` has quit IRC | 12:24 | |
| *** akuznetsov has quit IRC | 12:34 | |
| *** jdob has quit IRC | 12:37 | |
| *** ramishra has joined #tripleo | 12:38 | |
| *** jdob has joined #tripleo | 12:38 | |
| Ng | morning | 12:44 |
| *** julim has joined #tripleo | 12:48 | |
| *** chuckC has joined #tripleo | 12:49 | |
| openstackgerrit | Ryan Moore proposed a change to openstack/tripleo-heat-templates: Set vnc to use controller virtual_ip https://review.openstack.org/101563 | 12:50 |
| *** lucas-hungry is now known as lucasagomes | 12:55 | |
| *** ramishra has quit IRC | 12:57 | |
| *** ci-overcloud has joined #tripleo | 12:59 | |
| ci-overcloud | ************** ci-overcloud complete status=1 ************ | 12:59 |
| *** ci-overcloud has quit IRC | 12:59 | |
| *** rushiagr__ has joined #tripleo | 13:00 | |
| *** sballe has joined #tripleo | 13:01 | |
| TheJulia | good morning | 13:04 |
| *** morazi has quit IRC | 13:04 | |
| proffalken | slagle: any chance you could revist https://review.openstack.org/#/c/96448/ (creating databases from HEAT metadata) for me please? | 13:05 |
| *** morazi has joined #tripleo | 13:17 | |
| slagle | proffalken: sure | 13:20 |
| proffalken | thanks | 13:20 |
| proffalken | I hope I've answered the questions :) | 13:20 |
| *** mestery has joined #tripleo | 13:22 | |
| slagle | proffalken: when would you plan on installing the icinga-web/exim packages? | 13:24 |
| slagle | it sounds like at instance boot time, after the db's have been created | 13:24 |
| proffalken | slagle: the idea is that the packages get installed at image-build but the db is created and icinga/whatever is configured at run time | 13:32 |
| *** mrunge has quit IRC | 13:32 | |
| proffalken | I'm not too sure how else to approach the issue of software requiring a database but not knowing where/what it is | 13:33 |
| slagle | proffalken: ok, this is confusing me then: "I'm thinking particularly of systems such as Icinga-Web or Exim where you might want a database backend in place *before* you install the package so that any arguments given to dpkg-config or yum can be applied immediately on installation." | 13:34 |
| proffalken | bad wording on my part | 13:34 |
| proffalken | install != configure | 13:34 |
| proffalken | it's been a long few weeks | 13:34 |
| proffalken | The use case that sparked this is as follows: | 13:35 |
| proffalken | 1) Some packages require a database to be in place before they can be configured | 13:35 |
| proffalken | 2) Therefore we need to be able to create that database before we configure the software | 13:35 |
| proffalken | 3) It would be nice to configure the database in a standard, shared and structured manner across all packages that require it | 13:36 |
| proffalken | 4) HEAT works well across all packages for configuration management, let's use that to create the database | 13:36 |
| proffalken | (or something like that anyway) | 13:36 |
| proffalken | slagle: I'm happy to reword the commit message to reflect the above if you agree with the approach | 13:37 |
| *** pcrews has joined #tripleo | 13:40 | |
| *** nealph__ has joined #tripleo | 13:40 | |
| *** davidlenwell_ has joined #tripleo | 13:41 | |
| *** thegodfather has joined #tripleo | 13:43 | |
| *** lifeless1 has joined #tripleo | 13:43 | |
| *** GheRiver1 has joined #tripleo | 13:43 | |
| *** echohead has joined #tripleo | 13:43 | |
| *** jistr has quit IRC | 13:43 | |
| *** marios_ has joined #tripleo | 13:43 | |
| *** shadower_ has joined #tripleo | 13:43 | |
| *** SpamapS_ has joined #tripleo | 13:44 | |
| lxsli | any cores able to review this spec please? 2x+2 already https://review.openstack.org/#/c/94910/ | 13:44 |
| openstackgerrit | Eamonn O'Toole proposed a change to openstack/tripleo-image-elements: Add swift.proxy-memcache metadata to proxy-server.conf https://review.openstack.org/86312 | 13:46 |
| *** _sirushti has joined #tripleo | 13:46 | |
| *** mordred_ has joined #tripleo | 13:47 | |
| *** rlandy has quit IRC | 13:48 | |
| *** jprovazn has quit IRC | 13:48 | |
| *** nealph_ has quit IRC | 13:48 | |
| *** nijaba has quit IRC | 13:48 | |
| *** dkehn has quit IRC | 13:48 | |
| *** greghaynes has quit IRC | 13:48 | |
| *** panda has quit IRC | 13:48 | |
| *** shadower has quit IRC | 13:48 | |
| *** mordred has quit IRC | 13:48 | |
| *** uvirtbot has quit IRC | 13:48 | |
| *** fabbione has quit IRC | 13:48 | |
| *** NobodyCam has quit IRC | 13:48 | |
| *** mordred_ is now known as mordred | 13:48 | |
| *** GheRivero has quit IRC | 13:48 | |
| *** jogo has quit IRC | 13:48 | |
| *** dshulyak has quit IRC | 13:48 | |
| *** bhunter has quit IRC | 13:48 | |
| *** davidlenwell has quit IRC | 13:48 | |
| *** marios has quit IRC | 13:48 | |
| *** echohead_ has quit IRC | 13:48 | |
| *** hewbrocca has quit IRC | 13:48 | |
| *** SpamapS has quit IRC | 13:48 | |
| *** sirushti has quit IRC | 13:48 | |
| *** lifeless has quit IRC | 13:48 | |
| *** jomara has quit IRC | 13:48 | |
| *** anteaya has quit IRC | 13:48 | |
| *** _sirushti is now known as sirushti | 13:48 | |
| *** jogo has joined #tripleo | 13:49 | |
| *** andreaf has joined #tripleo | 13:51 | |
| *** eghobo has joined #tripleo | 13:52 | |
| *** yamahata has quit IRC | 13:52 | |
| *** yamahata has joined #tripleo | 13:53 | |
| *** rlandy has joined #tripleo | 13:54 | |
| *** jprovazn has joined #tripleo | 13:54 | |
| *** hewbrocca has joined #tripleo | 13:55 | |
| *** anteaya has joined #tripleo | 13:55 | |
| *** nijaba has joined #tripleo | 13:56 | |
| *** nijaba has quit IRC | 13:56 | |
| *** nijaba has joined #tripleo | 13:56 | |
| *** uvirtbot has joined #tripleo | 13:57 | |
| *** NobodyCam has joined #tripleo | 13:58 | |
| *** dkehn has joined #tripleo | 13:58 | |
| *** jomara has joined #tripleo | 13:58 | |
| *** dshulyak has joined #tripleo | 13:59 | |
| *** eghobo has quit IRC | 13:59 | |
| *** jistr has joined #tripleo | 14:00 | |
| *** bhunter has joined #tripleo | 14:01 | |
| *** mordred has quit IRC | 14:02 | |
| *** mordred has joined #tripleo | 14:02 | |
| slagle | proffalken: adding a good "why" to the commit message always helps :). i understand the intent now. not sure i'm totally for it as-is, but i can see its usefullness | 14:03 |
| *** jistr has quit IRC | 14:03 | |
| *** jistr has joined #tripleo | 14:04 | |
| *** thegodfather is now known as fabbione | 14:04 | |
| slagle | proffalken: i could see the software that requires the db doing the db creation itself | 14:04 |
| slagle | proffalken: so, have a common script for db creation (as you have here), but instead of just a single 50-mysql-databases creating all the databases, have the individual software elements create their own db | 14:05 |
| slagle | proffalken: i think there's also some ha implications here, assuming you only want the db created once from a bootstrap node | 14:06 |
| *** jcoufal has quit IRC | 14:06 | |
| proffalken | slagle: so this becomes more akin to "install-package" as opposed to being viewed as a config change?? | 14:09 |
| *** panda has joined #tripleo | 14:09 | |
| *** jcoufal has joined #tripleo | 14:10 | |
| slagle | proffalken: the software that requires the db could create it as part of o-r-c | 14:10 |
| *** greghaynes has joined #tripleo | 14:10 | |
| proffalken | sorry, not following. Where would the variables that are currently (based on the most recent patchset) be configured and set if we're not doing it in heat? | 14:11 |
| *** amitpp has quit IRC | 14:12 | |
| slagle | proffalken: you could still set needed vars in heat...it's just that the individual software elements that required a db could create it in their own o-r-c script | 14:13 |
| proffalken | ok, but why is that advantageous over the current solution? | 14:13 |
| proffalken | The current patchset means that you have one heat variable to set (and read from $wherever) to create ALLTHEDATABASES | 14:14 |
| slagle | more explictness about the db dependency | 14:15 |
| proffalken | you seem to be suggesting that each element that requires a database would need to add its own variable and set of values to the heat templates, leaving us with a load of variables that are connected but not grouped | 14:15 |
| proffalken | ok, | 14:15 |
| proffalken | that's a fair point, however I would *hope* that we can trust people to call databases by sensible logical names? ;) | 14:16 |
| slagle | instead of leaving it "in the hand of the deployer" to be sure to create the db by specifying the values, the element that requires it would do it | 14:16 |
| slagle | not saying this *the* way to go :). merely a suggestion | 14:16 |
| proffalken | ok, so let's continue to explore this approach :) | 14:16 |
| proffalken | I am a user | 14:16 |
| proffalken | I have an element that installs Django | 14:17 |
| proffalken | I need a database for Django | 14:17 |
| proffalken | I create my element and have it call the create-db-from-heat script | 14:17 |
| proffalken | and then I configure an element specific section of my heat template to provide things such as DB name/username etc? | 14:17 |
| proffalken | or have I misunderstood your approach? | 14:18 |
| slagle | yea, that's basically it | 14:18 |
| proffalken | ok | 14:18 |
| proffalken | so my approach basically removes the "I create my element and have it call the create-db-from-heat script" step | 14:19 |
| lxsli | Do you need the element-specific section of Heat template or could you just use passthrough? | 14:19 |
| proffalken | slagle: just been chatting this over with lxsli and I think we may have an option that satisfies both parties... ;) | 14:27 |
| *** dprince has quit IRC | 14:27 | |
| *** pblaho has quit IRC | 14:27 | |
| proffalken | The idea is that we have a "helper script" in mysql-common that is called with a number of arguments to setup the database | 14:28 |
| proffalken | this in turn is called by a more general "create_database" function from within an element | 14:28 |
| proffalken | the element gets its configuration from either HEAT or passthrough metadata and executes the create_database function accordingly | 14:29 |
| proffalken | this means that in future, the create_database function take take a "db type" as an argument and call the appropriate creation helper-script | 14:29 |
| proffalken | meaning that an element can request one or more databases that are MySQL/Postgresql/MongoDB/Whatever | 14:30 |
| lxsli | HEAT metadata == passthrough metadata | 14:30 |
| lxsli | and I suggested the create_db abstraction but in retrospect I don't think it's useful | 14:30 |
| lxsli | otherwise +1 | 14:30 |
| *** lazy_prince is now known as killer_prince | 14:33 | |
| proffalken | ok, further talking and I've almost been persuaded by lxsli that the create_db abstraction is a good thing ;) does this make more sense as an approach slagle? | 14:37 |
| lxsli | s/good/bad/ >.< | 14:37 |
| proffalken | ah, yeah, that :p | 14:37 |
| *** dkehnx1 has joined #tripleo | 14:41 | |
| Ng | lifeless1: FYI, I'm working on shifting tripleo.org to HP Cloud DNSaaS with a delegated user account that can manage it, without also being able to do terrible things to my other services/instances | 14:46 |
| *** morganfainberg_Z is now known as morganfainberg | 14:46 | |
| *** vinsh has joined #tripleo | 14:47 | |
| *** ci-overcloud has joined #tripleo | 14:47 | |
| ci-overcloud | ************** ci-overcloud complete status=1 ************ | 14:47 |
| *** ci-overcloud has quit IRC | 14:47 | |
| *** dprince has joined #tripleo | 14:48 | |
| proffalken | slagle, lxsli I'm away now effecitvely until Monday but I'll pick this up again on my return. | 14:50 |
| *** spzala has joined #tripleo | 14:52 | |
| *** noslzzp has joined #tripleo | 14:56 | |
| *** jprovazn has quit IRC | 14:59 | |
| *** untriaged-bot has joined #tripleo | 15:00 | |
| untriaged-bot | Untriaged bugs so far: | 15:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1339047 | 15:00 |
| uvirtbot | Launchpad bug 1339047 in tripleo "db sync must be run once from the bootstrap node" [Undecided,In progress] | 15:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1336115 | 15:00 |
| uvirtbot | Launchpad bug 1336115 in tripleo "instances no longer running post-upgrade" [Undecided,In progress] | 15:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1335105 | 15:00 |
| *** untriaged-bot has quit IRC | 15:00 | |
| uvirtbot | Launchpad bug 1335105 in tuskar-ui "Devtest init script: sourcing tripleorc fails" [Undecided,New] | 15:00 |
| *** morazi has quit IRC | 15:01 | |
| *** weshay has quit IRC | 15:02 | |
| *** rushiagr__ has quit IRC | 15:08 | |
| *** rushiagr__ has joined #tripleo | 15:08 | |
| *** rdopieralski has quit IRC | 15:12 | |
| *** morazi has joined #tripleo | 15:15 | |
| *** SpamapS_ is now known as SpamapS | 15:21 | |
| *** SpamapS has quit IRC | 15:21 | |
| *** SpamapS has joined #tripleo | 15:21 | |
| *** sballe has quit IRC | 15:21 | |
| shadower_ | local devtest completed with stevebaker's HOT patches (plus a few fixes) \o/ | 15:26 |
| *** shadower_ is now known as shadower | 15:26 | |
| *** killer_prince is now known as lazy_prince | 15:27 | |
| *** ramishra has joined #tripleo | 15:40 | |
| lxsli | lifeless1: added to https://etherpad.openstack.org/p/tripleo-incubator-rationalise-ui , please would you see if I'm barking up the wrong tree? | 15:44 |
| *** eghobo has joined #tripleo | 15:45 | |
| *** marun has joined #tripleo | 15:46 | |
| openstackgerrit | A change was merged to openstack/diskimage-builder: Add global exclusions to dib-lint https://review.openstack.org/104255 | 15:51 |
| *** martyntaylor has left #tripleo | 15:55 | |
| SpamapS | proffalken: there are two approaches to that | 16:00 |
| SpamapS | proffalken: either dpkg --unpack exim_blah.deb then dpkg --configure exim after boot.. | 16:01 |
| SpamapS | proffalken: _or_ you disable exim startup after it is fully configured during image build, and wipe out any state that is "wrong" | 16:01 |
| *** martyntaylor has joined #tripleo | 16:03 | |
| *** ramishra has quit IRC | 16:03 | |
| mordred | btw - we've run in to that very same issue in infra | 16:05 |
| *** markmc has quit IRC | 16:05 | |
| mordred | our current hack is a post-install script that un-disables service start, then install database things we need | 16:06 |
| mordred | we're not thrilled about it | 16:06 |
| *** rushiagr__ is now known as rushiagr | 16:07 | |
| * mordred would love an optional stage that would boot an image in an lxc container and let some commands be run on it live ... | 16:09 | |
| mordred | SpamapS: ^^ I can haz pony? | 16:10 |
| *** martyntaylor has quit IRC | 16:12 | |
| openstackgerrit | Haneef Ali proposed a change to openstack/tripleo-image-elements: Configure keystone with apache https://review.openstack.org/102304 | 16:12 |
| SpamapS | mordred: hm | 16:16 |
| SpamapS | mordred: arguably anything that needs a live running system should be done at boot and considered "state" | 16:16 |
| *** rushiagr has quit IRC | 16:16 | |
| SpamapS | mordred: the fact that Debian wants to do all this crazy stuff at package configuration time just makes that annoying hard to separate. | 16:17 |
| *** nati_ueno has joined #tripleo | 16:19 | |
| mordred | SpamapS: yah - I mean, I hear that argument ... | 16:21 |
| mordred | SpamapS: but it's also _super_ wasteful to do so in this case | 16:21 |
| mordred | since we're spinning up 20k copies of this image a day | 16:21 |
| mordred | and the database init functions are identical | 16:21 |
| mordred | so that means we're spending 20k x running the db init functions | 16:21 |
| SpamapS | mordred: what makes it bad to do in a chroot? | 16:21 |
| mordred | when we could have done that at image compile time | 16:21 |
| mordred | SpamapS: it's mysql | 16:22 |
| mordred | it is configured to listen on network ports | 16:22 |
| SpamapS | mordred: using a throw-away build box you can start mysql | 16:22 |
| SpamapS | which is really how dib has to work anyway because of the sudo abuse. | 16:22 |
| mordred | if I use a throwaway build box, I lose the ability to have local cache | 16:22 |
| *** martyntaylor has joined #tripleo | 16:23 | |
| SpamapS | http_proxy=http://some.permanent.cache/ ? | 16:23 |
| mordred | local cache | 16:23 |
| mordred | in the .cache dir | 16:23 |
| mordred | all that great work that has gone in to that | 16:23 |
| SpamapS | only slightly faster than fetching via squid really | 16:23 |
| mordred | I'm fetching 300 git repos | 16:23 |
| SpamapS | ccache is a notable exception | 16:23 |
| SpamapS | and git repos tend to be https .. so yeah.. ok | 16:24 |
| SpamapS | so throw away build LXC + bind mounts? thats something I coudl get behind for built into dib | 16:24 |
| mordred | yeah - that would be great - especially if dib knew how to do it | 16:24 |
| *** dshulyak_ has joined #tripleo | 16:24 | |
| SpamapS | yeah just have dib do it always | 16:24 |
| mordred | that, combined with a stage where init scripts are not disabled | 16:25 |
| mordred | and I think we'd be able to get rid of most of our crazy hacks | 16:25 |
| SpamapS | mordred: anything above 40 in cleanup.d fits that bill. ;) | 16:26 |
| SpamapS | elements/dpkg/cleanup.d/40-unblock-daemons:sudo rm $TARGET_ROOT/usr/sbin/policy-rc.d | 16:26 |
| mordred | SpamapS: :) | 16:26 |
| SpamapS | though you are actually allowed to start daemons manually too | 16:26 |
| SpamapS | doing a full boot feels like a violation | 16:26 |
| mordred | yah. although I will admit that running "puppet apply 'class {'slave_db': }" in cleanup.d is yuck | 16:26 |
| mordred | SpamapS: honestly, I think an option that lets us not block daemons would be a friendly | 16:27 |
| mordred | becaus while I understand the theory | 16:27 |
| mordred | it's not actually helpful in this case | 16:27 |
| mordred | (kinda like the cloud-init element beind an ubuntu dependency) | 16:28 |
| mordred | clarkb: ^^ how does the above strike you? | 16:29 |
| *** hewbrocca has quit IRC | 16:31 | |
| *** dsneddon_away is now known as dsneddon | 16:31 | |
| SpamapS | mordred: sorry I don't know the cloud-init bit. But not blocking daemons has the nasty side effect that they get started and do their initialization, write syslogs, etc. etc... this is undesirable for other reasons.. off the top of my head, saving things like the local host name inside the image's state is something we want to avoid. | 16:32 |
| *** hewbrocca has joined #tripleo | 16:32 | |
| mordred | SpamapS: yup. grok. it should definitely be an opt-in | 16:32 |
| mordred | SpamapS: but as a user of dib in this case, I can tell you I categorically do not care about that side effect - dancing around it is way more painful and crufty | 16:33 |
| SpamapS | so whats needed then, is a dpkg-unblocked-daemons that provides dpkg and then you manually specify dpkg-unblocked-daemons and I believe that will work | 16:34 |
| *** martyntaylor has left #tripleo | 16:35 | |
| mordred | that seems like it will also dance into strange - because I kinda need this to work with ubuntu and centos and fedora | 16:35 |
| mordred | oh - I guess they dont start daemons anyway | 16:35 |
| mordred | so it's safer, right? | 16:35 |
| *** morazi has quit IRC | 16:36 | |
| *** jistr has quit IRC | 16:37 | |
| *** rameshg87 has quit IRC | 16:37 | |
| *** jcoufal has quit IRC | 16:39 | |
| SpamapS | mordred: right, only debian's children have to deal with invoke-rc.d being invoked during package management | 16:41 |
| SpamapS | mordred: everybody else gets that software delivery != software configuration. | 16:41 |
| derekh_ | lifeless1: first attempt to redeploy the ci-overcloud failed because of a missing NeutronPublicInterfaceTagIP , I added that (and for the moment hardcoded to 138.35.77.5 ) see tripleo-incubator and tripleo-heat-templates on the bastion | 16:42 |
| derekh_ | Its now erroring out on editing the default route | 16:43 |
| derekh_ | + ip route prepend default via 138.35.77.1 -> RTNETLINK answers: Network is unreachable | 16:43 |
| *** GheRiver1 is now known as GheRivero | 16:43 | |
| derekh_ | lifeless1: and I'm a bit baffled cause I can ping that address | 16:43 |
| derekh_ | lifeless1: also scaled down to 2 compute nodes , I've deployed several times today trying various things, each time on of the compute nodes goes to ERROR state | 16:44 |
| derekh_ | lifeless1: and have also noticed controller loosing all of its routes ( over textcons ), but I think that might be a sideeffect of the route setup barfing | 16:46 |
| *** morazi has joined #tripleo | 16:49 | |
| openstackgerrit | Tomas Sedovic proposed a change to openstack/tripleo-heat-templates: Convert Fn::Select to extended get_attr https://review.openstack.org/105347 | 16:53 |
| openstackgerrit | Tomas Sedovic proposed a change to openstack/tripleo-heat-templates: merge.py use constants for key names https://review.openstack.org/105326 | 16:53 |
| openstackgerrit | Tomas Sedovic proposed a change to openstack/tripleo-heat-templates: merge.py --hot to process and generate HOT https://review.openstack.org/105327 | 16:53 |
| openstackgerrit | Tomas Sedovic proposed a change to openstack/tripleo-heat-templates: WIP Make target to port to HOT https://review.openstack.org/105329 | 16:53 |
| openstackgerrit | Tomas Sedovic proposed a change to openstack/tripleo-heat-templates: Port all templates to HOT https://review.openstack.org/105328 | 16:53 |
| *** derekh_ has quit IRC | 16:56 | |
| *** amitpp has joined #tripleo | 16:59 | |
| gfidente | greghaynes, you around? | 17:01 |
| *** dshulyak_ has quit IRC | 17:03 | |
| shadower | >.< gerrit, I wanted to upload just the last patch in the series | 17:04 |
| clarkb | shadower: it will upload any commits that have updated by default as it breaks when you upload a change with a dependency that gerrit doesn't have a change for | 17:06 |
| shadower | yeah I know | 17:06 |
| shadower | doesn't make it any less annoying | 17:06 |
| *** marun has quit IRC | 17:07 | |
| *** marun has joined #tripleo | 17:09 | |
| *** marun has quit IRC | 17:16 | |
| *** marun has joined #tripleo | 17:21 | |
| adam_g | mkerrin, around? | 17:23 |
| *** Penick has joined #tripleo | 17:26 | |
| *** lucasagomes is now known as lucas-dinner | 17:27 | |
| *** dshulyak_ has joined #tripleo | 17:27 | |
| *** cadenzajon has joined #tripleo | 17:31 | |
| *** regebro has quit IRC | 17:34 | |
| *** blamar has quit IRC | 17:35 | |
| *** jerryz has joined #tripleo | 17:38 | |
| *** rushiagr has joined #tripleo | 17:43 | |
| *** davidlenwell_ is now known as davidlelnwell | 17:44 | |
| *** davidlelnwell is now known as davidlelnwell__ | 17:44 | |
| *** davidlelnwell__ is now known as davidlelnwell | 17:44 | |
| *** davidlelnwell is now known as davidlenwell | 17:44 | |
| *** ChanServ sets mode: +v davidlenwell | 17:45 | |
| *** saurabhs has joined #tripleo | 17:45 | |
| greghaynes | gfidente: hey, yep | 17:46 |
| openstackgerrit | Ghe Rivero proposed a change to openstack-infra/tripleo-ci: Test the upgrade codepath works as well. https://review.openstack.org/87758 | 17:51 |
| gfidente | greghaynes, thanks can I add you to a couple of reviews? | 17:54 |
| gfidente | :) | 17:54 |
| greghaynes | sure | 17:54 |
| greghaynes | but pointing them out might be a bit more effective, im on a lot | 17:54 |
| gfidente | https://review.openstack.org/#/c/105289/ and https://review.openstack.org/#/c/105295/ | 17:54 |
| *** lazy_prince is now known as killer_prince | 17:55 | |
| *** weshay has joined #tripleo | 17:56 | |
| *** mestery has quit IRC | 18:00 | |
| *** mestery has joined #tripleo | 18:00 | |
| gfidente | greghaynes, also, is there anything open to balance rmq and mysql via haproxy? | 18:02 |
| *** rushiagr has quit IRC | 18:02 | |
| greghaynes | TheJulia: ^ is working on the mysql one right now | 18:03 |
| greghaynes | gfidente: do you know if we already install memcached? | 18:04 |
| gfidente | greghaynes, I added it as a dep for horizon yes | 18:05 |
| gfidente | we had the element already | 18:05 |
| greghaynes | Yes, im just curious if were already using it (and therefore whether its currently tested) | 18:05 |
| *** jcoufal has joined #tripleo | 18:05 | |
| greghaynes | If not we might need to make sure were not adding too much to our mem usage | 18:06 |
| gfidente | so I've one built with 8G nodes | 18:06 |
| gfidente | but I'm building some amd64 with 3G | 18:07 |
| gfidente | will let you know eventually | 18:07 |
| greghaynes | ok | 18:07 |
| *** amitpp has quit IRC | 18:09 | |
| *** amitpp has joined #tripleo | 18:10 | |
| *** marun has quit IRC | 18:11 | |
| *** amitpp has quit IRC | 18:11 | |
| *** marun has joined #tripleo | 18:11 | |
| openstackgerrit | James Slagle proposed a change to openstack/tripleo-specs: Configurable /mnt/state https://review.openstack.org/94876 | 18:19 |
| TheJulia | gfidente: inserting mysql has introduced some fun issues that I've been unwinding :( | 18:21 |
| *** eghobo has quit IRC | 18:21 | |
| *** eghobo has joined #tripleo | 18:21 | |
| gfidente | TheJulia, well if you have open reviews please ping me or add me there | 18:22 |
| gfidente | I am testing the horizon patches right now and trying to get a list of known issues we should solve for HA | 18:23 |
| gfidente | so it'd be nice to see if other are picking other 'known stuff' | 18:24 |
| gfidente | *if others are also working known stuff | 18:24 |
| TheJulia | gfidente: I'm hoping to begin uploading the reviews later today, will add you | 18:25 |
| *** morganfainberg is now known as morganfainberg_Z | 18:25 | |
| gfidente | thanks! | 18:25 |
| *** pelix has quit IRC | 18:27 | |
| TheJulia | No problem | 18:27 |
| *** morazi has quit IRC | 18:32 | |
| lifeless1 | morning | 18:32 |
| *** lifeless1 is now known as lifeless | 18:33 | |
| lifeless | jogo: so that nova bug | 18:33 |
| openstackgerrit | Jon-Paul Sullivan (jp_at_hp) proposed a change to openstack/diskimage-builder: Lock around source repositories setup https://review.openstack.org/104150 | 18:37 |
| openstackgerrit | Haneef Ali proposed a change to openstack/tripleo-image-elements: Configure keystone with apache https://review.openstack.org/102304 | 18:40 |
| *** morazi has joined #tripleo | 18:47 | |
| mordred | SpamapS: if I were to do the throwaway lxc option - any thoughts on where/how the best place to slot that in would be? | 18:48 |
| *** mestery has quit IRC | 18:48 | |
| *** mestery has joined #tripleo | 18:51 | |
| *** saurabhs has quit IRC | 18:52 | |
| greghaynes | huh, my networking is acting funky and im getting DHCPDISCOVER(tap9bc2eced-06) 00:e7:77:b0:6f:4a no address available in syslog on my seed... | 18:53 |
| openstackgerrit | James Slagle proposed a change to openstack/tripleo-image-elements: Move rabbitmq-server cluster port https://review.openstack.org/91704 | 18:53 |
| openstackgerrit | James Slagle proposed a change to openstack/tripleo-image-elements: Stop using sysctl -p in sysctl-set-value https://review.openstack.org/101784 | 18:53 |
| openstackgerrit | James Slagle proposed a change to openstack/tripleo-image-elements: Add a sysctl-append-value script https://review.openstack.org/101117 | 18:53 |
| *** jang1 has joined #tripleo | 18:56 | |
| *** jp_at_hp has quit IRC | 18:56 | |
| *** jang1 has quit IRC | 19:01 | |
| lifeless | greghaynes: been doing frequent deploys? You may have hit the doesn't-hand-out-addresses bug where dnsmasq still has lease active if the release packet doesn't reach it | 19:02 |
| lifeless | greghaynes: see syslog and there is a bug open on neutron | 19:02 |
| greghaynes | ah, fun | 19:02 |
| openstackgerrit | Richard Su proposed a change to openstack/tripleo-image-elements: Shift sysctl.d writing code to a new script https://review.openstack.org/101784 | 19:10 |
| openstackgerrit | Adam Gandelman proposed a change to openstack/tripleo-image-elements: Move libvirt's qemu configuration dir to state fs https://review.openstack.org/104407 | 19:13 |
| openstackgerrit | Adam Gandelman proposed a change to openstack/tripleo-image-elements: Move libvirt's qemu configuration dir to state fs https://review.openstack.org/104407 | 19:15 |
| *** panda has quit IRC | 19:20 | |
| *** jtomasek has quit IRC | 19:21 | |
| openstackgerrit | Richard Su proposed a change to openstack/tripleo-image-elements: Add a sysctl-append-value script https://review.openstack.org/101117 | 19:21 |
| *** panda has joined #tripleo | 19:21 | |
| openstackgerrit | A change was merged to openstack/tripleo-image-elements: Sync mariadb cluster init with mysql element https://review.openstack.org/102771 | 19:25 |
| openstackgerrit | Richard Su proposed a change to openstack/tripleo-image-elements: Move rabbitmq-server cluster port https://review.openstack.org/91704 | 19:31 |
| *** blamar has joined #tripleo | 19:32 | |
| SpamapS | mordred: so I think you'd want to have lxc-create call a dib entry point as a template to build the chroot, and then lxc-execute in place of chroot. | 19:33 |
| *** jtomasek has joined #tripleo | 19:36 | |
| *** morazi has quit IRC | 19:38 | |
| rwsu | lifeless: when you have time, can you review https://review.openstack.org/#/c/101784/ again, it has a bunch of dependencies waiting on it | 19:38 |
| *** amitpp has joined #tripleo | 19:39 | |
| mordred | SpamapS: yeah - I just poked around with this a little bit and words you are saying sound like words I was thinking - except for with better words | 19:43 |
| *** mestery has quit IRC | 19:46 | |
| *** mestery has joined #tripleo | 19:47 | |
| *** mestery has quit IRC | 19:49 | |
| *** amitpp has quit IRC | 19:52 | |
| *** morazi has joined #tripleo | 19:52 | |
| *** jcoufal has quit IRC | 19:56 | |
| *** morganfainberg_Z is now known as morganfainberg | 19:57 | |
| *** eguz has joined #tripleo | 19:59 | |
| openstackgerrit | Adam Gandelman proposed a change to openstack/tripleo-image-elements: Centralize all db migrations to boot-stack element https://review.openstack.org/105640 | 20:00 |
| *** Penick has quit IRC | 20:00 | |
| *** eguz has quit IRC | 20:02 | |
| *** eghobo has quit IRC | 20:02 | |
| *** eguz has joined #tripleo | 20:02 | |
| openstackgerrit | Julia Kreger proposed a change to openstack/tripleo-image-elements: Network init order and keepalived fixes https://review.openstack.org/105862 | 20:04 |
| *** Penick has joined #tripleo | 20:04 | |
| openstackgerrit | Ghe Rivero proposed a change to openstack-infra/tripleo-ci: Test the upgrade codepath works as well. https://review.openstack.org/87758 | 20:10 |
| jogo | lifeless: oh right | 20:12 |
| jogo | I was going through old bugs and stumbled on that one | 20:12 |
| openstackgerrit | A change was merged to openstack-infra/tripleo-ci: Remove support for legacy testenv-client https://review.openstack.org/97518 | 20:17 |
| *** nati_uen_ has joined #tripleo | 20:18 | |
| *** mestery has joined #tripleo | 20:20 | |
| *** nati_ueno has quit IRC | 20:20 | |
| greghaynes | huh. Build of instance 83c6d9d0-5758-4b1d-a939-cac2d0f7c14e was re-scheduled: Insufficient compute resources: Free memory 0.00 MB < requested 4096 MB. in nova-conductor but baremetal-node-list and hypervisor-stats show I have 10 free nodes | 20:20 |
| SpamapS | greghaynes: how much free memory tho? | 20:21 |
| greghaynes | | free_ram_mb | 40960 | | 20:21 |
| greghaynes | oh | 20:22 |
| *** rcarrillocruz has joined #tripleo | 20:22 | |
| greghaynes | oh | 20:22 |
| greghaynes | on the box | 20:22 |
| greghaynes | how the | 20:22 |
| SpamapS | whats in the box whats in the box?! | 20:22 |
| greghaynes | Total: 64421 Used: 63756 | 20:22 |
| greghaynes | how did I use that much rams | 20:23 |
| greghaynes | but 24g in cache | 20:24 |
| greghaynes | so... | 20:24 |
| *** ccrouch has quit IRC | 20:24 | |
| * greghaynes confused | 20:24 | |
| *** mestery has quit IRC | 20:26 | |
| openstackgerrit | A change was merged to openstack/tripleo-image-elements: Specify nova configuration file with --config-dir https://review.openstack.org/97785 | 20:27 |
| *** rbrady has quit IRC | 20:29 | |
| *** noslzzp has quit IRC | 20:36 | |
| openstackgerrit | Julia Kreger proposed a change to openstack/tripleo-image-elements: Add local_bind flag to my.cf https://review.openstack.org/105870 | 20:37 |
| *** ccrouch has joined #tripleo | 20:43 | |
| *** dshulyak_ has quit IRC | 20:49 | |
| openstackgerrit | A change was merged to openstack/os-cloud-config: Correctly deal with nodes with no pm_password set https://review.openstack.org/100400 | 20:51 |
| greghaynes | thats wierd, delete nodes and re-build and they all succeed | 20:55 |
| greghaynes | I wonder if theres some kind of race issue when we ask to build a lot of nodes at once | 20:56 |
| *** untriaged-bot has joined #tripleo | 21:00 | |
| untriaged-bot | Untriaged bugs so far: | 21:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1335105 | 21:00 |
| uvirtbot | Launchpad bug 1335105 in tuskar-ui "Devtest init script: sourcing tripleorc fails" [Undecided,New] | 21:00 |
| *** untriaged-bot has quit IRC | 21:00 | |
| *** jdob has quit IRC | 21:00 | |
| GheRivero | greghaynes: don't know if related but sometimes hypervisor stats are not fully registred/updated https://review.openstack.org/#/c/102248/ | 21:01 |
| greghaynes | ooo | 21:01 |
| greghaynes | that does sound like it | 21:01 |
| *** jtomasek has quit IRC | 21:09 | |
| *** dprince has quit IRC | 21:09 | |
| openstackgerrit | Julia Kreger proposed a change to openstack/tripleo-heat-templates: Change overcloud to use VIP for MySQL https://review.openstack.org/105878 | 21:10 |
| *** julim has quit IRC | 21:12 | |
| lifeless | rwsu: thanks for the ping | 21:21 |
| *** Penick has quit IRC | 21:23 | |
| *** Penick has joined #tripleo | 21:32 | |
| *** bgmccollum has joined #tripleo | 21:37 | |
| *** bgmccollum has left #tripleo | 21:38 | |
| *** Penick has quit IRC | 21:40 | |
| *** sballe has joined #tripleo | 21:42 | |
| *** mestery has joined #tripleo | 21:42 | |
| *** rlandy has quit IRC | 21:50 | |
| openstackgerrit | Julia Kreger proposed a change to openstack/tripleo-image-elements: Add detection of mysql restart failure https://review.openstack.org/104569 | 21:57 |
| vinsh | wow.. Julia is on a role with some great reviews today +1 | 22:09 |
| vinsh | Going to have to test some of this out, see how it compares to the versions we had for this. | 22:11 |
| *** dsneddon has quit IRC | 22:11 | |
| *** echohead has left #tripleo | 22:14 | |
| *** jang1 has joined #tripleo | 22:14 | |
| greghaynes | SpamapS: My resource controller1AllNodesConfig is wedged in CREATE_IN_PROGRESS, any idea what to poke at to debug? | 22:23 |
| greghaynes | doesnt look like updated_time is changing... | 22:23 |
| *** Penick has joined #tripleo | 22:23 | |
| greghaynes | and every other resource for overcloud is CREATE_COMPLETE... | 22:24 |
| *** weshay has quit IRC | 22:25 | |
| ccrouch | whats the general feeling about specifying values in .conf files in elements, that just match the existing defaults e.g. | 22:25 |
| ccrouch | https://github.com/openstack/tripleo-image-elements/blob/master/elements/nova/os-apply-config/etc/nova/nova.conf#L96 | 22:25 |
| ccrouch | and | 22:25 |
| ccrouch | https://github.com/openstack/nova/blob/8e4532e967f15b0e49e1bd4c7d66d02881687ffd/nova/volume/__init__.py#L25 | 22:25 |
| greghaynes | ccrouch: whats the benefit over just not specifying? | 22:26 |
| greghaynes | seems like a downside is potential drift | 22:26 |
| ccrouch | greghaynes: i dont see any benefit from specifying it. There are 8m other nova settings we dont give a default for in our configs | 22:29 |
| ccrouch | maybe at onetime the default was different? | 22:30 |
| greghaynes | ah | 22:31 |
| lifeless | ccrouch: I'm very much against having a setting in our conf that statically says what e.g. nova's default is | 22:32 |
| lifeless | ccrouch: its immediate debt we have to pay down on every release | 22:32 |
| *** weshay has joined #tripleo | 22:33 | |
| *** weshay has quit IRC | 22:33 | |
| *** eguz has quit IRC | 22:33 | |
| ccrouch | lifeless: great. if i get a chance I'll take it out | 22:34 |
| ccrouch | its been in since the original commit: https://github.com/openstack/tripleo-image-elements/commit/08098865718e75b67fada55eeba31ab3e77db322 | 22:34 |
| lifeless | ccrouch: aiee yeah | 22:36 |
| *** boris-42 has quit IRC | 22:37 | |
| *** boris-42 has joined #tripleo | 22:39 | |
| *** eghobo has joined #tripleo | 22:49 | |
| lifeless | bnemec: hey | 22:52 |
| lifeless | bnemec: thought I'd touch base on the multi-hypervisor stuff | 22:52 |
| lifeless | bnemec: I'm super keen to see that in; are you actively hacking on it? | 22:53 |
| *** andreaf has quit IRC | 22:53 | |
| bnemec | lifeless: I think slagle was doing the multi-hypervisor stuff. I was testing/quintupleo. | 22:53 |
| *** lucas-dinner has quit IRC | 22:55 | |
| lifeless | bnemec: man, I'm terrible aren't I. | 22:55 |
| lifeless | bnemec: so I'm keen on that too :) | 22:55 |
| lifeless | bnemec: Sorry about my epic confusion. | 22:55 |
| bnemec | lifeless: Yeah, it's not like you have anything else to keep track of. :-) | 22:55 |
| boris-42 | lifeless btw pinging you to say | 22:55 |
| boris-42 | lifeless did you have a time to take a look at osprofiler | 22:55 |
| boris-42 | lifeless and hi=) | 22:55 |
| *** noslzzp has joined #tripleo | 22:56 | |
| bnemec | lifeless: I haven't made much progress on either of my tripleo specs, which for quintupleo is going to be a problem because it's unlikely I'll get the necessary specs approved in the other projects for this cycle, if I understand the schedule correctly. | 22:56 |
| bnemec | I suppose I could start some PoC changes and just not submit them (or submit WIP). | 22:57 |
| lifeless | bnemec: so, I'd suggest you do PoC and specs in parallel | 22:57 |
| *** mestery has quit IRC | 22:57 | |
| lifeless | bnemec: or perhaps - ask the nova & neutron PTLs to weigh in on which things they want specs for | 22:58 |
| bnemec | lifeless: Yeah, I can do that, I'm just concerned about the timing with the specs at this point. | 22:58 |
| lifeless | boris-42: oh right, what tree is it in? | 22:58 |
| *** cadenzajon has quit IRC | 22:58 | |
| *** Penick has quit IRC | 23:00 | |
| bnemec | lifeless: Okay, gotta run, but I'll try to put some feelers out to the other projects yet this week. I know someone (mikal?) had mentioned that Rackspace has some way to PXE boot instances already, so I need to look into that too. | 23:05 |
| lifeless | yup | 23:05 |
| openstackgerrit | Adam Gandelman proposed a change to openstack/tripleo-image-elements: Centralize all db migrations to boot-stack element https://review.openstack.org/105640 | 23:07 |
| *** jang1 has quit IRC | 23:12 | |
| *** morganfainberg is now known as morganfainberg_Z | 23:17 | |
| *** marun has quit IRC | 23:21 | |
| greghaynes | adam_g: ^ not sure what you mean with incremental migrations on image upgrade | 23:27 |
| greghaynes | curious what the failing case that solves | 23:28 |
| adam_g | greghaynes, on a new db, db sync runs migrations 0 -> head. if we deploy new code via a new image / stack-updat / etc we need to run db syncs again to migrate from $last_head -> new_head | 23:29 |
| greghaynes | Does that need to happen on the first node to get the new image? | 23:30 |
| greghaynes | Im thinking, if were doing rolling updates its not guaranteed that the first node to get a new image is necessarially going to be a 'bootstrap-node' | 23:31 |
| greghaynes | er, is not | 23:31 |
| adam_g | greghaynes, it should be happening on the 'leader' node. if thats not the bootstrap-node, then there should be another way to query that? | 23:31 |
| adam_g | greghaynes, i need to go run an errand ill be back in a bit | 23:32 |
| greghaynes | ok | 23:32 |
| greghaynes | Ill comment on patch now that im more clear | 23:32 |
| *** ccrouch has quit IRC | 23:48 | |
| *** marun has joined #tripleo | 23:51 | |
| slagle | lifeless: https://review.openstack.org/#/c/94586/ can always use a review :) | 23:55 |
| slagle | i've got some WIP patches | 23:56 |
| openstackgerrit | Julia Kreger proposed a change to openstack/tripleo-image-elements: Network init order and keepalived fixes https://review.openstack.org/105862 | 23:59 |
| lifeless | slagle: oh, its still in edit... doh :) | 23:59 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!
