| mark-casey | I know you can just pass a regex as the last parameter of kolla-build after passing --no-cache, but I don't remember if a new image of the same tag will automatically superceed as the latest for that image | 00:00 |
|---|---|---|
| *** zhurong has quit IRC | 00:04 | |
| *** dwalsh has quit IRC | 00:05 | |
| Pavo | is there a log of the output of the build process? because I could have sworn as I was watching it build images I seen some errors on the horizon build | 00:06 |
| mark-casey | other than scrollback I don't think so, you may be able to pass a file name for logging but I don't think there is an implicit one | 00:07 |
| Pavo | ah ok | 00:08 |
| *** jamesdenton has quit IRC | 00:08 | |
| *** kbaegis has left #openstack-kolla | 00:13 | |
| *** kbaegis has joined #openstack-kolla | 00:28 | |
| *** diogogmt has quit IRC | 00:28 | |
| kbaegis | Huh | 00:28 |
| kbaegis | Accidentally left apparently | 00:28 |
| kbaegis | Did I miss anything? | 00:28 |
| *** duonghq has joined #openstack-kolla | 00:29 | |
| mark-casey | comings and goings only | 00:29 |
| kbaegis | hey duonghq | 00:29 |
| duonghq | morning kbaegis | 00:29 |
| kbaegis | morning! | 00:29 |
| duonghq | I'm looking at yesterday IRC log | 00:30 |
| duonghq | where are you kbaegis? | 00:31 |
| duonghq | Seem that our Horizon container is messed up? | 00:34 |
| kbaegis | US | 00:34 |
| kbaegis | Wake up at 3:00am local though :) | 00:34 |
| Pavo | well I have done so many edits, guess it best to just start from the beginning lol again | 00:34 |
| kbaegis | Still stuck on horizon | 00:34 |
| duonghq | haha | 00:34 |
| kbaegis | Everything else works | 00:34 |
| duonghq | so I think I'll checkout horizon back some commit | 00:35 |
| kbaegis | I can make images, networks, vms, etc | 00:35 |
| kbaegis | But the damn gui doesn't work | 00:35 |
| kbaegis | It was miscoded in kolla :) | 00:35 |
| duonghq | It was a pretty GUI huh? | 00:35 |
| Pavo | well now I am having deployment issues again but oh well practice makes perfection right | 00:35 |
| kbaegis | Horizon is pretty | 00:35 |
| kbaegis | Esp. the neutron portion | 00:35 |
| kbaegis | Heat too, I suppose | 00:35 |
| duonghq | hah | 00:36 |
| duonghq | kbaegis: I heart that Heat has trouble itself | 00:38 |
| duonghq | *heard | 00:38 |
| Pavo | so is there away to save my registry offline so I do not have to go through building images again? | 00:38 |
| duonghq | Pavo: you can setup a Docker local registry | 00:39 |
| *** thumpba_ has quit IRC | 00:39 | |
| Pavo | I was talking about my local registry lol | 00:39 |
| kbaegis | You can create an image from the container | 00:39 |
| *** thumpba has joined #openstack-kolla | 00:39 | |
| mark-casey | Pavo: on your mac or on a target node that doubles as a deploy host? | 00:39 |
| duonghq | so it should persist? | 00:39 |
| mark-casey | (that is, where is the registry) | 00:39 |
| kbaegis | Yes, but the cleanup images will remove it | 00:40 |
| Pavo | registry is on the deployer node, but about to blow that VM away | 00:40 |
| duonghq | Pavo: so you can setup it elsewhere | 00:40 |
| kbaegis | You can backup containers to tar files as well | 00:40 |
| mark-casey | yeah, and actually if you're replicating a multinode setup I've usually seen the deployer be a separate host | 00:40 |
| duonghq | just a Docker container | 00:40 |
| kbaegis | It'll be large | 00:40 |
| duonghq | Pavo: an insecure-registry setting is appropriate | 00:41 |
| Pavo | size no issue, 27Tb here | 00:41 |
| Pavo | and using TLS not insecure | 00:41 |
| duonghq | Still does not got your point, Pavo | 00:41 |
| Pavo | so export or save | 00:41 |
| duonghq | just grab your volume and backup it? | 00:42 |
| kbaegis | save creates an image, export copies the running container -I think | 00:43 |
| *** thumpba has quit IRC | 00:44 | |
| duonghq | save is for image and export is for container | 00:45 |
| duonghq | so I think export is the answer | 00:45 |
| Pavo | so to take a container and turn into a image I need to commit then save that image right | 00:47 |
| duonghq | I think its right, but why you need it be a image | 00:47 |
| Pavo | because export doesn't keep the data, ie.... images pushed to it | 00:48 |
| Pavo | but just commit the registry container and the size looks WAY to small to have all the images pushed in it | 00:49 |
| Pavo | lol | 00:49 |
| Pavo | 33MB | 00:49 |
| Pavo | can not be right | 00:49 |
| duonghq | you can check the underlying fs for the right size | 00:50 |
| duonghq | (approvimately) | 00:50 |
| Pavo | true | 00:51 |
| mark-casey | Pavo: https://github.com/docker/distribution/blob/master/docs/deploying.md#storage | 00:54 |
| mark-casey | so... if you copy /var/lib/registry to new VM perhaps it will only explode a little? :D | 00:54 |
| Pavo | lol | 00:56 |
| mark-casey | :) but I may have read the mount backwards. that may be the in-container path | 00:57 |
| duonghq | kbaegis: does the latest git code it work? | 00:57 |
| kbaegis | I can test | 00:57 |
| kbaegis | Horizon's doesn't | 00:58 |
| duonghq | okay | 00:58 |
| *** jamesdenton has joined #openstack-kolla | 01:01 | |
| *** jamesden_ has joined #openstack-kolla | 01:02 | |
| *** HyperJohnGraham has joined #openstack-kolla | 01:03 | |
| *** jamesdenton has quit IRC | 01:06 | |
| *** zhurong has joined #openstack-kolla | 01:09 | |
| kbaegis | holy &$#! | 01:19 |
| kbaegis | I have horizon | 01:19 |
| mark-casey | woot | 01:20 |
| kbaegis | :) | 01:20 |
| *** dave-mccowan has joined #openstack-kolla | 01:21 | |
| *** diogogmt has joined #openstack-kolla | 01:25 | |
| *** daneyon has joined #openstack-kolla | 01:26 | |
| kbaegis | Newton is pretty | 01:26 |
| *** jamesden_ has quit IRC | 01:27 | |
| *** jamesdenton has joined #openstack-kolla | 01:28 | |
| *** daneyon has quit IRC | 01:30 | |
| *** huikang has joined #openstack-kolla | 01:33 | |
| *** jamesdenton has quit IRC | 01:33 | |
| *** fragatina has quit IRC | 01:34 | |
| Pavo | how? | 01:35 |
| Pavo | latest git pull? | 01:36 |
| duonghq | kbaegis: nice | 01:39 |
| duonghq | kbaegis: how can you get this | 01:39 |
| kbaegis | No idea why it's working now | 01:40 |
| kbaegis | I rebuilt from centos source | 01:40 |
| Pavo | lol thats awesome | 01:40 |
| duonghq | lol, | 01:40 |
| kbaegis | No 500s from anything now except nova-compute :) | 01:40 |
| duonghq | Pavo: which base do you use? centos or ubuntu | 01:40 |
| duonghq | kbaegis: expect *nova-compute*??? oops | 01:41 |
| Pavo | now using latest git pull | 01:41 |
| Pavo | think its 3.0.0 | 01:41 |
| kbaegis | IDK. I'm getting scheduling errors :) | 01:41 |
| duonghq | its 3.0.0 for a while | 01:41 |
| kbaegis | I'll check the api logs and let you know | 01:41 |
| duonghq | you should get the devxxx revision | 01:41 |
| duonghq | or better if git revision | 01:42 |
| kbaegis | Total usable vcpus: 32, total allocated vcpus: 0 | 01:44 |
| kbaegis | Oh! :) | 01:44 |
| kbaegis | Just a neutron issue | 01:44 |
| kbaegis | ovs is restarting (1) | 01:44 |
| kbaegis | I can fallback to bridging | 01:44 |
| kbaegis | http://hastebin.com/ufixumetik.vbs in case it's interesting | 01:45 |
| kbaegis | tracebacks from neutron error | 01:45 |
| duonghq | oh | 01:46 |
| kbaegis | neutron-server log http://hastebin.com/olotecimod.py | 01:47 |
| duonghq | just got Running command: 'start-ovsdb-server 10.164.180.76 ' | 01:48 |
| duonghq | error | 01:48 |
| kbaegis | Saying that there's no l2 agent | 01:48 |
| duonghq | ah, l2 agent fails you | 01:48 |
| kbaegis | Yes | 01:48 |
| kbaegis | neutron-agent-list still around? | 01:48 |
| duonghq | kbaegis: after you do git pull | 01:48 |
| duonghq | just pip install kolla and rebuild the image? | 01:48 |
| duonghq | anything else? | 01:49 |
| duonghq | (given that I use local registry) | 01:49 |
| kbaegis | From current master? clean images, clean host, rebuild kolla | 01:49 |
| kbaegis | I'll give you a paste of the commands I used | 01:50 |
| duonghq | ah, you clean image too? | 01:50 |
| duonghq | so you rebuild your registry, too? | 01:51 |
| kbaegis | http://hastebin.com/wogiducofa.hs | 01:51 |
| kbaegis | Correct | 01:51 |
| kbaegis | Definitely changing tags now that I have a 3.0.0 chain running | 01:51 |
| duonghq | oh, the proress is time consuming? | 01:51 |
| kbaegis | Certainly :) | 01:52 |
| duonghq | thanks | 01:52 |
| kbaegis | Using centos-source | 01:52 |
| kbaegis | I run in screen so that timeouts don't end the procedure | 01:52 |
| kbaegis | tmux, just as good or better | 01:52 |
| duonghq | me tmux, | 01:52 |
| kbaegis | Nice | 01:53 |
| kbaegis | I was forced to learn devstack/screen | 01:53 |
| kbaegis | I always liked tmux better | 01:53 |
| kbaegis | Screen splitting and whatnot | 01:53 |
| kbaegis | But never used it vocationally | 01:53 |
| kbaegis | Much like dvorak : | 01:53 |
| kbaegis | :) | 01:53 |
| duonghq | devstack doesn't support tmux well but kolla is not this case | 01:53 |
| duonghq | I use dvorak :) | 01:54 |
| kbaegis | Yup. Everything can support screen too | 01:54 |
| kbaegis | Really? | 01:54 |
| kbaegis | See, that's awesome | 01:54 |
| kbaegis | dvorak for the win | 01:54 |
| kbaegis | I got up to 80wps on dvorak | 01:54 |
| kbaegis | And I haven't used it since | 01:54 |
| duonghq | actually I use drorak and its programmer variant | 01:55 |
| duonghq | much easier for my finger then qwerty | 01:55 |
| kbaegis | here's the new and lovely way in which neutron-ovs L2 is broken on gentoo :) http://hastebin.com/odujapizip.vbs | 01:55 |
| kbaegis | Oh, I need br-int? | 01:56 |
| duonghq | :) | 01:56 |
| kbaegis | Presumably on host | 01:56 |
| kbaegis | Hmm | 02:02 |
| kbaegis | I guess I get to find the lovely way in which ovs can be run INSIDE a container | 02:02 |
| kbaegis | Or simply revert | 02:02 |
| kbaegis | I'm guessing this is one of those times it matters that I'm running linux 4.8 w/ ovs 2.5.0 | 02:03 |
| *** unicell has quit IRC | 02:04 | |
| kbaegis | Oh. Actually, exactly same version of ovs | 02:05 |
| kbaegis | Well, if anyone was curious- kolla CAN run on gentoo. It just requires an insane sysadmin with 2 days on their hands | 02:06 |
| kbaegis | *Evil laugh* | 02:07 |
| mark-casey | kbaegis: lol. fwiw I believe ovs is the reason docker daemon runs with MountFlags=shared. I think some ovs config is there and different containers related to ovs needs to see one another updating it in realtime | 02:07 |
| mark-casey | with MountFlags=shared when using Kolla, I mean | 02:08 |
| kbaegis | Gotcha. Weird solution | 02:09 |
| mark-casey | I assume only way to do single-process containers | 02:09 |
| kbaegis | Well I mean, its fine for logs. I think it's weird for proc/sys/dev | 02:10 |
| kbaegis | That's just because I'm new to containers :) | 02:10 |
| *** sdake has quit IRC | 02:13 | |
| *** sdake has joined #openstack-kolla | 02:13 | |
| *** zhurong_ has joined #openstack-kolla | 02:19 | |
| *** huikang has quit IRC | 02:21 | |
| Pavo | grrrrr now having docker issues | 02:22 |
| Pavo | my luck is horribad | 02:23 |
| *** zhurong has quit IRC | 02:23 | |
| Pavo | docker service will not start on deployer | 02:23 |
| openstackgerrit | Leon Zachery proposed openstack/kolla: Simplify install process for faster quickstart execution https://review.openstack.org/356181 | 02:23 |
| Pavo | anyone know what this is level=fatal msg="Error starting daemon: error initializing graphdriver: prerequisites for driver not satisfied (wrong filesystem?)" | 02:23 |
| *** mewald1 has joined #openstack-kolla | 02:25 | |
| *** haplo37 has quit IRC | 02:26 | |
| *** mewald has quit IRC | 02:26 | |
| openstackgerrit | Leon Zachery proposed openstack/kolla: Simplify install process for faster quickstart execution https://review.openstack.org/356181 | 02:27 |
| Pavo | wait wth how did it set this ExecStart=/usr/bin/dockerd --storage-driver btrfs | 02:27 |
| openstackgerrit | Duong Ha-Quang proposed openstack/kolla: Port with_ loop variable syntax to Ansbile 2 https://review.openstack.org/356183 | 02:27 |
| Pavo | how do I fix that | 02:27 |
| *** haplo37 has joined #openstack-kolla | 02:28 | |
| openstackgerrit | Duong Ha-Quang proposed openstack/kolla: Port with_ loop variable syntax to Ansbile 2 (finish) https://review.openstack.org/356183 | 02:30 |
| kbaegis | Pavo You may be able to just remove —storage-driver | 02:31 |
| Pavo | from what file? | 02:31 |
| mark-casey | sudo systemctl status docker | 02:32 |
| mark-casey | will say which | 02:32 |
| Pavo | https://paste.pound-python.org/show/b26OEv9trAVACv7FeNnU/ | 02:33 |
| Pavo | so... /usr/lib/systemd/system/docker.service? | 02:33 |
| mark-casey | should be /usr/lib/systemd/system/docker.service | 02:33 |
| kbaegis | Hmmm. >:( | 02:33 |
| Pavo | ok | 02:33 |
| kbaegis | linuxbridge still isn't | 02:34 |
| Pavo | not seeing that storage in that file though | 02:34 |
| Pavo | found it | 02:34 |
| Pavo | was in /etc/systemd/system/docker.service.d/kolla.conf | 02:35 |
| mark-casey | ah, that makes more sense | 02:35 |
| kbaegis | Stdout: ; Stderr: iptables-restore: line 36 failed | 02:35 |
| Pavo | have no clue how it got there | 02:35 |
| duonghq | Pavo: it is on kolla-kubernetes docs already | 02:35 |
| duonghq | iirc | 02:35 |
| mark-casey | well, it's a common setting used here. makes more sense than the default suddenly changing | 02:35 |
| kbaegis | Here's the fun way in which linux bridging is broken http://hastebin.com/opuruwowom.vbs | 02:38 |
| *** zhurong_ has quit IRC | 02:39 | |
| mark-casey | kbaegis: so, what you're saying is that super-intense iptables rules that probably took a while to cumulate need to function slightly differently on Gentoo? | 02:39 |
| *** zhurong has joined #openstack-kolla | 02:39 | |
| kbaegis | I think it's not interpreting lines prefixed by # | 02:40 |
| *** huikang has joined #openstack-kolla | 02:40 | |
| kbaegis | appropriately | 02:40 |
| mark-casey | are those special? like # Completed by ... # Generated by? | 02:41 |
| kbaegis | They shouldn't be, but possibly :) | 02:41 |
| kbaegis | It's saying it fails on the first line with that | 02:41 |
| kbaegis | Last line, line 36 | 02:41 |
| kbaegis | http://hastebin.com/okurusiref.avrasm | 02:41 |
| *** g3ek has quit IRC | 02:42 | |
| mark-casey | is the localhost interface named differently on Gentoo? look at the stacktrace at the top of the second-to-last paste | 02:43 |
| *** haplo37_ has quit IRC | 02:43 | |
| mark-casey | that trace at least shows the same "Error in agent loop. Devices info" | 02:43 |
| *** haplo37 has quit IRC | 02:43 | |
| mark-casey | oh, n/m. I see. that's iptables_manager exploding | 02:44 |
| mark-casey | so of course its the same | 02:44 |
| kbaegis | Yeah | 02:46 |
| kbaegis | It says line 36 | 02:46 |
| kbaegis | Which is the first one with a hash comment/escape | 02:46 |
| kbaegis | What file is that reading? | 02:46 |
| mark-casey | dunno. but I wonder if it's just choking on the COMMIT and the error is in the commands staged above | 02:47 |
| mark-casey | I think you're missing the command ipset | 02:48 |
| mark-casey | http://heavenkong.blogspot.com/2016/04/resolved-mitaka-linux-bridge-agent.html | 02:49 |
| mark-casey | hrm... except, isn't that happening inside a container? | 02:49 |
| *** yuanying has quit IRC | 02:50 | |
| kbaegis | Yes | 02:52 |
| kbaegis | Incidentally, the hypervisor doesn't have ipset | 02:53 |
| kbaegis | Hypervisor/docker host whatever :) | 02:54 |
| kbaegis | Nice find | 02:54 |
| kbaegis | tyvvvm | 02:54 |
| mark-casey | kbaegis: ty. must either be that or perhaps centos' ipset not running when container is on Gentoo | 02:54 |
| sdake | rtnetlink interface | 02:54 |
| kbaegis | I mean maybe different versions of the linux kernel | 02:55 |
| sdake | ioctl interface | 02:55 |
| sdake | could be different | 02:55 |
| kbaegis | Shouldn't break it like that though | 02:55 |
| sdake | here is the deal | 02:55 |
| mark-casey | heh. the most common failure building centos on ubuntu is for httpd to not install ;? | 02:55 |
| sdake | a number is assigned in a header somewhere | 02:55 |
| mark-casey | :/ | 02:55 |
| sdake | the kernel compiles that number | 02:55 |
| sdake | someone changes the order of how the number is passed or the nubmer itself | 02:55 |
| sdake | bam, you just broke container portability | 02:56 |
| sdake | kernel developers are doing this | 02:56 |
| sdake | and even something inaquaious such as a change in behavior (such as bug removal) could change the behavior of the underlying libraries | 02:56 |
| sdake | 10's of layers of code between openstack and the kernel on the host os | 02:56 |
| sdake | anything out of alignment - bad things happen | 02:57 |
| kbaegis | You're talking about mknod numbers? | 02:57 |
| sdake | this is one type of magic number but those are fixed | 02:57 |
| sdake | i am talking about rtnetlink and ioctl interfaces | 02:58 |
| sdake | the underlying c interfacess in the kernel | 02:58 |
| sdake | there are other interfaces as well | 02:58 |
| sdake | dbus | 02:58 |
| kbaegis | I still need to read up on those | 02:58 |
| kbaegis | Yeah, gentoo 4.8 w/ sysvinit could be problematic | 02:58 |
| kbaegis | who knows :) | 02:58 |
| sdake | the problem with container portability is the APIs change ;) | 02:58 |
| sdake | thats why when you buy software, its compiled for a specific distro | 02:59 |
| sdake | or if you download softwrae, you compiel it for that specific distro | 02:59 |
| *** fragatina has joined #openstack-kolla | 03:00 | |
| mark-casey | sdake: what's the current thought on coreos in kolla? | 03:01 |
| mark-casey | I got prechecks/deploy and pulled up horizon the other day... but it's whole thing is to have next-to-nothing installed and bring everything in via container | 03:02 |
| mark-casey | ubuntu is cross building on it now, but seems fragile | 03:02 |
| mark-casey | err... I assume it fragile for the convo just now | 03:03 |
| kbaegis | The compiler checks and catches that though right? | 03:03 |
| *** fragatina has quit IRC | 03:04 | |
| mark-casey | kbaegis: starting to guess a bit here, but centos is installing a binary bit for bit to disk... may not know it 'splodes until it is later run | 03:05 |
| kbaegis | Right, but how about source? | 03:08 |
| kbaegis | That seems to work for me :) | 03:08 |
| mark-casey | yes, I think if you had tools like that compiling from source hacked into container build it would be more compatible with the running kernel (I think). whether that is a good idea is another matter | 03:09 |
| kbaegis | Well, no one ever accused my style of system management of being a good idea :) | 03:11 |
| kbaegis | It's GREAT experience for break/fix though | 03:11 |
| mark-casey | funny, me either :D | 03:11 |
| *** daneyon has joined #openstack-kolla | 03:14 | |
| sdake | mark-casey no idea on coreos | 03:16 |
| sdake | mark-casey i'd sort of like to implement it | 03:16 |
| sdake | mark-casey but the portability is an issue if coreos provides no userspace packaging | 03:16 |
| sdake | we could do coreos from source of course | 03:16 |
| mark-casey | aren't non-openstack packages still binary in that case? | 03:17 |
| sdake | not if coreos provides them | 03:17 |
| mark-casey | not in the least. it doesn't even have a package manager | 03:17 |
| sdake | or they are binary - but in a compatible way | 03:17 |
| sdake | yes I know | 03:17 |
| kbaegis | Is coreos bsd licensed? | 03:17 |
| sdake | no idea on license | 03:18 |
| mark-casey | not sure. but it is based on gentoo. | 03:18 |
| mark-casey | I don't even think they ship a compiler | 03:18 |
| sdake | oh well no compiler would be nog ood | 03:18 |
| *** satya has joined #openstack-kolla | 03:18 | |
| *** daneyon has quit IRC | 03:18 | |
| *** thumpba has joined #openstack-kolla | 03:19 | |
| mark-casey | well... it's passing so far with ubuntu containers. I'll give a holler if I get a vm running normally but I'm now rather scared of supporting it and wish I'd checked before heh | 03:19 |
| duonghq | sdake: it seem that nobody take care the #322384 commit? | 03:22 |
| duonghq | https://review.openstack.org/#/c/322384/ | 03:23 |
| patchbot | duonghq: patch 322384 - kolla - Check 127.0.1.1 in /etc/hosts on Ubuntu | 03:23 |
| *** satya has quit IRC | 03:23 | |
| *** diogogmt has quit IRC | 03:28 | |
| *** mark-casey has quit IRC | 03:30 | |
| *** sdake has quit IRC | 03:34 | |
| *** zhurong has quit IRC | 03:35 | |
| *** zhurong has joined #openstack-kolla | 03:36 | |
| *** dave-mccowan has quit IRC | 03:38 | |
| duonghq | Pavo, kbaegis: now I got Horizon 500 too | 03:39 |
| duonghq | lol | 03:39 |
| Pavo | lol | 03:39 |
| Pavo | welcome to the club | 03:41 |
| duonghq | Pavo: do you get out of this error? | 03:42 |
| Pavo | nop | 03:42 |
| Pavo | nope | 03:42 |
| duonghq | I guess that I should rollback to find some luck | 03:42 |
| Pavo | how do you rollback? | 03:43 |
| duonghq | just check out horizon docker and horizon ansible some commit | 03:43 |
| duonghq | if I have enough luck | 03:43 |
| *** caowei has joined #openstack-kolla | 03:45 | |
| duonghq | Pavo: https://bugs.launchpad.net/searchlight/+bug/1612268 sound familiar? | 03:49 |
| openstack | Launchpad bug 1612268 in OpenStack Search (Searchlight) "searchlight-ui settings.py out of date" [Undecided,Fix released] | 03:49 |
| *** yuanying has joined #openstack-kolla | 03:49 | |
| Pavo | yeap | 03:49 |
| openstackgerrit | Leon Zachery proposed openstack/kolla: Simplify install process for faster quickstart execution https://review.openstack.org/356181 | 03:51 |
| *** yuanying has quit IRC | 03:55 | |
| *** rminmin has joined #openstack-kolla | 04:01 | |
| *** huikang has quit IRC | 04:01 | |
| *** kbaegis has quit IRC | 04:02 | |
| *** coolsvap has joined #openstack-kolla | 04:04 | |
| *** huikang has joined #openstack-kolla | 04:11 | |
| *** zhurong has quit IRC | 04:14 | |
| *** zhurong has joined #openstack-kolla | 04:16 | |
| *** yuanying has joined #openstack-kolla | 04:16 | |
| Pavo | are the setup scripts in kolla/tools suppose to get the nodes ready for openstack or for kolla? | 04:24 |
| duonghq | Pavo: seem that for Kolla | 04:27 |
| Pavo | oh ok | 04:28 |
| Pavo | could be very useful to make get it ready for target nodes | 04:28 |
| *** unicell has joined #openstack-kolla | 04:30 | |
| *** unicell has quit IRC | 04:31 | |
| *** unicell has joined #openstack-kolla | 04:32 | |
| *** unicell has quit IRC | 04:34 | |
| *** unicell has joined #openstack-kolla | 04:34 | |
| Pavo | what was that heka cmd again to check logs | 04:36 |
| Pavo | finish the deployment and seems horizon still broke | 04:37 |
| Pavo | and rabbitmq container restarting | 04:37 |
| Pavo | ERROR: epmd error for host 192: badarg (unknown POSIX error) | 04:38 |
| *** unicell has quit IRC | 04:44 | |
| openstackgerrit | Duong Ha-Quang proposed openstack/kolla: Add guide to auto mount /run as shared https://review.openstack.org/356222 | 04:44 |
| duonghq | Pavo: can you paste the rabbitmq log, it is quite critical | 04:45 |
| Pavo | yeah trying to | 04:46 |
| Pavo | nothing under /var/lib/docker/volumes/kolla_logs/_data/rabbitmq/ | 04:47 |
| *** unicell has joined #openstack-kolla | 04:47 | |
| Pavo | docker logs show this https://paste.pound-python.org/show/v0gXwTUhSkE8gyYLhypS/ | 04:48 |
| *** jtriley has quit IRC | 04:51 | |
| openstackgerrit | Duong Ha-Quang proposed openstack/kolla: Add guide to auto mount /run as shared https://review.openstack.org/356222 | 04:57 |
| *** huikang has quit IRC | 05:03 | |
| Pavo | ok going to bed, will try some more tomorrow | 05:04 |
| Pavo | night | 05:04 |
| duonghq | http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/2014-March/034269.html | 05:04 |
| duonghq | night Pavo | 05:04 |
| *** yuanying_ has joined #openstack-kolla | 05:29 | |
| *** yuanying has quit IRC | 05:29 | |
| openstackgerrit | Duong Ha-Quang proposed openstack/kolla: Fix guide formating errors https://review.openstack.org/356235 | 05:36 |
| *** thumpba has quit IRC | 05:40 | |
| *** unicell1 has joined #openstack-kolla | 05:43 | |
| *** unicell has quit IRC | 05:43 | |
| *** jtriley has joined #openstack-kolla | 05:48 | |
| *** thumpba has joined #openstack-kolla | 05:51 | |
| *** jtriley has quit IRC | 05:53 | |
| *** thumpba has quit IRC | 05:56 | |
| *** daneyon has joined #openstack-kolla | 05:56 | |
| *** daneyon has quit IRC | 06:01 | |
| *** gfidente has quit IRC | 06:18 | |
| *** egonzalez90 has joined #openstack-kolla | 06:28 | |
| *** egonzalez has joined #openstack-kolla | 06:33 | |
| *** egonzalez90 has quit IRC | 06:36 | |
| *** egongu90 has joined #openstack-kolla | 06:36 | |
| *** caowei has quit IRC | 06:36 | |
| *** egonzalez has quit IRC | 06:40 | |
| *** caowei has joined #openstack-kolla | 06:41 | |
| duonghq | kbaegis: do you got ssh unreachable error when deploy multinode | 06:41 |
| *** egonzalez has joined #openstack-kolla | 06:41 | |
| *** dasTor has joined #openstack-kolla | 06:44 | |
| *** caowei has quit IRC | 06:45 | |
| *** caowei has joined #openstack-kolla | 06:45 | |
| *** caowei has quit IRC | 06:45 | |
| *** egongu90 has quit IRC | 06:45 | |
| *** caowei has joined #openstack-kolla | 06:46 | |
| duonghq | coolsvap: do you got ssh unreachable error when deploy multinode | 06:46 |
| coolsvap | duonghq: i have not deployed multinode recently | 06:47 |
| duonghq | thask | 06:48 |
| *** jtriley has joined #openstack-kolla | 06:48 | |
| *** egonzalez has quit IRC | 06:49 | |
| *** Jeffrey4l has joined #openstack-kolla | 06:49 | |
| openstackgerrit | Duong Ha-Quang proposed openstack/kolla: Add guide to auto mount /run as shared https://review.openstack.org/356222 | 06:51 |
| *** thumpba has joined #openstack-kolla | 06:52 | |
| *** belmoreira has joined #openstack-kolla | 06:53 | |
| *** jtriley has quit IRC | 06:53 | |
| duonghq | Daviey do you got ssh unreachable error when deploy multinode | 06:55 |
| duonghq | mewald1: are you there | 06:56 |
| *** b_bezak has joined #openstack-kolla | 06:56 | |
| *** thumpba has quit IRC | 06:56 | |
| duonghq | mntdevops: has you deployed multinode? | 06:59 |
| *** gfidente has joined #openstack-kolla | 07:00 | |
| duonghq | coolsvap: it seem that in https://review.openstack.org/#/c/322384/ V.Hosakote does active for a while | 07:06 |
| patchbot | duonghq: patch 322384 - kolla - Check 127.0.1.1 in /etc/hosts on Ubuntu | 07:06 |
| duonghq | can I take over | 07:06 |
| openstackgerrit | zhongshengping proposed openstack/kolla: Fix service_type of mistral https://review.openstack.org/356271 | 07:08 |
| coolsvap | duonghq: seems like it, can you ask on the review and wait for couple of days ? | 07:09 |
| duonghq | coolsvay: okay | 07:11 |
| *** b_bezak has quit IRC | 07:17 | |
| *** bootsha has joined #openstack-kolla | 07:19 | |
| *** athomas has joined #openstack-kolla | 07:24 | |
| *** b_bezak has joined #openstack-kolla | 07:24 | |
| *** b_bezak has quit IRC | 07:29 | |
| *** b_bezak has joined #openstack-kolla | 07:35 | |
| mntdevops | duonghq: not yet | 07:39 |
| duonghq | thanks | 07:39 |
| mntdevops | work projects in the way | 07:39 |
| mntdevops | why, what's up | 07:39 |
| mntdevops | if you're getting ssh unreachable, that's an ansible issue | 07:41 |
| mntdevops | add a -vvvv right after the "ansible-playbook" command in question | 07:41 |
| mntdevops | and rerun | 07:41 |
| mntdevops | headed to sleep | 07:42 |
| mntdevops | ttyl | 07:42 |
| openstackgerrit | zhongshengping proposed openstack/kolla: ironic.conf file missing enabled_drivers info https://review.openstack.org/356290 | 07:44 |
| *** daneyon has joined #openstack-kolla | 07:45 | |
| *** caowei has quit IRC | 07:47 | |
| *** jtriley has joined #openstack-kolla | 07:49 | |
| *** daneyon has quit IRC | 07:49 | |
| *** thumpba has joined #openstack-kolla | 07:52 | |
| openstackgerrit | Duong Ha-Quang proposed openstack/kolla: Add guide to auto mount /run as shared https://review.openstack.org/356222 | 07:55 |
| *** jtriley has quit IRC | 07:56 | |
| *** thumpba has quit IRC | 07:57 | |
| *** thumpba has joined #openstack-kolla | 08:01 | |
| *** ad_rien_ has joined #openstack-kolla | 08:06 | |
| *** thumpba has quit IRC | 08:06 | |
| *** coolsvap is now known as coolsvap_ | 08:09 | |
| *** Serlex has joined #openstack-kolla | 08:14 | |
| *** belmoreira has quit IRC | 08:18 | |
| *** bootsha_ has joined #openstack-kolla | 08:23 | |
| *** bootsha has quit IRC | 08:24 | |
| *** matrohon has joined #openstack-kolla | 08:26 | |
| *** anbehl has joined #openstack-kolla | 08:35 | |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: Bump the ansible version to 2.1.1.0 in kolla-toolbox https://review.openstack.org/356311 | 08:39 |
| rstarmer | For those that are interested, I've created an ansible script that seems to be fairly reasonable at getting the base Kolla environmet built (and works reasonably well on centos and ubuntu systems). https://github.com/rstarmer/openstack-kolla-ansible | 08:43 |
| *** jtriley has joined #openstack-kolla | 08:51 | |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: Move the keystone_*_url variable from common role to group vars https://review.openstack.org/356319 | 08:56 |
| openstackgerrit | Md Nadeem proposed openstack/kolla: Added Dockerfile for congress https://review.openstack.org/305626 | 08:57 |
| *** jtriley has quit IRC | 08:57 | |
| *** thumpba has joined #openstack-kolla | 09:02 | |
| pbourke | coolsvap_: ping | 09:06 |
| *** gfidente has quit IRC | 09:07 | |
| openstackgerrit | Md Nadeem proposed openstack/kolla: Added Dockerfile for congress https://review.openstack.org/305626 | 09:08 |
| *** thumpba has quit IRC | 09:08 | |
| pbourke | any core team around this morning | 09:08 |
| pbourke | could use some reviews | 09:08 |
| openstackgerrit | Merged openstack/kolla: Add jq https://review.openstack.org/355003 | 09:08 |
| *** gfidente has joined #openstack-kolla | 09:26 | |
| *** bootsha_ has quit IRC | 09:28 | |
| *** vbel has quit IRC | 09:28 | |
| *** rminmin has quit IRC | 09:29 | |
| *** anbehl has quit IRC | 09:29 | |
| *** bootsha has joined #openstack-kolla | 09:30 | |
| openstackgerrit | Md Nadeem proposed openstack/kolla: Added Dockerfile for congress https://review.openstack.org/305626 | 09:35 |
| *** jtriley has joined #openstack-kolla | 09:52 | |
| *** HyperJohnGraham has quit IRC | 09:56 | |
| *** openstack has joined #openstack-kolla | 10:16 | |
| *** sdake_ has quit IRC | 10:18 | |
| *** sdake_ has joined #openstack-kolla | 10:19 | |
| *** sdake has quit IRC | 10:19 | |
| sdake_ | did anyone fix the horizon problem | 10:19 |
| sdake_ | i was in the midst of looking into it | 10:19 |
| sdake_ | before ptfoing | 10:19 |
| sdake_ | pbourke did you do any work on horizon? | 10:20 |
| sdake_ | Jeffrey4l ? | 10:20 |
| pbourke | which one | 10:20 |
| pbourke | the css on osic prob? | 10:20 |
| sdake_ | moment log will helpl | 10:21 |
| Jeffrey4l | sdake_, yep | 10:21 |
| Jeffrey4l | i fixed the horizon issue | 10:21 |
| Jeffrey4l | sdake_, pbourke https://review.openstack.org/355232 | 10:22 |
| pbourke | Jeffrey4l: sdake_: any way we can tackle the gates before merging much more patches | 10:23 |
| *** Mr_Broken has joined #openstack-kolla | 10:23 | |
| pbourke | https://review.openstack.org/#/c/353942/ | 10:24 |
| patchbot | pbourke: patch 353942 - kolla - Fix bandit gate jobs | 10:24 |
| pbourke | https://review.openstack.org/#/c/355808/ | 10:24 |
| patchbot | pbourke: patch 355808 - kolla - Fix Neutron bootstrap failure due to missing fixtu... | 10:24 |
| Mr_Broken | Hey guys | 10:24 |
| Mr_Broken | anyone had simmiliar issue during deployment ? | 10:24 |
| Mr_Broken | http://paste.openstack.org/show/558995/ | 10:24 |
| Mr_Broken | rabbitmq keeps failing me :/ | 10:24 |
| Mr_Broken | I was thinking ntp issue, but have the same ntp server setup on all hosts | 10:25 |
| Mr_Broken | (its stable/mitaka , ubuntu/binary, multinode topology ) | 10:25 |
| sdake_ | moment let me look at gates | 10:26 |
| sdake_ | didn't know they were busted | 10:26 |
| pbourke | Mr_Broken: im not familiar with that error but I would start by running the getent command on your control nodes | 10:26 |
| pbourke | Mr_Broken: see what it outputs and why its not matching up to what the deploy expects | 10:27 |
| sdake_ | Jeffrey4l that fix - that fixes the panel backtrace? | 10:27 |
| Jeffrey4l | https://review.openstack.org/#/c/355808/ this is a bug from neutron and it is fixed. | 10:27 |
| patchbot | Jeffrey4l: patch 355808 - kolla - Fix Neutron bootstrap failure due to missing fixtu... | 10:27 |
| Jeffrey4l | please do not merge it. | 10:27 |
| sdake_ | Jeffrey4l so abandon change then? | 10:28 |
| Jeffrey4l | sdake_, yep. please. | 10:28 |
| sdake_ | you can abandon | 10:28 |
| sdake_ | ;) | 10:28 |
| *** bootsha has quit IRC | 10:28 | |
| Jeffrey4l | the deploy gate is successful now. | 10:28 |
| Jeffrey4l | no fixture issue. | 10:28 |
| sdake_ | oh pbourke is the patch author | 10:28 |
| sdake_ | you can still abandon it but pbourke really should | 10:28 |
| openstackgerrit | Paul Bourke proposed openstack/kolla: Add force flag to cleanup-images https://review.openstack.org/356364 | 10:29 |
| Jeffrey4l | i -2 on it with some comments. | 10:29 |
| Mr_Broken | @pbourke: thanks m8, my /etc/hosts were cleaned after reboot by cloud-init :P | 10:30 |
| pbourke | thanks Jeffrey4l have abandoned | 10:30 |
| Jeffrey4l | thanks. | 10:30 |
| pbourke | Jeffrey4l: can you check the bandit patch? | 10:31 |
| Jeffrey4l | ok.np | 10:31 |
| sdake_ | Jeffrey4l read this log http://eavesdrop.openstack.org/irclogs/%23openstack-horizon/%23openstack-horizon.2016-08-16.log.html#t2016-08-16T19:49:58 | 10:32 |
| sdake_ | there is a link to locals_conf.j2 line 80 | 10:32 |
| sdake_ | several horizon experts have said we are doing it wrong | 10:33 |
| * Jeffrey4l is checking | 10:33 | |
| sdake_ | from ust now: [03:25:29] <sdake_>looks like regression on our end.. | 10:34 |
| sdake_ | [03:25:39] <sdake_>we have no ci for horizon unfortunately :( | 10:34 |
| sdake_ | [03:26:07] <@robcresswell>That looks like a different issue no? | 10:34 |
| sdake_ | [03:26:25] <@robcresswell>The issue you have will be caused by L80 in that local_settings.j2 | 10:34 |
| sdake_ | [03:26:36] <@robcresswell>Overwriting that dict is a no-no :) | 10:34 |
| sdake_ | [03:27:45] <+tsufiev>+1 | 10:34 |
| sdake_ | [0 | 10:34 |
| sdake_ | i got impressio nline 80 should be entirely removed | 10:35 |
| Jeffrey4l | hmm no idea about that dict. | 10:35 |
| sdake_ | are you in openstack-horizon? | 10:35 |
| sdake_ | #openstack-horizon | 10:35 |
| Jeffrey4l | i am in | 10:36 |
| *** bootsha has joined #openstack-kolla | 10:37 | |
| *** bootsha has quit IRC | 10:39 | |
| sdake_ | cool so durign the day | 10:41 |
| sdake_ | there were 5+ peopel complaing about that particular problem | 10:42 |
| sdake_ | if its as smple as deleting some lines | 10:42 |
| sdake_ | shouldl make it happen asap | 10:42 |
| sdake_ | unfortunately my test box is being redeploeyd atm | 10:42 |
| Jeffrey4l | sdake_, how the issue is happened? what the people did caused this issue? | 10:42 |
| sdake_ | no idea why it just started happening | 10:44 |
| sdake_ | not sure what the scenario was - it hink horizon was backtracing on startup | 10:44 |
| sdake_ | with centos binary | 10:44 |
| sdake_ | maybe it was backtracing during runtime | 10:44 |
| sdake_ | it wasn't clear to me | 10:45 |
| sdake_ | the other problem people constantly coplain about - someone has fixed! | 10:45 |
| sdake_ | https://review.openstack.org/#/c/356181/ | 10:45 |
| patchbot | sdake_: patch 356181 - kolla - Simplify install process for faster quickstart exe... | 10:45 |
| sdake_ | pbourke Jeffrey4l can you both review that pls | 10:45 |
| Jeffrey4l | ok | 10:48 |
| *** Mr_Broken has quit IRC | 10:53 | |
| *** jtriley has joined #openstack-kolla | 10:53 | |
| *** Mr_Broken has joined #openstack-kolla | 10:53 | |
| sdake_ | pbourke arey ou sure about the operation from inside git repo? | 10:53 |
| *** Mr_Broken has joined #openstack-kolla | 10:53 | |
| pbourke | sdake_: which one | 10:53 |
| sdake_ | pbourke i am 99% sure that doen't work as you claim | 10:53 |
| sdake_ | your first comment on that last review | 10:53 |
| pbourke | 'pip install .' ? | 10:54 |
| sdake_ | kolla (atleast used to) always default to the pip installed version even if working out of a gi trepo | 10:54 |
| pbourke | oh | 10:54 |
| pbourke | it depends on the commands you use | 10:54 |
| *** duonghq has quit IRC | 10:54 | |
| pbourke | if you use tools/kolla-ansible it use the git repo | 10:55 |
| pbourke | if you use 'kolla-ansible' it uses the installed location | 10:55 |
| pbourke | that's my experience | 10:55 |
| sdake_ | can you double check that | 10:55 |
| pbourke | ok | 10:55 |
| sdake_ | from the past, and constant support in channel, people run into problem where their git content (such as dockerfiles and ansible) are not being used inside the git repo | 10:56 |
| sdake_ | the solution is pip remove kolla | 10:56 |
| *** bootsha has joined #openstack-kolla | 10:56 | |
| sdake_ | kolla-ansible binary and python runtime work correctly - but ansible and dockerfilels not so much | 10:57 |
| pbourke | sdake_: http://paste.openstack.org/show/558997/ | 10:57 |
| pbourke | look at the location of site.yml that's being used in each case | 10:57 |
| *** Mr_Broken has quit IRC | 10:58 | |
| *** jtriley has quit IRC | 10:58 | |
| sdake_ | yup | 10:58 |
| sdake_ | you convinced me kolla-ansible is working correctly | 10:58 |
| sdake_ | what about build? | 10:58 |
| pbourke | sdake_: same behaviour | 10:58 |
| pbourke | sdake_: just tested I wont bother pasting | 10:58 |
| sdake_ | that bug must have recenty been fixed | 11:00 |
| sdake_ | i wonder why we see the tagging problems | 11:00 |
| sdake_ | 3.0.0 gets tagged if pip installed | 11:01 |
| sdake_ | even when working rom a stable/mitaka branch | 11:01 |
| sdake_ | from git | 11:01 |
| sdake_ | can you verify that is not the case? | 11:01 |
| * sdake_ needs to go setup his testbed | 11:01 | |
| sdake_ | perhaps the python runtime is not running from the git cwd | 11:02 |
| sdake_ | well thanks pbourke - that solves alot of my problems :) | 11:06 |
| sdake_ | pbourke i'll be back in a bit - need to finish ceentos install | 11:06 |
| pbourke | sdake_: ok cool | 11:06 |
| sdake_ | i think the tagging problem either needs to be fixed or those docs can go in sort of as is | 11:06 |
| sdake_ | but i need to verify there is actualy a tagging problem :) | 11:07 |
| pbourke | yeah Im not quite following you on what that problem is | 11:08 |
| pbourke | is it in the gate or elsewhere | 11:08 |
| pbourke | ? | 11:08 |
| sdake_ | pbourke elsewhere | 11:11 |
| sdake_ | pbourke here is the deal | 11:11 |
| sdake_ | people follow the quickstart guide | 11:12 |
| sdake_ | and then i tell them "ok now install stable branch" | 11:12 |
| sdake_ | they install stable branch | 11:12 |
| sdake_ | and the tags are 3..0.0 | 11:12 |
| sdake_ | perhaps the issue is more simply they are not running the commands from the tools directory | 11:12 |
| sdake_ | rather not install stable branch | 11:13 |
| pbourke | hmm | 11:13 |
| sdake_ | i mean check otu stable branch | 11:13 |
| sdake_ | while master branch is installed | 11:13 |
| sdake_ | sorry brain wrming up | 11:13 |
| sdake_ | warming up | 11:13 |
| sdake_ | if you check out stable branch while master branch is installed, a build will produce 3.0.0 tags | 11:13 |
| sdake_ | but i am not sure if its only when using kolla-build or also when using tools/build.py | 11:13 |
| sdake_ | need testbed :) | 11:14 |
| sdake_ | brb | 11:14 |
| pbourke | well if they checked out the stable branch it should be the same | 11:14 |
| pbourke | code in both places | 11:14 |
| *** ad_rien_ has quit IRC | 11:15 | |
| pbourke | the issue seems to be the source of all my pain in life | 11:15 |
| pbourke | pbr | 11:16 |
| openstackgerrit | Merged openstack/kolla: Fix the memcache location value in horizon local settings https://review.openstack.org/355232 | 11:19 |
| openstackgerrit | Merged openstack/kolla: Add guide to auto mount /run as shared https://review.openstack.org/356222 | 11:20 |
| *** daneyon has joined #openstack-kolla | 11:21 | |
| sdake_ | pbourke stable branch is different htten master | 11:21 |
| sdake_ | centos installing | 11:21 |
| sdake_ | i should know in about 10 mins :) | 11:21 |
| pbourke | sdake_: the issue is pbr is returning 3.0.0 | 11:21 |
| pbourke | sdake_: likely from the latest git tag its finding | 11:22 |
| pbourke | sdake_: pbr is the worst thing ever, its never worked correctly for me | 11:22 |
| sdake_ | but correct content is used? | 11:22 |
| pbourke | yes | 11:22 |
| sdake_ | pbr returns 3.0.0 even without a pip install of master? | 11:23 |
| pbourke | yes | 11:24 |
| * sdake_ GROANS | 11:24 | |
| *** sdake_ is now known as sdake | 11:24 | |
| sdake | daviey had a solution to fix the pbr problem using "current" | 11:24 |
| sdake | to represent master | 11:25 |
| sdake | not sure how he planned to solve the stable branches | 11:25 |
| sdake | ok well here is the deal - I spend like 3 hrs a day explaining how to workaround this problem | 11:25 |
| *** daneyon has quit IRC | 11:25 | |
| sdake | I asked leon to look at one of our irc logs where it was explained and turn it into docs | 11:25 |
| sdake | so I can see how he got to where he is | 11:26 |
| sdake | i'd like suggestions for doc fixes in the interim | 11:26 |
| sdake | that leon can integrate - simple as a --tag? | 11:26 |
| *** jtriley has joined #openstack-kolla | 11:27 | |
| *** bootsha has quit IRC | 11:27 | |
| sdake | it may take awhile to sort out this actual pbr problem | 11:27 |
| *** bootsha has joined #openstack-kolla | 11:27 | |
| sdake | and endless days of answering the same questions over and over is not a good use of our time :) | 11:27 |
| pbourke | sdake: maybe im wrong, a fresh install of stable in a virtualenv is showing tag 2.0.3 | 11:31 |
| *** jtriley has quit IRC | 11:32 | |
| pbourke | sdake: tbh, Im staying out of pbr, there are clearly some people out there who grok it but im not one | 11:32 |
| sdake | pbourke - house on fire - lets put out the fire first - which is via doc changes | 11:32 |
| sdake | my box shoudl be up in about 5 mins | 11:33 |
| sdake | i'll let daviey sort out pbr - he seems to understand it :) | 11:33 |
| sdake | (i.e. the long term fix) | 11:33 |
| sdake | so just to clarify - content is always run from either git location or global location depending on where tools are run | 11:34 |
| sdake | so this is just a tag return issue? | 11:34 |
| sdake | or your uncertain now? | 11:34 |
| pbourke | im pretty confident content is depending on where tools are run | 11:35 |
| sdake | wht about kolla/kolla/*py | 11:35 |
| pbourke | so yeah its just a pbr issue | 11:35 |
| sdake | that is where the tag is hardcoded | 11:35 |
| pbourke | the tag isn't hardcoded - it's figured out with pbr magic | 11:36 |
| *** bootsha has quit IRC | 11:37 | |
| Jeffrey4l | python -c 'import pbr.version; print(pbr.version.VersionInfo("kolla").release_string())' | 11:38 |
| Jeffrey4l | python -c 'import pbr.version; print(pbr.version.VersionInfo("kolla").cached_version_string())' | 11:38 |
| Jeffrey4l | here is the pbr magic | 11:38 |
| Jeffrey4l | kolla using cached_version_string now. I think we should move to the release_string | 11:39 |
| sdake | its hardcoded somewhere - pretty sure | 11:40 |
| sdake | let me check | 11:40 |
| *** narasimha_SV_ has joined #openstack-kolla | 11:42 | |
| narasimha_SV_ | after deploying ironic using kolla | 11:42 |
| narasimha_SV_ | when i try to do pxe boot of a baremetal | 11:42 |
| narasimha_SV_ | http://paste.openstack.org/show/559000/ | 11:42 |
| narasimha_SV_ | i am getting this issue | 11:42 |
| *** kbaegis has joined #openstack-kolla | 11:42 | |
| narasimha_SV_ | i dont see a folder called /var/lib/ironic in ironic-conductor container | 11:42 |
| sdake | hmm i thought there was one in common/config.py | 11:43 |
| sdake | but git show shows no such change about that | 11:43 |
| * sdake needs to do more reviewing... | 11:44 | |
| kbaegis | Morning | 11:44 |
| sdake | Jeffrey4l mind giving me a tldr on the delta between release_string and cached_version_string | 11:44 |
| *** bootsha has joined #openstack-kolla | 11:45 | |
| sdake | narasimha_SV_ ironic implementation is on the way | 11:46 |
| *** dave-mccowan has joined #openstack-kolla | 11:46 | |
| sdake | narasimha_SV_ - intel cats got it working a few days ago | 11:46 |
| sdake | narasimha_SV_ not sure who exactly - one of sean-k-mooney teammates | 11:46 |
| wznoinsk | miau | 11:46 |
| sdake | narasimha_SV_ it will still be tech preview in newton | 11:46 |
| sdake | but it hould be working :) | 11:47 |
| wznoinsk | or english: meow | 11:47 |
| sdake | but it is not workign now | 11:47 |
| sdake | hey wznoinsk | 11:47 |
| wznoinsk | hi sdake | 11:47 |
| *** dwalsh has joined #openstack-kolla | 11:47 | |
| wznoinsk | I plan to get my working ironic tested on master in next few days, I'm about a click away from giving myself a baremetal to do it ;-) | 11:48 |
| sdake | wznoinsk hot - apologies for not recalling who does the work | 11:49 |
| wznoinsk | narasimha_SV_: https://bugs.launchpad.net/kolla/+bug/1611758 | 11:49 |
| openstack | Launchpad bug 1611758 in kolla mitaka "ironic_conductor - Permission denied: '/var/lib/ironic'" [High,Confirmed] - Assigned to Waldemar Znoinski (wznoinsk) | 11:49 |
| sdake | wznoinsk there are a whole lotta poeple working on this project ;) | 11:49 |
| wznoinsk | sdake: no worries, it's my job to get my name recognized | 11:50 |
| sdake | it just got recognized :) | 11:50 |
| kbaegis | Yes it did :) | 11:50 |
| wznoinsk | thanks guys | 11:50 |
| kbaegis | Very cool work | 11:50 |
| wznoinsk | narasimha_SV_: let me know if you have further ironic issues in kolla, if you can't find them at https://bugs.launchpad.net/kolla (I've reported a couple and my colleagues did too) then ping me | 11:51 |
| sdake | Jeffrey4l - i rtfm'ed and can't find any description of those two methods | 11:52 |
| sdake | Jeffrey4l - reading the code will be painful - mind giving me a tldr ;) | 11:52 |
| *** jeh has quit IRC | 11:54 | |
| narasimha_SV_ | Thanks wznoinsk | 11:54 |
| narasimha_SV_ | can I have your personal mail id | 11:55 |
| sdake | narasimha_SV_ - mailing list | 11:55 |
| sdake | narasimha_SV_ all discussions should be public on mailing list - its part of the 4 opens - a foundation of openstack guiding principles | 11:55 |
| wznoinsk | narasimha_SV_: I don't mind sharing knowledge here or on the mailing list | 11:55 |
| sdake | yup 4 opens require recording of all knowledge transfer in essence | 11:56 |
| sdake | irc, mailing list, team meetings, those are all good to go | 11:56 |
| coolsvap_ | hi sdake pbourke morning | 11:56 |
| kbaegis | sdake horizon started working last night. Was that your doing? | 11:56 |
| narasimha_SV_ | no no not like that I just want to talk | 11:56 |
| sdake | kbaegis Jeffrey4l is sorting out the proboel mthe openstack-horizon folks pointed out | 11:57 |
| sdake | whether it works or not, it is broken atm :) | 11:57 |
| *** jeh has joined #openstack-kolla | 11:57 | |
| *** jeh has quit IRC | 11:57 | |
| *** jeh has joined #openstack-kolla | 11:57 | |
| coolsvap_ | just came from a PTO today, sorry for the delayed response | 11:57 |
| pbourke | welcome back coolsvap_ | 11:58 |
| * coolsvap_ just busy with some other work | 11:58 | |
| sdake | coolsvap_ all good - i was on pto friday->modnay :) | 11:58 |
| kbaegis | Jeffery4l Thanks for the new local_settings! | 11:58 |
| *** ad_rien_ has joined #openstack-kolla | 11:58 | |
| *** Mr_Broken has joined #openstack-kolla | 11:58 | |
| coolsvap_ | sdake: yeah, give me couple of hours, I will be bit more active just around the meeting time and onwards | 11:58 |
| sdake | coolsvap_ roger - pbourke working on a rallly script of some sort | 11:58 |
| * sdake behind | 11:59 | |
| sdake | coolsvap_ so we can have repeatable runs of rally - coould probably use your help on conig there | 11:59 |
| sdake | config there | 11:59 |
| *** ad_rien_ has quit IRC | 12:00 | |
| coolsvap_ | sdake: roger that | 12:00 |
| sdake | centos finally up | 12:00 |
| *** ad_rien_ has joined #openstack-kolla | 12:00 | |
| sdake | off to try Jeffrey4l 's suggestion bbiaf | 12:00 |
| coolsvap_ | are we having any config issues with rally deployment on the node? | 12:00 |
| *** sdake_ has joined #openstack-kolla | 12:01 | |
| pbourke | coolsvap_: none that I know of | 12:01 |
| pbourke | coolsvap_: give me a shout when you have a minute and we'll have a look at where its at | 12:02 |
| Daviey | pbourke: Hey.. Who found that the VLAN issue was kernel releated? | 12:02 |
| pbourke | Daviey: me, inc0 and Jeffrey4l | 12:02 |
| *** bjolo has joined #openstack-kolla | 12:03 | |
| *** coolsvap_ is now known as coolsvap | 12:03 | |
| *** sdake has quit IRC | 12:05 | |
| *** sdake_ is now known as sdake | 12:05 | |
| coolsvap | pbourke: ack | 12:05 |
| *** rhallisey has joined #openstack-kolla | 12:06 | |
| sdake | Daviey can you give me a tldr on the delta between these two mthods - docs show nothing | 12:07 |
| sdake | 04:38:45] <Jeffrey4l>python -c 'import pbr.version; print(pbr.version.VersionInfo("kolla").release_string())' | 12:07 |
| sdake | [04:38:50] <Jeffrey4l>python -c 'import pbr.version; print(pbr.version.VersionInfo("kolla").cached_version_string())' | 12:07 |
| sdake | [04:38:57] <Jeffrey4l>here is the pbr magic | 12:07 |
| sdake | [04:39:16] <Jeffrey4l>kolla using cached_version_string now. I think we should move to the release_string | 12:07 |
| *** rhallisey_ has joined #openstack-kolla | 12:08 | |
| sdake | morning rhallisey | 12:09 |
| rhallisey_ | morning | 12:10 |
| *** rhallisey has quit IRC | 12:12 | |
| *** rhallisey_ is now known as rhallisey | 12:13 | |
| sdake | Jeffrey4l Daviey - looks like tag is correct in either case without master installed -> http://paste.fedoraproject.org/409562/14714359/ -> http://paste.fedoraproject.org/409562/14714359 | 12:13 |
| NachoDuck | Hi, following the instructions here - http://docs.openstack.org/developer/kolla/swift-guide.html#rings it seems to be searching for an image at docker.io/kolla/centos-binary-swift-base which doesn't exist? | 12:15 |
| sdake | NachoDuck - working on that problem now with a docs fix | 12:16 |
| sdake | NachoDuck - mind running a few commands - i can get you rollling | 12:16 |
| sdake | pbourke ^^ see what i'm talking about ;) | 12:17 |
| NachoDuck | sdake: Sure thing, I'm trying this multinode btw | 12:17 |
| sdake | NachoDuck - run pip show kolla | 12:17 |
| pbourke | :) | 12:17 |
| NachoDuck | sdake: https://gist.github.com/anonymous/7a5c4cdf764cbd56c0e876dfa18605c2 | 12:18 |
| sdake | NachoDuck are you doing an eval or want to develop for kolla? | 12:18 |
| NachoDuck | eval | 12:18 |
| sdake | nice you hae teh stable branch installed | 12:18 |
| sdake | thats a new one :) | 12:18 |
| NachoDuck | heh :) | 12:18 |
| sdake | ok our code is incompatible with pbr at present | 12:18 |
| sdake | we are sorting that out - but want to provide doc fix in short term to put out the fire ;) | 12:19 |
| sdake | can you run docker images -> paste | 12:19 |
| NachoDuck | https://gist.github.com/anonymous/3acd0b30ca21f8c2f3395c8f741cc9ad | 12:20 |
| sdake | set openstack_version: 2.0.2" in globals.yml | 12:20 |
| sdake | openstack_version: "2.0.2" inglobals.yml | 12:21 |
| NachoDuck | Yep already had that set | 12:21 |
| NachoDuck | ah wait | 12:22 |
| sdake | Daviey the problem I see with the current mechanism you propose is around the stable branches | 12:23 |
| sdake | Daviey - in one case people can build their own images - current would work - in another case people pull from ddocker hub - in which case current would not work | 12:23 |
| NachoDuck | sdake: Ok done, had confused it with openstack_release: "2.0.2" | 12:23 |
| sdake | sorry openstack_releae | 12:23 |
| sdake | not version | 12:23 |
| NachoDuck | ah, well its set either way | 12:23 |
| sdake | do you have openstack_releae: "2.0.2" | 12:23 |
| NachoDuck | yep | 12:23 |
| sdake | paste your globals.yml pls ;) | 12:23 |
| sdake | NachoDuck - did you ahe a running install previously? | 12:24 |
| NachoDuck | sdake: Yes got it working fine eventually with Ceph but I'd like to include Swift in our eval too | 12:25 |
| NachoDuck | https://gist.github.com/anonymous/37afdff3f91498b76c50641926acd56f | 12:25 |
| sdake | got it | 12:25 |
| sdake | moment | 12:25 |
| sdake | this may be a different problem | 12:26 |
| NachoDuck | Ok no worries | 12:26 |
| sdake | NachoDuck - not sure why your trying to pull swift-base - tag is set in dockerhub: https://hub.docker.com/r/kolla/centos-binary-swift-base/tags/ | 12:28 |
| sdake | NachoDuck can you paste your deploy operation for me? | 12:28 |
| NachoDuck | Sure, one sec | 12:28 |
| * sdake feels like a squirrel sometimes... | 12:29 | |
| NachoDuck | Might be a mistake I made when I chopped this together as included that add commands from Oracle's documentation to facilitate the multinode deployment - https://gist.github.com/anonymous/79d9a8e48526d9e9a181fdca151512ce | 12:29 |
| sdake | oh - that is where your getting the error - when running that script? | 12:30 |
| sdake | pbourke didn't we get rid of ringbuilder? | 12:31 |
| sdake | rather you iirc ;) | 12:31 |
| NachoDuck | yeh | 12:31 |
| pbourke | sdake: yeah | 12:32 |
| sdake | the docker run commands are missing tags NachoDuck | 12:32 |
| pbourke | is that causing issues | 12:32 |
| sdake | pbourke - ithink there must have been a docimpact there | 12:32 |
| sdake | pbourke - NachoDuck is running a script in oracle's documentation that depends on it | 12:33 |
| sdake | pbourke - https://gist.github.com/anonymous/79d9a8e48526d9e9a181fdca151512ce | 12:33 |
| pbourke | sdake: actually it seems to still be referenced in our doc | 12:33 |
| pbourke | my bad :/ | 12:33 |
| sdake | pbourke - right doc impact ;) | 12:33 |
| NachoDuck | For reference, its quite similar to what you have http://docs.oracle.com/cd/E73172_01/E73174/html/setup-swift.html | 12:33 |
| pbourke | I better re-instate it | 12:33 |
| pbourke | NachoDuck: that's a mistake by me, sorry. Will re-instate the script | 12:34 |
| sdake | NachoDuck - yup - working on it ;) | 12:34 |
| NachoDuck | No worries, thanks :) | 12:34 |
| sdake | is the long term answer to remove that from the docs as well? | 12:34 |
| sdake | pbourke ^ | 12:35 |
| sdake | pbourke - I recall seeing your change, but don't recall the rationale - but it seemed fine at the time | 12:35 |
| pbourke | sdake: I dont think so as it appears to be the easiest way to create rings | 12:35 |
| pbourke | sdake: I haven't looked at swift in ages | 12:35 |
| sdake | pbourke ya me either | 12:35 |
| sdake | pretty sure sbezverk looks at it regularly tho | 12:36 |
| sdake | if our gate wasnt so leaky - we would knwo for sure ;) | 12:36 |
| pbourke | wait hang on | 12:36 |
| pbourke | we're getting confused | 12:36 |
| pbourke | swift-ring-builder is a swift command | 12:36 |
| Jeffrey4l | sdake, you need install kolla to make the pbr works or run the script in the kolla folder. | 12:36 |
| pbourke | the script I removed is no longer used, and that patch hasn't even merged yet | 12:36 |
| sdake | pbourke thanks for clarification | 12:37 |
| sdake | Jeffrey4l - playing around with it - before chasing another nut :) | 12:37 |
| pbourke | so NachoDuck, what exact error are you seeing? | 12:37 |
| pbourke | NachoDuck: doing a lot of things at once, sometimes get wires crossed | 12:37 |
| sdake | [05:15:42] <NachoDuck>Hi, following the instructions here - http://docs.openstack.org/developer/kolla/swift-guide.html#rings it seems to be searching for an image at docker.io/kolla/centos-binary-swift-base which doesn't exist? | 12:38 |
| NachoDuck | pbourke: This is the sort of output I get when running that script - https://gist.github.com/anonymous/6a0c6e54244c1e366ff80f07794d2b81 | 12:38 |
| sdake | NachoDuck ya that script needs tags | 12:38 |
| NachoDuck | rightio | 12:39 |
| sdake | or the docs do | 12:39 |
| sdake | or both :) | 12:39 |
| pbourke | NachoDuck: ok that's easy, yeah its what sdake said | 12:39 |
| pbourke | NachoDuck: you need to tweak the vars at the top to match your environment | 12:39 |
| pbourke | docker images | grep swift-base | 12:39 |
| sdake | pbourke - there is no tags in the vars | 12:39 |
| pbourke | that also | 12:39 |
| sdake | our documentaiton is wrong | 12:39 |
| NachoDuck | pbourke: gotcha, will see what I can do | 12:39 |
| pbourke | we could probably write a playbook to do this | 12:40 |
| sdake | probably someday :) | 12:40 |
| sdake | NachoDuck do you have neoug hto work with or need further guidance? | 12:41 |
| sdake | tag should be 2.0.2 | 12:41 |
| sdake | just add :2.0.2 after each image name | 12:41 |
| sdake | in the docker run commands | 12:41 |
| NachoDuck | sdake: Ok thanks, wasn't too sure. Will give it a go | 12:42 |
| sdake | NachoDuck if you run into trouble - i'll be here | 12:42 |
| *** jamesdenton has joined #openstack-kolla | 12:42 | |
| NachoDuck | Awesome, thanks both | 12:42 |
| sdake | i guess that tells us nobody is using swift... :( | 12:44 |
| sdake | that script and docs have been broken for months | 12:44 |
| sdake | context switching at 5am = ftl :( | 12:45 |
| sdake | Jeffrey4l http://paste.fedoraproject.org/409593/14714382/ -> http://paste.fedoraproject.org/409593/14714382 | 12:50 |
| sdake | that is with master installed | 12:50 |
| sdake | it is showing 3.0.0 tags even on stable branch from pbr | 12:50 |
| sdake | with both incantations | 12:51 |
| Jeffrey4l | sdake, so 1. install the kolla 2. checkout to mitaka 3 print the version? | 12:51 |
| Jeffrey4l | sdake, so 1. install the kolla master 2. checkout to mitaka 3 print the version? | 12:51 |
| sdake | that is exactly what i did | 12:51 |
| Jeffrey4l | right? | 12:51 |
| Jeffrey4l | sdake, how did you installed the kolla? | 12:52 |
| Jeffrey4l | pip install .? | 12:52 |
| sdake | sudo pip install kolla/ | 12:52 |
| Jeffrey4l | that's it | 12:52 |
| Jeffrey4l | pbr load the kolla version from the python side-packages. | 12:52 |
| Jeffrey4l | even u change the kolla code base from master to mitaka. it do not affect the kolla version | 12:53 |
| Jeffrey4l | u can check by using `pip freeze | grep kolla` | 12:53 |
| Jeffrey4l | in the python env, u still have the kolla 3.0 | 12:53 |
| sdake | i am more concerned about the content that the tools run | 12:53 |
| sdake | such as the ansible and dockerfiles | 12:53 |
| Jeffrey4l | hmm? | 12:54 |
| sdake | i think pbourke confirmed the content was run was the correct content - | 12:54 |
| sdake | if from git, running git content | 12:54 |
| sdake | if from non-git dir, running system content | 12:54 |
| pbourke | perhaps the version is a hole in the mechanism | 12:55 |
| pbourke | as in it's the only part that comes from the installed version regardless | 12:55 |
| sdake | possible - if we are certain the content is run correctlly from git :) | 12:55 |
| Jeffrey4l | you can try pip install -e . | 12:55 |
| Jeffrey4l | add the `-e` parameter. | 12:55 |
| sdake | i know pp -e will work | 12:55 |
| Jeffrey4l | OK. | 12:56 |
| sdake | but pip -e doesn't work on some folks environments for whatever rason | 12:56 |
| sdake | moment - running a tet | 12:56 |
| sdake | test | 12:56 |
| *** rstarmer has joined #openstack-kolla | 13:00 | |
| *** rstarmer has quit IRC | 13:00 | |
| sdake | python content run from git repo is run from git repo | 13:01 |
| sdake | i put a raise in build.py main in master installed version - which shoudl have triggered a crater | 13:01 |
| sdake | so its simply an issue of pbr doing the wrong thing | 13:01 |
| *** jtriley has joined #openstack-kolla | 13:03 | |
| openstackgerrit | Josh Lothian proposed openstack/kolla: Customizations for heka https://review.openstack.org/346861 | 13:04 |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: Move to transport_url for rabbitmq configuration https://review.openstack.org/356449 | 13:04 |
| sdake | power supplyy not working | 13:04 |
| sdake | bbiaf | 13:04 |
| sdake | boy not really waht i needed today | 13:05 |
| *** zhurong has joined #openstack-kolla | 13:05 | |
| *** jamesdenton has left #openstack-kolla | 13:05 | |
| sdake | got it | 13:06 |
| sdake | wiggling ftl | 13:06 |
| sdake | bad sign | 13:06 |
| *** zhurong has quit IRC | 13:07 | |
| *** zhurong has joined #openstack-kolla | 13:09 | |
| sdake | Daviey - any idea how to fix this pbr issue? | 13:11 |
| NachoDuck | The swift deployment appears to have gone smoothly, in case you were wondering. Thanks again :) | 13:11 |
| *** coolsvap is now known as _coolsvap_ | 13:11 | |
| sdake | NachoDuck was wondering - thanks for the confirmation! | 13:11 |
| sdake | NachoDuck could you also confirm if its functiona lfor you? | 13:12 |
| *** bootsha has quit IRC | 13:12 | |
| NachoDuck | sdake: Yep just going to start playing with it a bit now, have been able to create a container and upload. So far so good | 13:12 |
| sdake | thats good news :) | 13:12 |
| sdake | since it looks like this code path hasn't been tested in a few months.. | 13:13 |
| sdake | our ci doesn't test it approprateily | 13:13 |
| *** duonghq has joined #openstack-kolla | 13:14 | |
| NachoDuck | Ah I see, well perhaps testament to good work that it's still functional despite that | 13:14 |
| sdake | NachoDuck thanks :) | 13:14 |
| sdake | NachoDuck - we keep our default configs very minnimal or this exact reason | 13:14 |
| sdake | NachoDuck just enough to work out of the box and nothign more | 13:15 |
| sdake | sometimes upstream changes config options - this is when things break | 13:15 |
| *** bootsha has joined #openstack-kolla | 13:15 | |
| sdake | and sometimes master is plan broken in various parts of openstack ;-) | 13:15 |
| sdake | plain | 13:15 |
| NachoDuck | sdake: It's been refreshing to work with so far after looking at our other deployment options, I can't say I was looking forward to customising Heat templates and the like | 13:16 |
| sdake | i haven' tevaled other options for quite awhile | 13:17 |
| sdake | its very time consuming | 13:17 |
| sdake | but thanks fo the positive feedback! | 13:17 |
| *** inc0 has joined #openstack-kolla | 13:17 | |
| NachoDuck | Yeah, it took me a while to arrive at this point. No problem! | 13:17 |
| *** sean-k-mooney has quit IRC | 13:18 | |
| sdake | NachoDuck if I haven't already asked in the past - mind telling your top 3 pain points with kolla? | 13:18 |
| inc0 | good morning | 13:22 |
| sdake | sup inc0 | 13:23 |
| *** Guest52040 has joined #openstack-kolla | 13:23 | |
| duonghq | morning inc0 | 13:24 |
| *** bootsha has quit IRC | 13:27 | |
| kbaegis | good morning | 13:28 |
| kbaegis | So with the registry- even if docker images doesn't show anything (i.e. after a cleanup), those images will still be cached and available inside the registry w/ appropriate tags, correct? | 13:29 |
| kbaegis | It will implicitly pull as long as kolla-build pushed the images there, right? | 13:29 |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes: Glance bootstrap to create service/project/role https://review.openstack.org/346215 | 13:30 |
| duonghq | what do you mean "implicitly pull? | 13:31 |
| kbaegis | when I do a kolla-ansible deploy | 13:31 |
| kbaegis | It checks the local docker images, else pulls from registry? | 13:31 |
| duonghq | yes | 13:31 |
| kbaegis | I have a kolla-build.conf which is tagging with "alpha", and I wanted to be sure about behavior | 13:31 |
| kbaegis | tyvm | 13:31 |
| NachoDuck | sdake: Sure, sorry all of these at the moment are related to docs, there might be more but I've been trying to get Kolla working for us for a while so I've probably forgot some :) To be honest once I got around these issues I've found very little issues with the deployment itself, which is good I suppose! | 13:33 |
| NachoDuck | 1) Might just be me but maybe some of the basics is missing from the documentation that can be forgotten, it was a while before I thought of disabling selinux and the firewall and instead was chasing around what I assume were either misconfigurations on my part or issues with the version of the project I was using. | 13:34 |
| NachoDuck | 2) Again with documentation some key points are missing in places, for example it wasn't until I happend to catch in here that I need to use Ansible 1.9 for a stable branch deployment and that tripped me up for a while | 13:34 |
| NachoDuck | 3) Bit of a weak one but I might come up with something better later on - It wasn't clear to me for multinode whether or not I had to build my own images to begin with, so I wasted a bit of time there where I didn't have to either. The docs might be a bit too develop orientated in places I guess? | 13:34 |
| *** narasimha_SV_ has quit IRC | 13:34 | |
| Daviey | sdake: Sorry, i've been distrated with meetings all morning | 13:35 |
| *** bootsha has joined #openstack-kolla | 13:35 | |
| Daviey | sdake: Stable should be OK. If kolla continues to build images for all tagged releases, then if i pip install kolla, it will work | 13:35 |
| *** ad_rien_ has quit IRC | 13:35 | |
| Daviey | sdake: If i choose to install from git, stable/* - then i need to build my own images. That seams like a reasonable compromise | 13:36 |
| duonghq | ah, anybody try multinode deployment? | 13:36 |
| *** caowei has joined #openstack-kolla | 13:36 | |
| Daviey | sdake: We should probably be supporting people installing from releases (both distro and pip), rather than from git? | 13:36 |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: Sync the local_settings.j2 with horizon https://review.openstack.org/356471 | 13:37 |
| NachoDuck | duonghq: Yep, got it working eventually | 13:37 |
| duonghq | NachoDuck: Did you got ansible cannot established connection error? | 13:38 |
| NachoDuck | duonghq: That's new to me unfortunately I think. If it's just general ansible connection errors, make sure you've setup your public key auth? | 13:40 |
| *** ad_rien_ has joined #openstack-kolla | 13:40 | |
| duonghq | yes, I can ssh with public key | 13:40 |
| duonghq | I used ansible 1.9 before and does not get any error | 13:42 |
| duonghq | but I havenot had much experience with ansible 2.x | 13:42 |
| NachoDuck | You need to use 1.9 if you're running from stable iirc | 13:42 |
| duonghq | I use master | 13:42 |
| duonghq | just Ansible error | 13:42 |
| *** dwalsh has quit IRC | 13:42 | |
| *** dwalsh has joined #openstack-kolla | 13:45 | |
| NachoDuck | I see, what's the full error message you get? | 13:46 |
| duonghq | : UNREACHABLE! => {"changed": false, "msg": "Failed to connect to the host via ssh.", "unreachable": true} | 13:48 |
| duonghq | like this | 13:48 |
| duonghq | I google this | 13:48 |
| duonghq | i tried some result but cannot got it run | 13:48 |
| *** sdake has quit IRC | 13:50 | |
| *** thumpba has joined #openstack-kolla | 13:52 | |
| NachoDuck | Odd if you're able to connect with that same user using your key as that's pretty much what Ansible is doing anyway | 13:53 |
| *** egonzalez90 has joined #openstack-kolla | 13:54 | |
| *** egonzalez90 has quit IRC | 13:54 | |
| Serlex | hello, how may i change the timezone of the containers? | 13:56 |
| *** egonzalez90 has joined #openstack-kolla | 13:57 | |
| *** b_bezak has quit IRC | 13:57 | |
| duonghq | NachoDuck: I'm retrying | 13:57 |
| *** huikang has joined #openstack-kolla | 13:58 | |
| *** lrensing_ has joined #openstack-kolla | 13:58 | |
| *** sdake_ has joined #openstack-kolla | 14:01 | |
| *** daneyon has joined #openstack-kolla | 14:03 | |
| sdake_ | NachoDuck - 1 - ya actually kolla runs with selinux but the version thing is a real problem we are trying to solve rapidly :) - 2 there is a review in the queue to fix this dependency list for stable vs nonstable branches - 3 yup hear ya docs are developer oriented | 14:04 |
| sdake_ | NachoDuck i htink the key thing you point out is our docs are weak - which we know - and are working to fix :) | 14:04 |
| *** Pavo has quit IRC | 14:04 | |
| *** bootsha has quit IRC | 14:05 | |
| sdake_ | Daviey the issue with supporting people only from releases rather then a git checkout is people sometimes want to use master (for example to join the kolla developer community) | 14:05 |
| *** bootsha has joined #openstack-kolla | 14:06 | |
| sdake_ | people use master for other reasons as well | 14:06 |
| *** Pavo has joined #openstack-kolla | 14:06 | |
| sdake_ | inc0 - they should really just start out everyone at level 70, no? :) | 14:07 |
| inc0 | sdake_, well, first time off it's fun to go through story | 14:08 |
| sdake_ | i guess buid your own images is a reasonable compromise | 14:08 |
| inc0 | but I doubt there are much first timers around now | 14:08 |
| *** daneyon has quit IRC | 14:08 | |
| sdake_ | not sure how to document that tho | 14:08 |
| sdake_ | ya when they made it hard and then DOUBLED it, it was unbearable to get through cts 1-4 solo | 14:09 |
| duonghq | NachoDuck: when I retry at home, it's ok, tomorrow I'll recheck at work, | 14:09 |
| *** hrito has joined #openstack-kolla | 14:09 | |
| *** hrito has left #openstack-kolla | 14:10 | |
| *** bootsha has quit IRC | 14:11 | |
| *** hrito has joined #openstack-kolla | 14:11 | |
| *** hrito has left #openstack-kolla | 14:13 | |
| inc0 | pbourke, any news on osic cluster? | 14:13 |
| pbourke | inc0: noone working on it since yesterday | 14:14 |
| inc0 | ok, I'll try to run rally on top of what we have | 14:14 |
| pbourke | inc0: i can show you what we have in progress | 14:14 |
| inc0 | yes plz | 14:14 |
| inc0 | give me a sec | 14:14 |
| pbourke | inc0: connecting | 14:14 |
| *** hrito has joined #openstack-kolla | 14:14 | |
| inc0 | to get on tmux | 14:14 |
| pbourke | inc0: | 14:15 |
| pbourke | so two files | 14:15 |
| pbourke | inc0: all-scenarios are all the rally scenarios. Then we've commented out the ones that aren't relevant | 14:16 |
| pbourke | inc0: the second script then runs each scenario, and stores the results in some sort of db | 14:16 |
| inc0 | kk | 14:16 |
| pbourke | inc0: its pretty simple. there's one more command needed to dump the results to json | 14:17 |
| inc0 | yeah, dump json something | 14:17 |
| pbourke | inc0: other thing that needs to be done is - _coolsvap_ has mentioned the settings in some of these scenarios are appropriate for the scale of our cluster | 14:17 |
| pbourke | I dont quite understand what needs to change there though | 14:17 |
| pbourke | possibly examples there in that diff | 14:17 |
| pbourke | but i dont know where those numbers are coming from (e.g. 3 -> 25) | 14:18 |
| pbourke | once we nail down the script though it will make rally after each deploy scenario a doddle | 14:18 |
| pbourke | so that's it basically, I want to finish but no time right now | 14:18 |
| inc0 | ok | 14:19 |
| inc0 | I'll play with it | 14:19 |
| *** caowei has quit IRC | 14:21 | |
| *** haplo37 has joined #openstack-kolla | 14:22 | |
| *** haplo37_ has joined #openstack-kolla | 14:22 | |
| *** caowei has joined #openstack-kolla | 14:22 | |
| *** g3ek has joined #openstack-kolla | 14:23 | |
| sdake_ | pbourke - think tht will be helpful | 14:33 |
| sdake_ | pbourke thanks! | 14:33 |
| pbourke | np | 14:34 |
| *** ad_rien_ has quit IRC | 14:34 | |
| openstackgerrit | Paul Bourke proposed openstack/kolla: Set 'balance source' for Horizon in haproxy https://review.openstack.org/356514 | 14:35 |
| *** michauds has joined #openstack-kolla | 14:40 | |
| *** diogogmt has joined #openstack-kolla | 14:47 | |
| *** zhurong has quit IRC | 14:59 | |
| *** _coolsvap_ is now known as coolsvap | 15:01 | |
| *** openstackgerrit has quit IRC | 15:03 | |
| *** openstackgerrit has joined #openstack-kolla | 15:04 | |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla: Start using orchestration_engine variable https://review.openstack.org/356538 | 15:12 |
| openstackgerrit | Vladislav Belogrudov proposed openstack/kolla: Cinder cannot use external non-LVM iSCSI storage https://review.openstack.org/356540 | 15:15 |
| hrito | Hello, I want to propose additional functions to kolla. for that, what steps are required? after writing blueprint, need specs? | 15:16 |
| rhallisey | hrito, a blueprint in launchpad is perfect | 15:17 |
| rhallisey | assign it to yourself and desrcibe it | 15:17 |
| rhallisey | hrito, then you're good to go :) | 15:18 |
| openstackgerrit | Paul Bourke proposed openstack/kolla: Allow Neutron to be run in DVR mode https://review.openstack.org/356544 | 15:18 |
| rhallisey | and feel free to ask questions if you get stuck | 15:18 |
| *** vbel has joined #openstack-kolla | 15:20 | |
| *** huikang has quit IRC | 15:21 | |
| openstackgerrit | Paul Bourke proposed openstack/kolla: Allow Neutron to be run in DVR mode https://review.openstack.org/356544 | 15:21 |
| *** huikang has joined #openstack-kolla | 15:21 | |
| *** HyperJohnGraham has joined #openstack-kolla | 15:22 | |
| hrito | rhallisey: you meen after writing blueprint, just push patches? | 15:23 |
| sdake_ | hrito at this point in time we do not require specs | 15:23 |
| rhallisey | hrito, ya | 15:23 |
| sdake_ | that may change in the future especially to facilitate the new ptg structure | 15:23 |
| hrito | sdake_: rhallisey: ok! thanks! | 15:24 |
| kbaegis | dvr would be very nice | 15:24 |
| sdake_ | hrito - our team meeting is in 30 minutes if you want to find out more about ptg | 15:24 |
| *** huikang has quit IRC | 15:26 | |
| hrito | sdake_: ok, i'll join meeting :) | 15:26 |
| *** caowei has quit IRC | 15:29 | |
| jrich523 | should i be using kolla to build a node as well, or is it strictly contoller based? | 15:33 |
| sdake_ | build a node - which type | 15:35 |
| sdake_ | I dont understand the Q:) | 15:35 |
| openstackgerrit | Kevin Fox proposed openstack/kolla-kubernetes: WIP- openvswitch ip https://review.openstack.org/356114 | 15:35 |
| kfox1111 | grr. | 15:35 |
| jrich523 | sorry, compute | 15:35 |
| openstackgerrit | Kevin Fox proposed openstack/kolla-kubernetes: WIP- openvswitch ip https://review.openstack.org/356114 | 15:36 |
| sdake_ | jrich523 kolla will deploy controller, compute, and storage nodes for you | 15:36 |
| sdake_ | all you ahve to do is specify the machines in the inventory file | 15:36 |
| openstackgerrit | Kevin Fox proposed openstack/kolla: Add crudini https://review.openstack.org/356155 | 15:36 |
| jrich523 | oh, well where is this inventory file? | 15:36 |
| sdake_ | jrich523 - it will also operate them afterwards :) | 15:36 |
| jrich523 | well thats ideal :) | 15:36 |
| sdake_ | ansible/inventory directory | 15:37 |
| sdake_ | it can be overridden with kolla-ansible -i inventoryfile | 15:37 |
| sdake_ | thosse in there are just samples | 15:37 |
| jrich523 | let me go poke | 15:37 |
| jrich523 | thanks | 15:37 |
| kfox1111 | awesome... k8s scheduledjobs made it in for 1.4 :) | 15:37 |
| kfox1111 | we may want kolla-kubernetes to depend on 1.4 :) | 15:38 |
| kfox1111 | would give us a nice mechanism to do fernet key updating. | 15:38 |
| sdake_ | kfox1111 we have a release indepndent release schedule | 15:38 |
| sdake_ | and at this point its not 1.0.0 | 15:38 |
| sdake_ | so we can do whatever we like with changing things and have no concern for backward compatibility | 15:38 |
| sdake_ | or following any deprecation policy | 15:39 |
| rhallisey | kfox1111, considering how new this all is, we'll be constantly chasing the newest until we hit a release | 15:39 |
| sdake_ | i think the time to lock down a version is when kolla-ansible==1.0.0 | 15:39 |
| sdake_ | rhallisey can probably define what that looks like for us in our team meeting | 15:39 |
| jrich523 | sdake_: do you mean hosts? I found a /etc/ansible/hosts file, which seems to have the info in it | 15:39 |
| sdake_ | jrich523 oh you pip isntalled a stable version? | 15:40 |
| jrich523 | 1.9.4 | 15:40 |
| jrich523 | i cant remember if it was pip or yum | 15:40 |
| jrich523 | but yeah | 15:40 |
| sdake_ | it should be in the /usr/include/kolla/ansible/inventory directory | 15:40 |
| kfox1111 | rhallisey: I'd argue the other direction. due to how new kolla-kubernetes is, its better to get it working with a good archetecture, even thougth the features might be a little green as by the time we stabalize, so will the k8s features. | 15:40 |
| jrich523 | k, thanks | 15:40 |
| *** dmueller-evoila has joined #openstack-kolla | 15:40 | |
| jrich523 | god files are all over the place lol | 15:40 |
| sdake_ | jrich523 ya ansible unfortunately is a bit messy - /usr/share i mean | 15:41 |
| sdake_ | /usrs/hare is where we store the dockerfiles and ansible code that is run | 15:41 |
| kfox1111 | by kolla-kubernetes 1.0, k8s will probably be at least 1.6 | 15:41 |
| rhallisey | kfox1111, kolla-kube won't be 1.0 for a bit | 15:41 |
| ntpttr___ | hey inc0, you there? | 15:41 |
| kfox1111 | exactly. | 15:41 |
| kfox1111 | so lets kind of target where kubernetes will be when kolla-kubernetes is ready. | 15:42 |
| rhallisey | when we hit 1.0 we will develop features that are appropriate for the kube release | 15:42 |
| sdake_ | jrich523 fwiw - we follow fhs standaards :) | 15:42 |
| kfox1111 | otherwise, we'd be obsolete before we're ready. | 15:42 |
| rhallisey | we will be continuing to move the kube version as we add new features that require a new cube | 15:42 |
| rhallisey | releases will define what version that is | 15:42 |
| kfox1111 | by 1.0, init containers, deployments, petsets, and scheduledjobs should all be stable. | 15:43 |
| rhallisey | but I'm not sure yet when that will be so we can continue to roll | 15:43 |
| sdake_ | rhallisey i can provide some insight here - once 1.0.0 is created, we createe a stable branch | 15:43 |
| jrich523 | im not sure what fhs is :-/ | 15:43 |
| sdake_ | new features go in master, stable left alone :) | 15:43 |
| sdake_ | filesystem hiearchy standard | 15:43 |
| rhallisey | my estimate will be O | 15:43 |
| sdake_ | i think so too | 15:43 |
| jrich523 | yeah, im a windows guy, the linux structure has always confused me | 15:43 |
| rhallisey | for a stable branch | 15:43 |
| jrich523 | not that windows is straight forward, but you know... its what i use | 15:43 |
| sdake_ | that is when 1.0.0 happens | 15:43 |
| rhallisey | roger | 15:44 |
| sdake_ | 1.0.0 = stable branch | 15:44 |
| kfox1111 | jrich523: I generally like the fhs, but the problem with it is so few folks know it, there are are far to many exceptions. | 15:44 |
| jrich523 | well, i mean really, what does /etc or /var really mean... as far as i know they dont relate to a word that describes what they are | 15:44 |
| kfox1111 | the fhs defines that. | 15:44 |
| jrich523 | at least in windows you have 'program data', 'program files' fairly obvious | 15:44 |
| kfox1111 | /etc is basically for config files. | 15:44 |
| *** thumpba has quit IRC | 15:44 | |
| kfox1111 | /var is stuff that needs to change as the machine runes. | 15:45 |
| sdake_ | jrich523 /var = writeable /usr = readonly | 15:45 |
| kfox1111 | runs | 15:45 |
| kfox1111 | /usr/ readonly. | 15:45 |
| jrich523 | well, yeah but why do i need a decoder ring? i guess back in the day went folder length mattered... or tab complete didnt exist :) | 15:45 |
| kfox1111 | so if you nfs mount and share your root file system, /usr goes in there. | 15:45 |
| sbezverk | rhallisey kfox1111 it is nice to version planning :-) but we have bunch of reviews in the queue | 15:45 |
| sdake_ | jrich523 it really does help to have these things standard across distros | 15:45 |
| kfox1111 | jrich523: because unlike windows, you can take one os install and spread it across 2000 machines. :) | 15:45 |
| sdake_ | so we dont have to figure out 15 different ways to install the ODM | 15:45 |
| *** matrohon has quit IRC | 15:46 | |
| kfox1111 | sbezverk: +1 | 15:46 |
| jrich523 | im not sure i follow kfox1111, i spew windows all over the place :) | 15:46 |
| rhallisey | :) | 15:46 |
| sdake_ | ya the review queue is in overload | 15:46 |
| kfox1111 | jrich523: for linux clusters, you take a nfs server (smb equivilent) install linux on that one share, | 15:48 |
| jrich523 | so there are two files in the inventory folder, all-in-one and multinode, looking at both of them, they look like config types | 15:48 |
| kfox1111 | and pxe boot a kernel pointing to that share as the root. | 15:48 |
| jrich523 | oh a shared os, gotcha... guess you could do that with windows... but... probably not cool :) | 15:48 |
| kfox1111 | there is no install. there is one os install running across 2000 nodes. | 15:49 |
| kfox1111 | windows can't do that. you have to install windows on each individual machine. | 15:49 |
| kfox1111 | doing that way lets you make one change to the share and instantaniously all 2000 nodes are updated. | 15:49 |
| lrensing_ | hey guys- when running kolla-ansible deploy i started getting this error and can't seem to resolve it: | 15:49 |
| lrensing_ | Hostname has to resolve to IP address of api_interface | 15:49 |
| duonghq | lrensing_ do you run Ubuntu/Debian-based? | 15:50 |
| lrensing_ | ubuntu, yes | 15:50 |
| duonghq | you should remove 127.0.1.1 in /etc/hosts | 15:50 |
| duonghq | replace with real ip | 15:50 |
| lrensing_ | doh | 15:51 |
| duonghq | the 127.0.1.1 is placed as as fallback for some software but cause trouble to Kolla | 15:51 |
| jrich523 | sdake_: so it looks like the inventory folder you sent me to, contains configs (all-in-one, and multinode) and the place to add a new host is in the /etc/ansible/hosts file i found | 15:51 |
| jrich523 | with those two configs tho, it seems like i'd need to build another for a compute only node? | 15:52 |
| sdake_ | you need to make your own or edit the all-in-one | 15:52 |
| sdake_ | the all-in-one is default | 15:52 |
| sdake_ | but it can be overidden with the -i flag | 15:52 |
| jrich523 | ok | 15:52 |
| sdake_ | how I'd like to see this work in teh future is invetory files stored in /etc | 15:52 |
| sdake_ | since its a configuration file nto a read only file :) | 15:53 |
| jrich523 | so copy that to computenode (lets say) and modify it, do i need to import that file after or is the location good enough? | 15:53 |
| kfox1111 | rhallisey: is there an example of launching the same service template with different args defined in the serservice_resources file? | 15:53 |
| jrich523 | that does make sense :) | 15:53 |
| sdake_ | jrich523 keep it on the deployment node | 15:53 |
| sdake_ | nothing needs to be copied to the compute nodes | 15:53 |
| sdake_ | jrich523 if you open that file up it has a [compute] section | 15:53 |
| jrich523 | no i mean, to a new file (deployment config) | 15:54 |
| sdake_ | just add your ips in | 15:54 |
| jrich523 | so cp all-in-one computenote | 15:54 |
| jrich523 | edit computenode | 15:54 |
| sdake_ | oh right, i'd copy all-in-one to /etc/kolla | 15:54 |
| kfox1111 | rhallisey: oh, I think I see it. vars: ... | 15:54 |
| jrich523 | add my box to the hosts file | 15:54 |
| sdake_ | right that will do it | 15:54 |
| jrich523 | so if i do it in /etc/kolla i have to do the kolla-ansible -i computenode | 15:54 |
| rhallisey | kfox1111, I think var's may be what you're after | 15:54 |
| jrich523 | which, basically just copies it to the inventory folder? | 15:54 |
| duonghq | jrich523: I think you should take a scratch course about ansible, this will help | 15:54 |
| jrich523 | duonghq: yeah reading their docs now | 15:55 |
| sdake_ | jrich523 you have to specify the full path to the inventory file with kolla-ansible | 15:55 |
| jrich523 | k | 15:55 |
| kfox1111 | sbezverk: I think in the case of different interfaces on different types of hardware, we need a way to run multiple daemonsets with overrides of the interface. :/ | 15:55 |
| kfox1111 | I think we can use the templating to do that though. | 15:55 |
| kfox1111 | we really need to discuss node selectors | 15:55 |
| jrich523 | i think i've got enough to go break some things, thanks :) | 15:55 |
| kfox1111 | though. | 15:55 |
| sdake_ | jrich523 happy to help | 15:56 |
| kfox1111 | rhallisey: sbezverk: what do you think about kolla-compute=v1 and kolla-controller=v1 for now? | 15:56 |
| *** dmueller-evoila has quit IRC | 15:57 | |
| rhallisey | kfox1111, as long as it's templated | 15:57 |
| rhallisey | we can default those however | 15:58 |
| *** ajpower has joined #openstack-kolla | 15:58 | |
| inc0 | you know what time is it? Its meeeeeeting time! | 15:58 |
| rhallisey | s/v1/<something_more_descriptive>/ | 15:58 |
| duonghq | count down | 15:58 |
| sdake_ | big countdown atleast 120 numbers to count :) | 15:59 |
| duonghq | 64sec actually :) | 16:00 |
| ajpower | #startmeeting kolla | 16:00 |
| *** lrensing_ has quit IRC | 16:00 | |
| openstack | Meeting started Wed Aug 17 16:00:25 2016 UTC and is due to finish in 60 minutes. The chair is ajpower. Information about MeetBot at http://wiki.debian.org/MeetBot. | 16:00 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 16:00 |
| openstack | The meeting name has been set to 'kolla' | 16:00 |
| *** sdake_ is now known as sdake | 16:00 | |
| rhallisey | ajafo, wrong location | 16:00 |
| *** sdake is now known as sdake_ | 16:00 | |
| rhallisey | ajpower, , #openstack-meeting-4 | 16:01 |
| rhallisey | ajafo, sorry wrong person | 16:01 |
| *** egonzalez has joined #openstack-kolla | 16:02 | |
| *** ajpower has quit IRC | 16:03 | |
| *** ajpower has joined #openstack-kolla | 16:04 | |
| *** dmueller-evoila has joined #openstack-kolla | 16:04 | |
| ajpower | #openstack-meeting-4 | 16:04 |
| *** egonzalez90 has quit IRC | 16:05 | |
| *** egonzalez is now known as egonzalez90 | 16:05 | |
| openstackgerrit | Merged openstack/kolla: Replace horizon default config with custom config https://review.openstack.org/306928 | 16:05 |
| *** dmueller-evoila_ has joined #openstack-kolla | 16:07 | |
| rhallisey | ajafo, `/join #openstack-meeting-4` | 16:07 |
| rhallisey | ajpower, ^ | 16:07 |
| rhallisey | sorry ajafo | 16:07 |
| rhallisey | :/ | 16:07 |
| *** mark-casey has joined #openstack-kolla | 16:08 | |
| ajpower | join #openstack-meeting-4 | 16:08 |
| *** dmueller-evoila has quit IRC | 16:08 | |
| rhallisey | ajafo, add a / | 16:08 |
| *** dmueller-evoila_ has quit IRC | 16:13 | |
| *** Serlex has quit IRC | 16:17 | |
| kbaegis | So what's the procedure/strategy in kolla to upgrade a single container? | 16:17 |
| *** vhosakot has joined #openstack-kolla | 16:18 | |
| *** v1k0d3n has joined #openstack-kolla | 16:20 | |
| *** ChanServ changes topic to "Kolla IRC meetngs on Wednesdays @ 16:00 UTC - see agenda @ https://goo.gl/OXB0DL - Community Submitted Talks for Occata https://goo.gl/s8Wldv - OSIC cluster testing etherpad https://goo.gl/vAMGB6 - IRC channel is *LOGGED* @ http://goo.gl/3mzZ7b (old logs from #kolla http://goo.gl/VKpPzA)" | 16:27 | |
| kbaegis | Is there an individual runbook I can deploy to move horizon between versions, for example? | 16:33 |
| *** unicell1 has quit IRC | 16:33 | |
| rhallisey | kbaegis, you can run play specific for a service with tags | 16:33 |
| rhallisey | so you can run the upgrade play with the horizon tags and that should get what you want | 16:33 |
| *** HyperJohnGraham has quit IRC | 16:34 | |
| kbaegis | so the command would be kolla-ansible upgrade horizon? | 16:35 |
| kbaegis | or would I alter globals and select a horizon version? | 16:35 |
| rhallisey | kbaegis, kolla-ansible upgrade --tags horizon | 16:36 |
| kbaegis | Oh, nice :) | 16:37 |
| *** JoseMello has joined #openstack-kolla | 16:37 | |
| kbaegis | Latest issue: http://hastebin.com/pijeqimofi.vbs | 16:47 |
| kbaegis | Anyone played around with non-cooperative iptables implementations and willing to help? | 16:47 |
| *** ajpower has quit IRC | 16:48 | |
| kbaegis | Tap device is created successfully, bridge too | 16:48 |
| kbaegis | It's just these dang iptables rules :) | 16:48 |
| *** huikang has joined #openstack-kolla | 16:49 | |
| kbaegis | Running the commands from the log against iptables manually I get errors like | 16:50 |
| kbaegis | iptables: No chain/target/match by that name. | 16:50 |
| kbaegis | or iptables: Index of insertion too big | 16:50 |
| *** Jeffrey4l has quit IRC | 16:52 | |
| kbaegis | Huh. Maybe it needs CONFIG_INET_DIAG_DESTROY | 16:53 |
| inc0 | coolsvap, so ad rally | 16:53 |
| * britthouser listens in | 16:53 | |
| inc0 | it should work now, we just need to change deployments | 16:53 |
| duonghq | hey sdake_, the bug, do we have any replacement for deprecated tag option? | 16:53 |
| inc0 | cuz access creds are invalid | 16:53 |
| ntpttr___ | hey inc0 I'm running into an issue with my cinder/ceph deployment in kolla, do you have a moment? | 16:54 |
| inc0 | sure | 16:54 |
| inc0 | what's up Nate? | 16:54 |
| sdake_ | duonghq i dont have a clear picture of the issue yet duonghq | 16:54 |
| ntpttr___ | inc0: so everything is starting up fine and I can create and attach volumes and all, but the backup service isn't working | 16:54 |
| sdake_ | duonghq - but there are docs in teh queue addressing it | 16:54 |
| sdake_ | they are just wrong | 16:54 |
| ntpttr___ | inc0: basically whenever I try to backup a volume, it hangs in the 'creating' phase forever | 16:54 |
| sdake_ | duonghq so the solution first is to document a workaround | 16:54 |
| coolsvap | inc0: hi | 16:54 |
| inc0 | and no errors? | 16:54 |
| sdake_ | duonghq then to fix the pbr issue | 16:54 |
| ntpttr___ | and it seems like privsep just isn't starting up | 16:54 |
| coolsvap | inc0: where are we currently wrt rally | 16:55 |
| ntpttr___ | in the logs, here's mine: http://paste.openstack.org/show/558606/ and here's one where it works: http://paste.openstack.org/show/558537/ | 16:55 |
| ntpttr___ | the privsep daemon isn't starting, but there's no errors | 16:55 |
| duonghq | sdake_: I look through the code and cannot find any replacement for deprecated option, I don't think its normal | 16:55 |
| inc0 | coolsvap, can you fix deployments in rally? our creds are wrong | 16:56 |
| duonghq | sdake_ fix me if I wrong :) | 16:56 |
| *** bootsha has joined #openstack-kolla | 16:56 | |
| inc0 | ntpttr___, it might be a bug | 16:56 |
| coolsvap | inc0: sure let me connect to vpn | 16:56 |
| inc0 | I haven't seen this before | 16:56 |
| ntpttr___ | inc0: it looks like that might be the case, this is on a clean install with no errors or anything | 16:56 |
| ntpttr___ | has anyone here deployed cinder w/ ceph and then successfully used the backup service before? | 16:57 |
| inc0 | our tests on osic cluster should show that | 16:57 |
| duonghq | sdake_: I think it should be a option in kolla-build.conf for tagging? | 16:58 |
| sdake_ | i dont think --tag should be a deprecated option at all | 16:59 |
| ntpttr___ | inc0: do you think I should go ahead and file a bug against kolla? | 16:59 |
| sdake_ | i dont know why it is | 16:59 |
| coolsvap | inc0: we can update the credentials in osic.json file on the node | 17:00 |
| inc0 | ntpttr___, yeah, file it | 17:00 |
| duonghq | sdake_: somebody marked it deprecated? | 17:00 |
| *** HyperJohnGraham has joined #openstack-kolla | 17:00 | |
| inc0 | coolsvap, might doing that? | 17:01 |
| coolsvap | inc0: i am updating it from /etc/kolla/admin-openrc.sh | 17:01 |
| inc0 | I dunno where it is | 17:01 |
| coolsvap | yes | 17:01 |
| inc0 | tmux a plz | 17:01 |
| inc0 | I'll watch it | 17:01 |
| coolsvap | yeah sorry | 17:01 |
| coolsvap | just a min | 17:01 |
| inc0 | np:) | 17:01 |
| *** rstarmer has joined #openstack-kolla | 17:02 | |
| *** egonzalez90 has quit IRC | 17:03 | |
| ntpttr___ | okay got the bug report up https://bugs.launchpad.net/kolla/+bug/1614180 | 17:03 |
| openstack | Launchpad bug 1614180 in kolla "Backup w/ ceph hangs when creating a backup" [Undecided,New] | 17:03 |
| *** fragatina has joined #openstack-kolla | 17:05 | |
| *** hrito has left #openstack-kolla | 17:07 | |
| openstackgerrit | Kevin Fox proposed openstack/kolla-kubernetes: WIP- openvswitch ip https://review.openstack.org/356114 | 17:09 |
| ntpttr___ | inc0: do we need to be installing oslo.privsep somewhere? I don't see it anywhere in the kolla repo, or in this file where I see a lot of other oslo libraries https://github.com/openstack/kolla/blob/5f4ef0d84bf1d1f5f72f245a0dc9e95d25e0b92d/docker/openstack-base/Dockerfile.j2 | 17:10 |
| ntpttr___ | in devstack they're installing it https://github.com/openstack-dev/devstack/blob/06f3639a70dc5884107a4045bef5a9de1fb725a5/lib/oslo | 17:10 |
| openstackgerrit | Kevin Fox proposed openstack/kolla-kubernetes: WIP- openvswitch ip https://review.openstack.org/356114 | 17:11 |
| *** inc0 has quit IRC | 17:13 | |
| *** HyperJohnGraham has quit IRC | 17:16 | |
| *** bootsha has quit IRC | 17:18 | |
| Pavo | evening everyone | 17:19 |
| *** Guest52040 has quit IRC | 17:20 | |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack/kolla: Update cirros image visibility in init-runonce https://review.openstack.org/353933 | 17:20 |
| *** bootsha has joined #openstack-kolla | 17:21 | |
| *** bootsha has quit IRC | 17:22 | |
| *** bootsha has joined #openstack-kolla | 17:23 | |
| kbaegis | gah. Linuxbridging broken still on gentoo | 17:26 |
| *** unicell has joined #openstack-kolla | 17:26 | |
| *** bootsha has quit IRC | 17:26 | |
| coolsvap | pbourke: do yo uwhave the kolla-rally-tests.sh script open in your terminal? | 17:27 |
| *** fragatina has quit IRC | 17:30 | |
| kfox1111 | sbezverk: rhallisey: so, for the glance bootstrap one. we postponed talking about dealing with the init-container templating thing for dealing with the json as we didn't have access to jq. that patch just merged though. | 17:31 |
| *** unicell has quit IRC | 17:31 | |
| *** fragatina has joined #openstack-kolla | 17:31 | |
| sbezverk | kfox1111: I would suggest to merge what I have now as I tested it multiple times and it is working | 17:32 |
| kfox1111 | can we reconsider this: https://review.openstack.org/#/c/353740/4..6/services/neutron/neutron-bootstrap-job.yml.j2 | 17:32 |
| patchbot | kfox1111: patch 353740 - kolla-kubernetes - Neutron bootstrap to create service/project/role | 17:32 |
| Pavo | kbaegis was it you asking about deploying only 1 of the images with kolla-ansible yesterday? | 17:32 |
| *** duonghq has left #openstack-kolla | 17:32 | |
| rhallisey | kfox1111, I mean we can iterate on it | 17:32 |
| sbezverk | if you feel like modifying it, please submit PS and we can have discussion | 17:32 |
| *** bootsha has joined #openstack-kolla | 17:32 | |
| kfox1111 | k. | 17:32 |
| sbezverk | it is just so many things are hanging because of this | 17:33 |
| kfox1111 | sure. but I just want to avoid having to do more when we add more bootstrapping to nova and friends too. | 17:34 |
| sbezverk | kfox1111: I do not think we add more than just nova | 17:34 |
| sbezverk | any time soon | 17:34 |
| kfox1111 | we can merge these 2 as is. | 17:34 |
| kfox1111 | cinder? | 17:34 |
| kfox1111 | heat? | 17:34 |
| sbezverk | so if you propose PS after, I will help you to apply on other components | 17:34 |
| kfox1111 | k | 17:35 |
| sbezverk | If I undersatand the primary goal is get VM | 17:35 |
| kfox1111 | ah. we can claim vm without cinder/heat, true. | 17:36 |
| sbezverk | kfox1111 yep, I think it make sense | 17:36 |
| *** rstarmer has quit IRC | 17:37 | |
| kfox1111 | one more thing and I think the glance one's good then. | 17:37 |
| kfox1111 | do we actually need the pvc in the bootstrap? | 17:37 |
| kfox1111 | I don't really see where its used, and it shouldn't be there if using the ceph backend. | 17:38 |
| kfox1111 | I think we can probably just drop it all together. | 17:38 |
| *** bootsha has quit IRC | 17:39 | |
| sbezverk | kfox1111: I will need to retest it | 17:39 |
| sdake_ | pbourke you still around? | 17:40 |
| sdake_ | i dont blame ya | 17:40 |
| sdake_ | sweet dreams :) | 17:40 |
| kfox1111 | wait.. | 17:42 |
| kfox1111 | its used here: | 17:43 |
| *** bootsha has joined #openstack-kolla | 17:43 | |
| kfox1111 | https://review.openstack.org/#/c/346215/11/services/glance/glance-bootstrap-job.yml.j2 line 76 | 17:43 |
| patchbot | kfox1111: patch 346215 - kolla-kubernetes - Glance bootstrap to create service/project/role | 17:43 |
| kfox1111 | whats the api container being used for? | 17:43 |
| sbezverk | kfox1111: it looks like it is used during bootstrap process. | 17:44 |
| kfox1111 | where? | 17:44 |
| kfox1111 | I see database creation, and keystone registration. but not much else? | 17:44 |
| sbezverk | kfox1111: it does this sudo chown -R glance: /var/lib/glance/ | 17:46 |
| kfox1111 | hmm.... | 17:47 |
| *** HyperJohnGraham has joined #openstack-kolla | 17:47 | |
| kfox1111 | oh. cause the KOLLA_BOOTSTRAP. | 17:47 |
| sbezverk | kfox1111: if we do not have it mounted, it will be done against virtual directory | 17:47 |
| sbezverk | kfox1111: right | 17:47 |
| kfox1111 | k. | 17:48 |
| sbezverk | kfox1111: it is not much but please do not ask me to move it somewhere else | 17:48 |
| kfox1111 | so then I think we just need to do something like: https://review.openstack.org/#/c/354895/4/services/glance/glance-api-pod.yml.j2 | 17:48 |
| patchbot | kfox1111: patch 354895 - kolla-kubernetes - WIP - Deployment, Safe Shutdown, & Scaling for gla... | 17:48 |
| kfox1111 | with glance_backend_ceph | 17:48 |
| kfox1111 | around the volume and volumeMount. | 17:49 |
| kbaegis | Pavo Yes :) | 17:49 |
| sbezverk | kfox1111: can you add it on top after it is merged? | 17:49 |
| sbezverk | I will not be able to test it | 17:49 |
| sbezverk | I just do not have means | 17:50 |
| kfox1111 | k. | 17:50 |
| Pavo | you can use the --tags argument to only deploy whatever containers you want | 17:50 |
| kfox1111 | +2 | 17:50 |
| sbezverk | kfox1111: thank you, as soon as it is merged please submit your patch | 17:51 |
| kfox1111 | I guess I can just rebase and fix in https://review.openstack.org/#/c/354895/4/services/glance/glance-api-pod.yml.j2 once merged. | 17:51 |
| patchbot | kfox1111: patch 354895 - kolla-kubernetes - WIP - Deployment, Safe Shutdown, & Scaling for gla... | 17:51 |
| kbaegis | Pavo Or so I've been told :) | 17:51 |
| kbaegis | It's a cool feature | 17:51 |
| *** inc0 has joined #openstack-kolla | 17:52 | |
| *** HyperJohnGraham has quit IRC | 17:52 | |
| Pavo | example kolla-ansible deploy -i ansible/inventory/multinode --tags horizon, will only redeploy horizon on the node thats in your inventory list | 17:52 |
| *** HyperJohnGraham_ has joined #openstack-kolla | 17:53 | |
| *** HyperJohnGraham_ is now known as HyperJohnGraham | 17:53 | |
| sbezverk | kfox1111: rhallisey: we need to decide what to do with secret generator. All these bootstraps are consuming secrets. So it would make sense to include it now just for a timebeing as a separate tool.. | 17:55 |
| rhallisey | sbezverk, I think we might be putting the cart ahead of the horse a little | 17:55 |
| kfox1111 | I'm ok merging the secret generator as is for now. | 17:56 |
| rhallisey | we need to focus on a few things, accomplish them, and tackle new ones | 17:56 |
| rhallisey | we have a bunch of efforts happening and it's hard to track | 17:56 |
| sbezverk | kfox1111: did not want to ack without it ;-) | 17:56 |
| kfox1111 | sbezverk: makes sense. | 17:56 |
| rhallisey | 1) lets get the core services working | 17:56 |
| rhallisey | 2) CLI/ansible | 17:56 |
| rhallisey | so hold on to this script for noew | 17:57 |
| *** HyperJohnGraham_ has joined #openstack-kolla | 17:57 | |
| rhallisey | the secrets bit let's handle after we get the services working | 17:57 |
| inc0 | coolsvap, O | 17:57 |
| inc0 | I'm backl | 17:57 |
| rhallisey | kfox1111, sbezverk is that ok? | 17:57 |
| rhallisey | we need to focus on the big tasks first | 17:58 |
| sbezverk | then we need to revert bootstrap changes | 17:58 |
| kfox1111 | rhallisey: the secrets are already staring to land. we just need a way to get them into k8s now. | 17:58 |
| coolsvap | inc0: i am logging off | 17:58 |
| kfox1111 | the script is less then ideal, but pretty simple and works. | 17:58 |
| rhallisey | kfox1111, I'm saying let's not block on that | 17:58 |
| kfox1111 | we can just merge it and then refactor it later when we have more time. | 17:58 |
| rhallisey | service can land | 17:58 |
| rhallisey | then we can do that | 17:58 |
| inc0 | coolsvap, did you move anything forward? | 17:58 |
| coolsvap | i validate the scenarios, some amount of work is needed | 17:58 |
| kfox1111 | ah. | 17:58 |
| coolsvap | validated the scenarios | 17:58 |
| *** HyperJohnGraham has quit IRC | 17:58 | |
| kfox1111 | ok. that works too. | 17:58 |
| kfox1111 | we can review the generator independendly then. | 17:59 |
| rhallisey | kfox1111, ya let's tackle these problems in smaller bits | 17:59 |
| kfox1111 | its just weird to leave trunk unusable. without other patches. | 17:59 |
| *** papacz has quit IRC | 17:59 | |
| kfox1111 | though we have the same issue with kolla. | 17:59 |
| rhallisey | kfox1111, it | 17:59 |
| rhallisey | it's the nature of master in a new project to be a little unstbale | 17:59 |
| kfox1111 | yup. I get that. | 18:00 |
| rhallisey | cool cool | 18:00 |
| kfox1111 | kk. | 18:00 |
| sbezverk | kfox1111: rhallisey: we need to revert back to old bootstraps if you do not want to use the script.. | 18:00 |
| *** HyperJohnGraham has joined #openstack-kolla | 18:00 | |
| rhallisey | sbezverk, is there a way we can have it for one service? | 18:01 |
| rhallisey | glance has it right? | 18:01 |
| kfox1111 | sbezverk: he's saying we should just note that you need the extra script from review xxx to work. | 18:01 |
| sbezverk | glance, neutron and keystone | 18:01 |
| rhallisey | sbezverk, we can doc it | 18:01 |
| kfox1111 | we're already doing that with kolla too. | 18:01 |
| openstackgerrit | Merged openstack/kolla-kubernetes: Glance bootstrap to create service/project/role https://review.openstack.org/346215 | 18:02 |
| rhallisey | we're getting close to landing the core services so we can return to this soon | 18:02 |
| sbezverk | rhallisey eitherway works for me as lons as we move forward :-) | 18:02 |
| rhallisey | kfox1111, yes true, it will be growing pains for now | 18:02 |
| kfox1111 | rhallisey: thoughts on https://review.openstack.org/#/c/356114 | 18:02 |
| patchbot | kfox1111: patch 356114 - kolla-kubernetes - WIP- openvswitch ip | 18:02 |
| *** HyperJohnGraham_ has quit IRC | 18:02 | |
| rhallisey | we won't really get a sense of stability until we have all the core services in place with our CLI/workflow done | 18:02 |
| kfox1111 | I think we can probably pull the WIP off of it, but I haven't really been able to test it yet. | 18:03 |
| rhallisey | brb guys | 18:03 |
| kfox1111 | yeah, I was alittle afraid of that... all the json stuff in the kolla-config all conflict with each other. | 18:05 |
| kfox1111 | so neutron/keystone bootstrap secret reviews now have merge issues. | 18:05 |
| *** HyperJohnGraham has quit IRC | 18:05 | |
| *** bootsha has quit IRC | 18:07 | |
| *** unicell has joined #openstack-kolla | 18:07 | |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes: Neutron bootstrap to create service/project/role https://review.openstack.org/353740 | 18:12 |
| sbezverk | kfox1111: resolved it | 18:13 |
| kfox1111 | +2'ed it. :) | 18:13 |
| *** Pavo has quit IRC | 18:17 | |
| *** Pavo has joined #openstack-kolla | 18:19 | |
| kfox1111 | rhallisey: sbezverk: we should probably talk more about https://review.openstack.org/#/c/354895 too. | 18:19 |
| patchbot | kfox1111: patch 354895 - kolla-kubernetes - WIP - Deployment, Safe Shutdown, & Scaling for gla... | 18:20 |
| *** tummy has joined #openstack-kolla | 18:20 | |
| kfox1111 | I think we will need to set the api address to 127.0.0.1 for it to work properly, but then all services will need similar patches as there is only one var for api ip right now. | 18:20 |
| openstackgerrit | Kevin Fox proposed openstack/kolla-kubernetes: WIP - Deployment, Safe Shutdown, & Scaling for glance-api https://review.openstack.org/354895 | 18:21 |
| *** tummy has quit IRC | 18:21 | |
| openstackgerrit | Kevin Fox proposed openstack/kolla-kubernetes: WIP - Deployment, Safe Shutdown, & Scaling for glance-api https://review.openstack.org/354895 | 18:23 |
| *** gfidente has quit IRC | 18:24 | |
| sbezverk | kfox1111: have you tested this patch with other services communicating between each other? | 18:29 |
| sbezverk | cause this approach chnages also interprocess communication if I undersatnd correctly the idea | 18:30 |
| kfox1111 | I have not yet, as I need a solution to configuring it to listen on 127.0.0.1 instead of 0.0.0.0 | 18:31 |
| kfox1111 | I didn't think glance did rpc. does it? | 18:31 |
| kfox1111 | I think for now, so I don't break everything to test, I can use an init container and override the config from 0.0.0.0 to 127.0.0.1. | 18:32 |
| kfox1111 | but I need the crudini patch to land in kolla. | 18:32 |
| kfox1111 | then we should be able to apply the same pattern to the other apis. | 18:33 |
| kfox1111 | we probably want to split the rpc services from the api services anyway. | 18:33 |
| kfox1111 | mostly I think thats already done, except maybe neutron-server I think does both. | 18:34 |
| kfox1111 | the neutron-server api's should be deployments and the neutron-server rpc's should be petsets I think. | 18:34 |
| kfox1111 | thought neturon did say its safe to restart a neutron-server these days. | 18:35 |
| kfox1111 | so the rpc stuff should in theory be safe to do mixed up in the api server with a deployment if that actually is true. | 18:35 |
| *** bmace has joined #openstack-kolla | 18:36 | |
| *** inc0 has quit IRC | 18:37 | |
| *** tummy has joined #openstack-kolla | 18:44 | |
| *** tummy has quit IRC | 18:45 | |
| *** egonzalez has joined #openstack-kolla | 18:46 | |
| *** egonzalez is now known as egonzalez90 | 18:47 | |
| openstackgerrit | Kevin Fox proposed openstack/kolla-kubernetes: WIP - Deployment, Safe Shutdown, & Scaling for glance-api https://review.openstack.org/354895 | 18:52 |
| kfox1111 | k. once the crudini kolla patch lands, I think that one should probably work. needs a bit of testing once landed. | 18:52 |
| kfox1111 | we should then be able to apply the pattern to the other api services. | 18:52 |
| *** egonzalez90 has quit IRC | 18:59 | |
| *** athomas has quit IRC | 19:01 | |
| *** thumpba has joined #openstack-kolla | 19:05 | |
| *** wznoinsk has quit IRC | 19:07 | |
| *** egonzalez90 has joined #openstack-kolla | 19:08 | |
| sdake_ | 30 mintes afk | 19:08 |
| sdake_ | bbiaf | 19:08 |
| kbaegis | So I'm getting timeouts | 19:14 |
| kbaegis | between ovsdb and openvswitch | 19:14 |
| openstackgerrit | Leon Zachery proposed openstack/kolla: Simplify install process for faster quickstart execution https://review.openstack.org/356181 | 19:15 |
| *** inc0 has joined #openstack-kolla | 19:19 | |
| kbaegis | http://hastebin.com/xiyenufera.vbs | 19:24 |
| *** ad_rien_ has joined #openstack-kolla | 19:24 | |
| kbaegis | Linux bridging isn't working for me because of the iptables implementation, ovs is having timeouts between ovsdb and ovs-vswitchd | 19:25 |
| kbaegis | So I can create volumes, upload images, spawn instances, but the network doesn't work for the L2 agents available | 19:25 |
| *** wznoinsk has joined #openstack-kolla | 19:26 | |
| kbaegis | All devices are getting created just fine (taps and bridges). | 19:26 |
| kbaegis | OVS isn't up for more than 10 seconds due to not being able to reach ovsdb | 19:26 |
| *** britthou_ has joined #openstack-kolla | 19:27 | |
| *** britthouser has quit IRC | 19:27 | |
| *** ayoung has quit IRC | 19:27 | |
| openstackgerrit | Kevin Fox proposed openstack/kolla-kubernetes: WIP - Deployment, Safe Shutdown, & Scaling for glance-api https://review.openstack.org/354895 | 19:29 |
| sdake_ | kbaegis my bet is rtnetlink | 19:30 |
| sdake_ | but it could be the iptables integration with gentoo - which there probabyl sin't any | 19:31 |
| kbaegis | sdake_ I need to become an authority on rtnetlink then. :) | 19:32 |
| sdake_ | i think my second guess might yield more results | 19:32 |
| sdake_ | but not sure | 19:32 |
| kbaegis | sdake_ Like I said, it's able to create functioning tap devices | 19:32 |
| sdake_ | they are both speculation | 19:32 |
| kbaegis | And they'll be plugged into the bridge | 19:33 |
| sdake_ | i missed that part | 19:33 |
| sdake_ | probably some iptables issue | 19:33 |
| sdake_ | do you hae iptables enabled by default in gentoo? | 19:33 |
| kbaegis | It fails on the secgroup rules, yes | 19:33 |
| kbaegis | Here's the traceback http://hastebin.com/pijeqimofi.vbs | 19:33 |
| kbaegis | And yes, iptables is fully implemented in the kernel (all opts), and installed with all use-flags | 19:34 |
| kbaegis | I rebuilt/repushed centos from source after | 19:34 |
| kbaegis | I can validate that this is the only remaining issue for a base deployment on gentoo | 19:34 |
| kbaegis | So if I can crack this, I don't have to use fuel-cpp on k8s | 19:35 |
| kbaegis | Which is my personal preference :) | 19:35 |
| sdake_ | not sure - it could be either rtnetlink or iptables implementation in neutron | 19:36 |
| sdake_ | it is debuggable | 19:36 |
| sdake_ | since you can get a backtrace | 19:36 |
| sdake_ | protip print ftw :) | 19:36 |
| sdake_ | before doing that though, did you have iptables enabled by default in gentoo? | 19:37 |
| sdake_ | not built into the kernel | 19:37 |
| sdake_ | but enabled in the sysv scripts | 19:37 |
| sdake_ | or systemd or whatever one your using | 19:37 |
| kbaegis | Hmm | 19:38 |
| kbaegis | yikes. https://wiki.gentoo.org/ is down | 19:39 |
| sdake_ | wfm | 19:39 |
| *** ad_rien_ has quit IRC | 19:39 | |
| kbaegis | Huh. Did /etc/init.d/iptables restart and now I'm getting a new error- modprobe: FATAL: Module openvswitch not found | 19:41 |
| sdake_ | ya | 19:41 |
| kbaegis | Is ovs required as a module, not as a builtin? | 19:41 |
| sdake_ | dont do that | 19:41 |
| sdake_ | not sure on ovs being a module | 19:41 |
| sdake_ | iptables should be disabled | 19:41 |
| sdake_ | not in an enabled state | 19:42 |
| sdake_ | i wanted to know if you had it enabled previously | 19:42 |
| sdake_ | chkconfig status iptables ought to do it | 19:42 |
| kbaegis | I hadn't | 19:43 |
| sdake_ | so it wasn't enabled? | 19:43 |
| kbaegis | I just saved the default rules and | 19:43 |
| sdake_ | ok | 19:43 |
| kbaegis | It wasn't setup by sysvinit | 19:43 |
| sdake_ | lets start fresh | 19:44 |
| sdake_ | reboot your boxes | 19:44 |
| sdake_ | cleanup first | 19:44 |
| kbaegis | :'( okay | 19:44 |
| openstackgerrit | Leon Zachery proposed openstack/kolla: Simplify install process for faster quickstart execution https://review.openstack.org/356181 | 19:44 |
| sdake_ | but dont delete your image | 19:44 |
| sdake_ | s | 19:44 |
| sdake_ | all we are going to do is iptables --flush | 19:44 |
| sdake_ | prior to deploy | 19:44 |
| kbaegis | Okay | 19:44 |
| sdake_ | on the reboot | 19:44 |
| sdake_ | so whtaever you ahve eto do to make that happen, i'm sure you know by now :) | 19:44 |
| sdake_ | do on all nodes | 19:45 |
| -openstackstatus- NOTICE: The volume for logs.openstack.org filled up rather suddenly, causing a number of jobs to fail with a POST_FAILURE result and no logs; we're manually expiring some logs now to buy breathing room, but any changes which hit that in the past few minutes will need to be rechecked and/or approved again | 19:45 | |
| kbaegis | kolla-ansible destroy runs all the cleanup scripts, correct? | 19:45 |
| kbaegis | So just do that, reboot, iptables —flush, deploy? | 19:45 |
| sdake_ | remind me - your aio atm? | 19:45 |
| sdake_ | if multinode - need to specify an inventory file | 19:46 |
| kbaegis | I don't know what aio stands for :) | 19:46 |
| kbaegis | Oh, all-in-one | 19:46 |
| sdake_ | aio = all in one | 19:46 |
| kbaegis | Yes. That's step 1 | 19:46 |
| kbaegis | Then I'm going to spin the controllers up on 2 additional nodes (primarily into swap) | 19:46 |
| sdake_ | if your aio then destroy will do the job | 19:46 |
| kbaegis | Since I only have 4gigs ram on those two | 19:46 |
| sdake_ | it may not be iptables --flush as well | 19:47 |
| sdake_ | i have a meeting in 10 mins to prep for, so i don't have time to read the manual page | 19:47 |
| kbaegis | okay | 19:47 |
| sdake_ | ill be back in an hourish | 19:47 |
| kbaegis | good prepping, ty for all the help | 19:48 |
| sdake_ | ya | 19:48 |
| sdake_ | if you could write down what you did to make it all work that might come in handy | 19:48 |
| kbaegis | I will. | 19:48 |
| sdake_ | for others that follow in your path | 19:48 |
| sdake_ | etherpad is good | 19:48 |
| sdake_ | or an email to the mailing list | 19:49 |
| kbaegis | There's a lot of dependencies for those compiling everything from source. Assumptions you get to make w/ distros :) | 19:49 |
| kbaegis | I think this will help the Kolla community as well. When Ubuntu decides to change something in the kernel, for instance. | 19:49 |
| sdake_ | quetion, did the buildl keepalived binary work for you? | 19:49 |
| kbaegis | I have it disabled at present | 19:49 |
| sdake_ | i see | 19:49 |
| sdake_ | so two things to solve not 1 :) | 19:49 |
| kbaegis | I could workaround earlier (centos-bin) | 19:49 |
| sdake_ | you will need keepalived for step 2 | 19:49 |
| kbaegis | by manually assigning the vip | 19:49 |
| kbaegis | But that's a hack against your checking scripts | 19:50 |
| kbaegis | It was iptables —flush | 20:00 |
| kbaegis | Redeploying, fingers crossed :) | 20:01 |
| *** bmace has quit IRC | 20:11 | |
| *** bmace has joined #openstack-kolla | 20:11 | |
| *** coolsvap has quit IRC | 20:11 | |
| Pavo | anyone know what this error means? https://paste.pound-python.org/show/tXwye4n3DaYxKl95Rv20/ | 20:12 |
| kbaegis | Pavo I'm guessing it's a pip problem. Not able to find files | 20:13 |
| kbaegis | Did you do pip2 install -U git/ | 20:14 |
| kbaegis | and pip2 install -U ansible | 20:14 |
| Pavo | no, the docs only say pip install -U docker-py | 20:14 |
| kbaegis | It's in there | 20:14 |
| Pavo | installed ansible using yum | 20:14 |
| *** rhallisey_ has joined #openstack-kolla | 20:15 | |
| kbaegis | Could work :) | 20:15 |
| kbaegis | idk | 20:15 |
| *** ayoung has joined #openstack-kolla | 20:18 | |
| *** rhallisey_ has quit IRC | 20:20 | |
| kbaegis | Well, I have an ovsdb-server.log now at least | 20:20 |
| kbaegis | Evidently it requires ovs as a module | 20:22 |
| kbaegis | Breaks on compiling it in. You guys may want to fix that, idk | 20:23 |
| kbaegis | Easy enough for me to change; I don't personally care for managing kernel modules | 20:24 |
| kbaegis | More concerningly, this may actually tie the host and guest kernel versions together by necessity. idk | 20:26 |
| Pavo | this error doesn't make sense | 20:30 |
| *** egonzalez90 has quit IRC | 20:32 | |
| sdake_ | kbaegis - that is the openvswitch project's domain | 20:35 |
| sdake_ | kbaegis we have no input into their development | 20:35 |
| sdake_ | they are pretty much not directly part of openstack | 20:35 |
| sdake_ | i mean i could dsend a message to them | 20:35 |
| sdake_ | but i'm not sure it would do much :) | 20:35 |
| sdake_ | kbaegis we load kernel modules from the host - iirc | 20:36 |
| Pavo | sdake mind taking a look at my error | 20:37 |
| Pavo | https://paste.pound-python.org/show/tXwye4n3DaYxKl95Rv20/ | 20:37 |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla: Start using orchestration_engine variable https://review.openstack.org/356538 | 20:38 |
| *** bootsha has joined #openstack-kolla | 20:41 | |
| openstackgerrit | Merged openstack/kolla-kubernetes: Neutron bootstrap to create service/project/role https://review.openstack.org/353740 | 20:58 |
| kbaegis | recompiled;rebooted;checking to see if ovs will start without redeploy now | 20:58 |
| *** pbourke has quit IRC | 21:01 | |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes: Adding NOTE for Kubernetes Secret Generator https://review.openstack.org/356719 | 21:03 |
| *** pbourke has joined #openstack-kolla | 21:03 | |
| *** rhallisey has quit IRC | 21:04 | |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes: Adding NOTE for Kubernetes Secret Generator https://review.openstack.org/356719 | 21:06 |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes: Adding NOTE for Kubernetes Secret Generator https://review.openstack.org/356719 | 21:08 |
| *** bootsha has quit IRC | 21:10 | |
| *** jtriley_ has joined #openstack-kolla | 21:12 | |
| *** jtriley has quit IRC | 21:15 | |
| *** michauds has quit IRC | 21:15 | |
| *** ad_rien_ has joined #openstack-kolla | 21:16 | |
| *** jtriley_ has quit IRC | 21:17 | |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes: Modifying Keystone bootstrap to use secrets https://review.openstack.org/356098 | 21:19 |
| *** kbaegis has quit IRC | 21:24 | |
| sdake_ | pavo | 21:25 |
| sdake_ | i'm beat | 21:25 |
| sdake_ | been up since 4am | 21:25 |
| sdake_ | need power nap - can it wait | 21:25 |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes: Adding NOTE for Kubernetes Secret Generator https://review.openstack.org/356719 | 21:26 |
| *** gfidente has joined #openstack-kolla | 21:27 | |
| Pavo | sure | 21:28 |
| kfox1111 | weird... they make symlinks for configmaps... | 21:29 |
| *** dwalsh has quit IRC | 21:31 | |
| *** kbaegis has joined #openstack-kolla | 21:32 | |
| *** kbaegis1 has joined #openstack-kolla | 21:36 | |
| *** kbaegis has quit IRC | 21:37 | |
| *** ad_rien_ has quit IRC | 21:42 | |
| *** huikang has quit IRC | 21:49 | |
| *** huikang has joined #openstack-kolla | 21:50 | |
| *** huikang has quit IRC | 21:55 | |
| *** sdake_ has quit IRC | 22:06 | |
| openstackgerrit | Kevin Fox proposed openstack/kolla-kubernetes: Deployment, Readiness, Safe Shutdown, & Scaling for glance-api https://review.openstack.org/354895 | 22:14 |
| *** jeh has quit IRC | 22:14 | |
| *** inc0 has quit IRC | 22:14 | |
| *** ayoung has quit IRC | 22:14 | |
| kfox1111 | sbezverk: I think that one's ready for review now. | 22:15 |
| *** huikang has joined #openstack-kolla | 22:16 | |
| *** tummy has joined #openstack-kolla | 22:16 | |
| kbaegis1 | sdake_ do you know where the openvswitch module is expected to be found? | 22:16 |
| *** jeh has joined #openstack-kolla | 22:17 | |
| *** jeh has quit IRC | 22:17 | |
| *** jeh has joined #openstack-kolla | 22:17 | |
| kfox1111 | sbezverk: with that review, and a ceph backed glance, you should be able to do a no downtime minor version rolling upgrade. | 22:20 |
| *** JoseMello has quit IRC | 22:20 | |
| *** ad_rien_ has joined #openstack-kolla | 22:20 | |
| *** sdake has joined #openstack-kolla | 22:21 | |
| *** severion has joined #openstack-kolla | 22:22 | |
| *** v1k0d3n has quit IRC | 22:25 | |
| *** tummy has quit IRC | 22:28 | |
| *** huikang has quit IRC | 22:29 | |
| *** huikang has joined #openstack-kolla | 22:29 | |
| *** ad_rien_ has quit IRC | 22:31 | |
| *** huikang has quit IRC | 22:34 | |
| *** sdake has quit IRC | 22:38 | |
| *** egonzalez has joined #openstack-kolla | 22:48 | |
| *** inc0 has joined #openstack-kolla | 22:49 | |
| kbaegis1 | lol | 22:51 |
| kbaegis1 | "modprobe: ERROR: could not insert 'openvswitch': Exec format error" | 22:51 |
| kbaegis1 | Great | 22:51 |
| kbaegis1 | Falling back to linuxbridging, trying the iptables —flush option | 22:52 |
| *** vhosakot has quit IRC | 22:53 | |
| *** HyperJohnGraham has joined #openstack-kolla | 23:12 | |
| *** ayoung has joined #openstack-kolla | 23:19 | |
| *** mark-casey has quit IRC | 23:20 | |
| kbaegis1 | Sadly, no. iptables —flush didn't work :'( | 23:27 |
| *** kbaegis1 has quit IRC | 23:27 | |
| *** kbaegis has joined #openstack-kolla | 23:27 | |
| kbaegis | Oh well | 23:28 |
| kbaegis | is there a way to manually insert those rules? | 23:33 |
| kbaegis | To see rejections/conflicts? | 23:33 |
| kbaegis | Alright, I'm going to cave and install centos | 23:34 |
| kbaegis | :/ | 23:34 |
| kbaegis | Not happy about it, but it'll to in a pinch | 23:35 |
| kbaegis | :) | 23:35 |
| kbaegis | And I'd rather have a functioning lab than one done right :) | 23:35 |
| *** egonzalez has quit IRC | 23:38 | |
| kbaegis | Wish I knew more than that iptables.manager generically fails on COMMIT | 23:45 |
| kbaegis | I feel like that's solvable | 23:46 |
| *** zhurong has joined #openstack-kolla | 23:51 | |
| *** diogogmt has quit IRC | 23:56 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!