Wednesday, 2016-07-20

*** bjolo_ has joined #openstack-kolla00:06
*** bjolo has quit IRC00:06
*** zhurong has quit IRC00:07
*** huikang has joined #openstack-kolla00:09
*** mbound has joined #openstack-kolla00:11
*** huikang has quit IRC00:14
*** mbound has quit IRC00:16
*** banix has quit IRC00:16
*** banix has joined #openstack-kolla00:20
*** jtriley has joined #openstack-kolla00:21
*** sdake_ has quit IRC00:28
*** jtriley has quit IRC00:37
*** banix has quit IRC00:44
*** dwalsh has quit IRC00:46
*** dwalsh_ has quit IRC00:46
*** Jeffrey4l has joined #openstack-kolla00:51
*** Jeffrey4l has quit IRC00:51
*** huikang has joined #openstack-kolla01:01
*** ssurana has quit IRC01:09
*** bjolo_ has quit IRC01:10
*** zhurong has joined #openstack-kolla01:13
*** britthou_ has joined #openstack-kolla01:25
*** huikang has quit IRC01:26
*** huikang has joined #openstack-kolla01:27
*** britthouser has quit IRC01:28
*** huikang has quit IRC01:31
*** Liuqing has joined #openstack-kolla01:34
*** haplo37_ has joined #openstack-kolla01:34
*** bjolo has joined #openstack-kolla01:36
*** harlowja has joined #openstack-kolla01:37
openstackgerritJeffrey Zhang proposed openstack/kolla: implement Ansible Tempest role  https://review.openstack.org/32152201:43
*** jtriley has joined #openstack-kolla02:03
*** bjolo_ has joined #openstack-kolla02:06
*** bjolo has quit IRC02:06
*** huikang has joined #openstack-kolla02:30
*** huikang has quit IRC02:32
*** huikang has joined #openstack-kolla02:33
*** unicell has quit IRC02:38
*** huikang has quit IRC02:38
*** banix has joined #openstack-kolla02:47
*** yuanying has quit IRC02:51
openstackgerritMerged openstack/kolla: Add named volume for nova-libvirt  https://review.openstack.org/34413402:53
openstackgerritMerged openstack/kolla: Add named volume for nova-libvirt  https://review.openstack.org/34413502:54
openstackgerritMerged openstack/kolla: Fix the failure to create instance snapshot  https://review.openstack.org/34430902:54
*** ssurana has joined #openstack-kolla02:59
*** zhurong has quit IRC02:59
*** Liuqing has quit IRC02:59
*** Liuqing has joined #openstack-kolla03:00
*** zhurong has joined #openstack-kolla03:00
*** bjolof__ has joined #openstack-kolla03:06
*** bjolo_ has quit IRC03:06
*** huikang has joined #openstack-kolla03:07
*** jeh has quit IRC03:13
*** jeh has joined #openstack-kolla03:13
*** jeh has quit IRC03:13
*** jeh has joined #openstack-kolla03:13
*** huikang has quit IRC03:24
*** jtriley has quit IRC03:30
*** yuanying has joined #openstack-kolla03:47
*** david-lyle has joined #openstack-kolla03:49
*** banix has quit IRC03:55
*** aernhart has quit IRC03:56
*** banix has joined #openstack-kolla04:03
*** banix has quit IRC04:09
*** bjolof__ has quit IRC04:09
coolsvapssurana: hi are you still getting the issue reported in https://bugs.launchpad.net/kolla/+bug/149178204:10
openstackLaunchpad bug 1491782 in kolla "Fail to create haproxy mysql user" [Critical,Triaged] - Assigned to Sidharth Surana (ssurana)04:10
*** david-lyle_ has joined #openstack-kolla04:19
*** david-lyle has quit IRC04:19
*** bdaca has joined #openstack-kolla04:19
*** david-lyle_ has quit IRC04:26
*** ssurana has quit IRC04:26
*** daneyon_ has quit IRC04:31
*** daneyon has joined #openstack-kolla04:32
*** bjolo has joined #openstack-kolla04:36
*** salv-orlando has joined #openstack-kolla04:52
*** unicell has joined #openstack-kolla05:05
*** daneyon has quit IRC05:10
*** bjolo has quit IRC05:10
*** unicell1 has joined #openstack-kolla05:12
*** unicell has quit IRC05:12
*** dmueller-evoila has joined #openstack-kolla05:26
*** daneyon has joined #openstack-kolla05:44
*** daneyon_ has joined #openstack-kolla05:48
openstackgerritBartłomiej Daca proposed openstack/kolla: /run/nologin file removal  https://review.openstack.org/34457405:50
*** haplo37_ has quit IRC05:52
*** daneyon_ has quit IRC05:52
*** mewald has joined #openstack-kolla06:01
mewaldmorning06:01
bdacahello06:05
*** daneyon has quit IRC06:07
*** nradojevic has joined #openstack-kolla06:14
*** bjolo has joined #openstack-kolla06:17
bjolomorning06:17
*** bjolo_ has joined #openstack-kolla06:19
*** bjolo_ has quit IRC06:19
*** bjolo_ has joined #openstack-kolla06:19
*** bjolo has quit IRC06:23
bjolo_morning06:30
*** bjolo_ is now known as bjolo06:30
mewalddouble good morning bjolo :D06:32
bjolohigh five!06:32
bjolo:)06:33
mewald:D06:33
mewaldguys this shit drives me nuts: https://gist.github.com/mewald1/50678e1433c583b86e47bfa1f72021cc Can someone show me how to get the escaping stuff right in Ansible?06:33
mewaldcurrent error is https://gist.github.com/mewald1/17e43f2b9ece35f36551521158d65cc706:33
bjolotired today. had friends over for dinner so it got late, and then the kids kept me up most of the night06:33
bjolomewald, very helpful ansible error msg06:34
bjolonot06:34
bjoloi go on vacation friday and will be out for a few weeks.06:35
mewaldnice for you :D where to?06:35
bjoloso i really have to stop playing around and do my job! :) need to get a kolla 2 up and runing so users can play with it while im gone06:36
bjolowe have a summer house in karlskrona, sweden.06:36
bjoloon one of the islands06:37
bjologonna spend some time there and just unplug and relax06:37
bjoloafter that we are off to seattle for a wedding06:37
*** mewald1 has joined #openstack-kolla06:37
bjoloso i wont be back fulltime until end of august06:38
bjolomewald, are you good with SSL?06:39
*** JianqingJiang has joined #openstack-kolla06:39
mewald1I dont speak it natively :D What's the problem?06:40
bjoloi have a certificate (symantec) and kolla deploys it well and horizon works06:40
bjolobut not the openstack client06:40
*** mewald has quit IRC06:41
bjolounknown ssl error06:41
mewald1puhh unknown errors always suck :D Is the cert signed by an official CA who's Cert is installed on your local box?06:41
*** BernhardW has joined #openstack-kolla06:43
bjoloopenssl x509 -in mycert.crt -text -noout | grep -i issuer06:45
bjolo        Issuer: C=US, O=Symantec Corporation, OU=Symantec Trust Network, CN=Symantec Class 3 Secure Server CA - G406:46
bjolo                CA Issuers - URI:http://ss.symcb.com/ss.crt06:46
bjoloi downloaded the intermediate crt from symantec and put my crt and intermediate crt in the pem file06:46
bjoloit works for both my private docker registry and for horizon, but not for openstack client.06:47
mewald1I've had similar issues a few weeks ago … the problem was the order in which I concatenated the different cert files and keys06:48
mewald1maybe turn the order around or something06:49
bjoloi have also downloaded the intermediate crt to my client and used openstatck --os-cacert06:49
bjolook currently i have mycrt, mykey, interdediate crt06:49
BernhardWhi, I guess you'll need the root ca cert also, not only the intermediate06:49
BernhardWyou should be able to download that where you found the intermediate06:50
mewald1yeah you need the entire chain06:50
bjolook, will try. but as i said it works for horizon and my private docker reg06:51
BernhardWI guess you are accessing horizon from a workstation browser, which has the root cert installed by the os vendor06:52
BernhardWas symantec is a trusted ssl authority06:53
mewald1I have seen totally different behavior of browsers in such situations.06:56
bjoloso i should include the crt from  CA Issuers - URI:http://ss.symcb.com/ss.crt06:57
bjoloin my pem file06:57
mewald1Guys, does Kolla not create a "user" role in keystone? I am getting error messages from Horizon about non-existent default user of "_member_"06:58
mewald1can anyone confirm?06:58
BernhardWthis is a intermediate certificate also according to OSX06:58
bjolook just gonna reset my nodes (broken kolla 3) and deploy kolla 206:58
bjolomewald1, all my kolla 2 deploys have had the same issue06:59
BernhardWmewald1: I'll take alook at my setup06:59
bjolobefore i can create any projects or users, i need to create the _member_ role06:59
mewald1Ok, so it has to be a bug then … shit. Looks like people never actually try to use kolla deployed environments. Browser to VIP, see login screen, done, git push07:00
bjoloi to new to kolla, so i thought that it was intended to be like this :)07:01
*** belmoreira has joined #openstack-kolla07:01
bjoloBernhardW, so how do i find the root crt for  CA Issuers - URI:http://ss.symcb.com/ss.crt07:02
bjolowhat type of build and gating is kolla using? what infrastructure?07:02
BernhardWwhich OS are you using on the workstation, you should be able to export from there07:02
bjolowindows 7 and chrome is what ive been using07:03
bjolobut i run the openstack client on a centos 7 machine (also my deployer node)07:03
BernhardWit seems you can download them here, but don't ask me which one :( https://knowledge.symantec.com/support/code-signing-support/index?page=content&id=SO478507:04
BernhardWWindows also has a keystore somewhere, it should be possible to export the root ca cert from there, but it should also be downloadable from the symantec website07:05
bdacamewald1: still have a problem with ansible?07:06
bdacamaybe try this:07:06
bdacalocal_action: shell cat {{ cephx_keyring_files.files[0].path }} | grep -E 'key\s*=' | awk '{ print $3 }'07:06
BernhardWmewald1: no user role here either, I never noticed as I'm using admin for my evaluation purposes07:06
BernhardWguys, how do I change the hypervisor type? I edited /etc/kolla/nova-compute/nova.conf but every time I do a reconfigure it's overwritten by the default?07:10
mewald1BernhardW: Use the ini merge mechanism07:10
BernhardWshouldn't that get don automatically?07:11
BernhardWapart from that the documentation seems outdated07:11
bdacathe problem is the path I think, /etc/kolla/config/nova.conf will be ok07:12
BernhardWdocumentation states: "For this, change the virt_type option in the [libvirt] section of nova-compute.conf file inside the /etc/kolla/config/ directory." where I do not have a config directory but a directory for each service07:12
BernhardWor are these the templates and I need to create a new file for the merge algorithm to factor in?07:13
bdacajust put the nova-compute directory into the /etc/kolla/config07:13
mewald1https://github.com/openstack/kolla/blob/master/ansible/roles/nova/tasks/config.yml#L5607:13
bdacas/nova-compute/nova/07:14
*** Serlex has joined #openstack-kolla07:14
mewald1The file should be /etc/kolla/config/nova/nova-compute.conf07:14
BernhardWok, from the source it's clear, thanks07:15
BernhardWstill, the documentation is a bit misleading I think07:16
*** unicell has joined #openstack-kolla07:16
*** athomas has joined #openstack-kolla07:16
*** unicell1 has quit IRC07:16
mewald1BernhardW: full ack07:18
*** bootsha has joined #openstack-kolla07:19
BernhardWand how do I remove a config line during reconfig, it added virt_type=kvm now but still shows qemu, probably as "connection_uri = "qemu+tcp://192.168.50.11/system" is still in there?07:19
mewald1BernhardW: I think you cannot remove it. What you can do is set it empty though07:20
mewald1but the connection_uri is correct for kvm07:22
BernhardWI just read libvirt doc and came to the same conclusion07:24
BernhardWso then the question remains as horizon shows QEMU as hypervisor07:24
BernhardWeven though it's now kvm07:25
mewald1BernhardW: Have you ever seen it otherwise? My environment shows QEMU, too07:25
mewald1an in fact KVM is just accellerated qemu07:25
BernhardWsure, in my rdo-based bare-metal install it's showing KVM07:25
mewald1oh ok07:26
mewald1then I have a problem, too :D07:26
BernhardWprobably as this uses the local socket and not the tcp transport07:26
*** b_bezak has joined #openstack-kolla07:26
*** shardy has joined #openstack-kolla07:27
BernhardWprobably it's only a glitch in displaying it correctly07:27
mewald1BernhardW: https://gist.github.com/mewald1/f80fcdc60f560391a1fa9da003d55ed407:28
mewald1Is this using KVM now?07:28
BernhardWhmm, let me check, I read something as few min ago but do not remember07:30
*** britthouser has joined #openstack-kolla07:33
BernhardWso, from what I see it's using kvm07:34
BernhardWlibvirt states: "KVM hypervisor: The driver will probe /usr/bin for the presence of qemu-kvm and /dev/kvm device node. If both are found, then KVM fullyvirtualized, hardware accelerated guests will be available."07:35
mewald1yeah I think accel=kvm says it07:35
*** britthou_ has quit IRC07:36
*** bdaca_ has joined #openstack-kolla07:36
BernhardWquestion is: would the command fail if kvm is not available or would it fall back to qemu... but for now I'd say its fine07:36
mewald1and horizon still says qemu07:36
mewald1hmm good question ^^07:36
BernhardWseems it would complain: determine if kvm accelleration is used07:37
BernhardWsorry07:37
BernhardWhttp://www.linux-kvm.org/page/FAQ#How_can_I_check_that_I.27m_not_falling_back_to_QEMU_with_no_hardware_acceleration.3F07:37
*** bdaca__ has joined #openstack-kolla07:50
*** bdaca_ has quit IRC07:54
mewald1Can someone have a look ha https://gist.github.com/mewald1/06fcbad28572ac5a6f533354bca08edf I got a little close with the ansible command module here but still got an error as shown. If we get this to run quickly, I will submit a patch for external ceph for Nova today and we can close the bluepring.07:57
mewald1-g +t07:57
bdaca__mewald1: I wrote it to you several minutes ago07:57
bdacamaybe try this:07:58
bdacalocal_action: shell cat {{ cephx_keyring_files.files[0].path }} | grep -E 'key\s*=' | awk '{ print $3 }'07:58
mewald1bdaca__: oh I totally missed this07:58
mewald1will try now07:58
bdaca:)07:59
Kdecherfhey there08:02
bdaca__hello08:02
mewald1hi08:02
bjolojust some trivial knowledge for the kolla project :)08:05
bjolowe have the word kolla in swedish as well08:05
bjoloit means something along the line "check this out!" "look!"08:05
bjoloso it gets kinda funny when i tell another swede to try check kolla out08:06
bjolokolla in kolla08:06
Kdecherfkolla-ception08:08
bdaca__yo dawg!08:08
BernhardWbjolo: you also have a bank with that some or something I think, found references when searching for "kolla"08:09
bjolohmm let me google that08:10
BernhardWmewald1: /usr/libexec/qemu-kvm -name instance-00000001 -S -machine pc-i440fx-rhel7.2.0,accel=kvm,usb=off... I'm not sure why it says qemu-system-x86_64 in your snippet?08:10
*** bdaca_ has joined #openstack-kolla08:10
*** bdaca__ has quit IRC08:14
Kdecherfhm, interesting08:19
KdecherfI stopped my cluster yesterday, and now the container mariadb restarts itself continuously in recovery mode08:20
*** derekjhyang has quit IRC08:22
bdaca_Kdecherf: have you tried kolla-ansible mariadb_recovery ?08:22
Kdecherfbdaca_: i'm checking08:22
*** aernhart has joined #openstack-kolla08:24
*** gfidente has joined #openstack-kolla08:28
*** salv-orlando has quit IRC08:30
*** daneyon has joined #openstack-kolla08:30
*** aernhart_ has joined #openstack-kolla08:30
*** bdaca__ has joined #openstack-kolla08:31
Kdecherfbdaca_: it partially worked08:34
Kdecherfthe main node failed to restart and ansible timed out08:34
*** aernhart has quit IRC08:34
*** daneyon has quit IRC08:35
*** bdaca_ has quit IRC08:35
bjoloquestion about default options in multinode08:43
bjolorados GW and swift proxy are default to run on the controler nodes08:44
bjoloto me that sounds wrong. shouldnt they be on the network nodes?08:45
bjoloit is a public facing service just like horizon08:45
bjolohmm let me retract on what i just said08:46
*** unicell1 has joined #openstack-kolla08:50
*** bdaca_ has joined #openstack-kolla08:51
*** b_bezak has quit IRC08:51
*** unicell has quit IRC08:52
BernhardWbjolo: in my opinion they should be on a "storage" node, not on a network node, "controller" would be the fallback as it is for network too08:54
*** mgoddard_ has joined #openstack-kolla08:55
*** bdaca__ has quit IRC08:55
*** BernhardW is now known as bwe_08:57
*** mgoddard has quit IRC08:58
*** mgoddard has joined #openstack-kolla09:03
*** mgoddard_ has quit IRC09:03
bjolobwe_, can agree on that as well depending on your topology and how many nodes you have etc. In my setyp i have a small l2 segment for public facing services. Only network nodes have interfaces on it09:06
bjolomy point is that i dont think anyone wants to bugg down the controller nodes with massive traffic to/from object store09:07
bjolobut again, this is just the default location and configurable by operator09:08
Kdecherfwell, one of my mariadb node is completing down with this error: 160720 10:55:28 [ERROR] Found 3 prepared transactions! It means that mysqld was not shut down properly last time and critical recovery information (last binlog or tc.log file) was manually deleted after a crash. You have to start mysqld with --tc-heuristic-recover switch to commit or rollback pending transactions.09:08
Kdecherfhow can i handle it with kolla? (or what is the complete command to handle it manually?)09:08
bjolodefault option in multinode example file09:08
*** bdaca__ has joined #openstack-kolla09:11
*** bdaca_ has quit IRC09:14
*** mgoddard_ has joined #openstack-kolla09:15
*** mewald1 has quit IRC09:16
*** mewald has joined #openstack-kolla09:16
*** salv-orlando has joined #openstack-kolla09:17
*** mewald has quit IRC09:17
*** mgoddard has quit IRC09:18
*** salv-orl_ has joined #openstack-kolla09:19
*** sdake has joined #openstack-kolla09:21
*** salv-orlando has quit IRC09:22
*** sdake has quit IRC09:23
bwe_bjolo: you are free to change the assignment of nodes to "classes" in multinode config file09:23
bjoloKdecherf, is mariadb runing on all the nodes? i.e. can you find the most advanced node?09:23
bwe_and services to classes of course too09:23
Kdecherfbjolo: mariadb is running and healthy on the two other nodes09:24
bjoloKdecherf, looked at this? http://galeracluster.com/documentation-webpages/quorumreset.html09:24
bjolobwe_, would love to but i go on vacation friday and have pile of work to get out the door09:25
bwe_https://github.com/openstack/kolla/blob/master/doc/multinode.rst09:25
bwe_it's not that complicated09:25
bjoloi should not even be here on IRC to be honest! :)09:25
bjolook damn you, i will take a look :)09:25
bwe_"Edit the inventory file" is the section you are looking for09:26
*** mewald has joined #openstack-kolla09:26
nradojevicKdecherf, did you find a way to solve your mysql problem?09:31
*** bdaca_ has joined #openstack-kolla09:31
bwe_I've seen containers in the docker registry for trove but cannot find any reference to it in the kolla sources, does anyone know in what state that development is?09:31
bwe_ah, wrong, there is something, did look at the wrong place09:32
bwe_building a container now... ;-) sorry09:32
*** hanchao has quit IRC09:34
*** bdaca__ has quit IRC09:34
Mech422morning09:36
*** salv-orl_ has quit IRC09:37
Kdecherfnradojevic: not yet09:38
nradojevicOk, perhaps I can help you09:38
nradojevicI had the same error yesterday09:38
*** athomas has quit IRC09:39
Kdecherfyeah?09:40
nradojevictry to enter this command: "docker restart mariadb && docker exec -it -u root mariadb mysqld_safe --tc-heuristic-recover=ROLLBACK" on the host where your database is located09:40
nradojevicafterwards: "docker restart mariadb && docker exec -it -u root mariadb mysqld_safe --tc-heuristic-recover=COMMIT"09:41
*** bdaca_ has quit IRC09:42
Kdecherfhm, not very effective09:43
nradojevicok..worked for me09:51
*** aernhart__ has joined #openstack-kolla09:52
Kdecherfnradojevic: i managed to restart the missing node but i have the feeling that it forked from the other nodes09:52
Mech422Kdecherf: from the mysql container, you can login to mysql shell and 'show..like ws_rep'09:54
Mech422Kdecherf: that should show you the cluster size09:54
*** aernhart_ has quit IRC09:55
mewaldbwe_: Looks like somebody added the container but didnt contribute the ansible part09:56
KdecherfMech422: well, it splitted09:57
Kdecherfall nodes have the same cluster uuid09:57
Kdecherfbut they don't have the same cluster size09:57
Mech422Kdecherf: that sounds more like comms error then a 'real' split09:57
Mech422Kdecherf: are you sure the 'bad' node has finished its state transfer ?09:57
KdecherfMech422: the bad node does not seem to talk to other nodes09:58
*** b_bezak has joined #openstack-kolla09:58
Mech422Kdecherf: yeah - that sounds like a comms error09:59
Mech422Kdecherf: does it list the other nodes in the mysql config ?09:59
KdecherfMech422: how can i see that?09:59
Mech422Kdecherf: err..off the top of my head? I'd docker exec bash in the 'bad' mysql node and look in /etc/mysql?10:00
*** mgoddard_ has quit IRC10:00
Mech422Kdecherf: not sure if the containers keep configs in the 'normal' locations though10:01
*** mgoddard has joined #openstack-kolla10:01
KdecherfMech422: i checked the configuration from /etc/kolla/mariadb/galera.cnf and it looks good10:01
*** zhurong has quit IRC10:02
Mech422Kdecherf: check you can ping the other nodes, and look in heka container for logs10:03
*** Liuqing has quit IRC10:03
Mech422should be under /var/log/kolla or something in the heka container10:04
KdecherfMech422: checked the logs, there is no indication in the log that mariadb/galera talks to other nodes10:04
Mech422Kdecherf: Hmm - it should show a bunch of stuff about the cluster...10:04
Mech422Kdecherf: what does it say ?10:04
*** jmccarthy has joined #openstack-kolla10:04
mewaldBooting an instance I see this: https://gist.github.com/mewald1/55abfd5898ccbe84ed1c1497e8285e9c Anyone know what it is and how to solve?10:05
KdecherfMech422: on healthy nodes, galera ejected the failed node from the map10:06
KdecherfMech422: and the failed node decided to start a cluster alone, with the same id10:06
jmccarthyHiya ! Any suggestions about how to narrow down murano/keystone issue ? http://paste.openstack.org/show/538788/10:06
Mech422Kdecherf: well - I'm a lazy fuck...I'd probably just blow away the mysql data files and restart the container..10:07
Mech422Kdecherf: if that doesn't cause a full re-sync, you have bigger issues :-)10:07
Kdecherfmy main issue right now is that the galera's documentation is down10:07
*** aernhart__ has quit IRC10:07
*** mgoddard has quit IRC10:08
Mech422Kdecherf: the percona site ? or mariadb ?10:08
*** mgoddard has joined #openstack-kolla10:08
Mech422Kdecherf: I usually use the docs from percona10:08
KdecherfMech422: galeracluster.com10:09
Mech422oh - never saw that site10:09
Mech422try percona - galera is the 'xtradb cluster' stuff10:09
Kdecherfoh, thanks10:10
Mech422percona actually wrote galera, so the docs are pretty good10:10
Kdecherfit reminds me, the task "Starting first mariadb container" from the task recover_cluster actually starts the container with --wsrep-new-cluster10:11
*** dwalsh has joined #openstack-kolla10:12
*** dwalsh_ has joined #openstack-kolla10:12
bwe_mewald: it seems everything is there but I have a strange network problem to solve first, kolla-build is unable to download percona-release10:12
*** sdake has joined #openstack-kolla10:14
openstackgerritMathias proposed openstack/kolla: External Ceph - Implementation Nova  https://review.openstack.org/34470510:17
*** sdake_ has joined #openstack-kolla10:17
Mech422Kdecherf: yeah - I don't think you should start your 'bad' node like that though10:18
Mech422Kdecherf: you want to join the cluster, not start a new one10:19
*** JianqingJiang has quit IRC10:19
*** sdake has quit IRC10:19
Mech422bwe_: I'm running around trying to find an old ubuntu package for gnutls - it got updated on my machine and now ldap doesn't work10:20
Mech422bwe_: gnutls apperenty deprecated md5sum'd certs, and I don't control the ldap server :-/10:20
*** sdake_ has quit IRC10:22
bwe_I'm not really the ubuntu guy but a quick search pointed that out: http://askubuntu.com/questions/138284/how-to-downgrade-a-package-via-apt-get10:24
Mech422bwe_: yeah - I know HOW to install it, I just don't know WHERE to get it from..10:25
bwe_apt-get fetches it from the mirror10:25
*** sdake has joined #openstack-kolla10:25
Mech422bwe_: ubuntu seems to have wiped the version I need from archives.ubuntu.com10:25
Mech422bwe_: yeah - problem is, the .deb isn't available to apt anymore10:25
Mech422bwe_: I need to manually download it from somewhere and dpkg -i10:25
bwe_probably in the install ISOs?10:26
sdakemorning10:26
bwe_hi10:26
bwe_Mech422: probably breaks dependencies, this will get you to hell eventually10:27
Mech422bwe_: ohh - good idea... I'd have to check the full dvd though :-/  the NetInst images pull from repo's10:27
Mech422sdake: Morning10:27
sdakeup late or early Mech422  :)10:27
Mech422bwe_: yeah - I'm NOT a happy camper atm :-/10:27
sdakenothing like deprecations without warnings10:27
sdakeatleast openstack got that part right10:27
Mech422bwe_: I was hoping to pull it from /var/cache/apt on one of our working boxes, but our provisioning stuff cleans that10:27
Kdecherf/usr/bin/mysqld_safe: line 183:   178 Aborted                 (core dumped)10:28
Kdecherfok, i guess it gets better10:28
openstackgerritMathias proposed openstack/kolla: External Ceph - Implementation Nova  https://review.openstack.org/34470510:28
* Kdecherf sets fire on the node10:28
Mech422sdake: yeah - I can't believe ubuntu doesn't have a 'historical' archive with all the releases somewhere10:28
openstackgerritMathias proposed openstack/kolla: External Ceph - Implementation Nova  https://review.openstack.org/34470510:29
sdakethey probably do - you just dont know how to find it :)10:29
sdakeon fedora its called koji10:29
bwe_Mech422: you could try debian, still will get you to hell eventually ;-)10:29
Mech422sdake: it appears to have been pulled from archive.ubuntu.com - not sure where else to try10:30
sdakeMech422 what your looking for is ubuntu's build server10:30
Mech422sdake: for old stuff ? wouldn't that be for new stuff ?10:30
sdakeit should be for all stuff10:30
*** mbound has joined #openstack-kolla10:30
sdakekoji keeps everything ever built that was successful10:30
Mech422sdake: oh nice10:31
sdakei dont know if ubuntu has an analog10:31
sdakebut i'd think they do10:31
Mech422bwe_: I've had really good luck with debian - 15 years or so now, and no major issues..10:31
sdakei am not a ubuntu jockey sorry can't help more ;)10:31
Mech422bwe_: ubuntu has been a bit less fortunate :-P10:31
Mech422sdake: Thanks - it gives me more stuff to google anyway...maybe something will turn up10:32
bwe_mee too, you know what they say: "Ubuntu is an ancient african word, meaning: I can't install debian" ;-)10:32
Mech422bwe_: LOL10:32
Mech422bwe_: I use deb for my 'servers', and ubuntu for my disposable nodes/development10:32
Mech422bwe_: shorewall firewall + debian ROCK btw :-)10:33
bwe_ok, it seems trove support is only half way there, image building is there, deployment not :(10:34
KdecherfMech422: well, finally the node is back online10:35
Mech422Kdecherf: oh cool... setting fire to it always works!10:35
Mech422Kdecherf: gotta show it who's boss, and you mean business! :-)10:36
KdecherfMech422: haha, indeed10:36
Kdecherfit depends on the day but a hammer can be more effective than fire, sometimes10:37
Mech422Kdecherf: yeah - it just depends how pis^H^H^Hserious you are ...10:37
*** sdake has quit IRC10:38
Mech422Kdecherf: hammer is for attitude adjustment, fire is for kill-it-with-fire-and-scatter-the-ashes10:38
Kdecherfit also depends on how hungry you are, servers can make good bbq10:38
Kdecherf(true story)10:38
Mech422Kdecherf: hehe10:40
Mech422Kdecherf: on of the guys I work with has an old HD he put a musket shot thru...I guess these drives had a 'somewhat' high failure rate and he finally couldn't take it10:41
*** sdake has joined #openstack-kolla10:42
jmccarthyOk, I think I found my issue is to do with keystone, any suggestions for this here ? http://paste.openstack.org/show/538796/10:48
Mech422jmccarthy: did you forget to create the murano service user ?10:50
jmccarthyMech422: Maybe ? How do I check ?10:52
bdacajmccarthy: from console or horizon?10:54
Mech422jmccarthy: umm - I'd have to look it up - nova list users or keystone list users or something10:54
bdacain horizon log in as admin and see Identity->Users10:54
bdacaor from the console openstack user list10:55
bdaca"openstack user list" <- this is the command ;)10:55
jmccarthyopenstack user list shows murano10:56
*** britthou_ has joined #openstack-kolla10:58
bdacamaybe try to obtain a token as a murano?10:58
*** zhubingbing has joined #openstack-kolla10:59
jmccarthybdaca: Not sure how to test that ..11:00
*** britthouser has quit IRC11:00
Kdecherfhm, well, it seems that the network on my deployment with linuxbridge does not work11:03
*** aernhart__ has joined #openstack-kolla11:03
bdacajmccarthy: create openrc file for murano user, source it and run "openstack token issue"11:04
bdacado you know how to create openrc file?11:04
jmccarthybdaca: Ok I've got one of those, which things to I change for murano user ? Let me see11:06
KdecherfI have a lot of zombie dnsmasq processes in neutron-dhcp-agent, and the master node does not have healthy dnsmasq at all11:07
*** zhurong has joined #openstack-kolla11:07
jmccarthybdaca: Just change OS_USERNAME to murano is it ?11:08
bdacano, if you have everything as the defaults for murano, it should be:11:08
bdacaproject/user_domain = default, project/tenant_name = service, username = murano and password = XXX, proper for the murano user of course11:10
*** zhurong has quit IRC11:11
*** britthou_ has quit IRC11:11
bdacayou can find password in /etc/kolla/passwords.yml11:11
bdacaas murano_keystone_password11:11
*** sdake has quit IRC11:12
*** daneyon has joined #openstack-kolla11:12
jmccarthybdaca: Apologies, I'm fried - can you post a sample openrc ?11:13
bdacaof course, just give me a sec11:13
*** zhurong has joined #openstack-kolla11:14
bjoloKdecherf, what version of kolla and containers are you running?11:14
bdacahttp://paste.openstack.org/show/J0ccPXksozSgaMuVOGjn/11:15
Kdecherfbjolo: 2.0.211:15
bjoloKdecherf, same here11:15
*** britthouser has joined #openstack-kolla11:16
bdacajust change the OS_PASSWORD and OS_AUTH_URL11:16
Kdecherfhm, very nice11:16
bjoloim doing a new deployment right now. 2.0.2 but with my own images (updated sources to stable-mitaka tarballs for all core services)11:16
bjoloi remember also seing defunct dnsmasq processes. will check again once the deployment is done11:17
Kdecherfi run docker restart neutron_dhcp_agent and all my controllers are dying, waiting for oom-killer to hit some memory eaters11:17
*** daneyon has quit IRC11:17
*** shardy has quit IRC11:18
jmccarthybdaca: Thanks ! checking11:18
*** britthou_ has joined #openstack-kolla11:19
*** britthouser has quit IRC11:20
openstackgerritPrithiv proposed openstack/kolla: Support Networking-SFC Container  https://review.openstack.org/34476211:20
Mech422Woot! Got my ldap back!  Found a repo someone had setup in the company11:21
mewaldKdecherf: had the same thing, I killed everything, reployed, worked11:21
jmccarthybdaca: Ok interesting ! When I do that, I am able to get a token:11:22
jmccarthy+------------+----------------------------------+11:22
jmccarthy| Field      | Value                            |11:22
jmccarthy+------------+----------------------------------+11:22
jmccarthy| expires    | 2016-07-20T12:19:51.045887Z      |11:22
jmccarthy| id         | 23accd441f664cff8ac49b261e4b2f65 |11:22
jmccarthy| project_id | c2b4d2efd1124eb3b0bd990e479fe096 |11:22
jmccarthy| user_id    | 9716b8b745304b60b3ddbf01df1f02b3 |11:22
jmccarthy+------------+----------------------------------+11:22
Kdecherfwell, not really a production-friendly way of fixing things11:22
jmccarthy(9716b8b745304b60b3ddbf01df1f02b3 is murano)11:22
mewaldKdecherf: nope :D I was never able to dig into it as the system becomes entirely unavailable11:24
Kdecherfhaha nice, my mariadb cluster died, again11:25
*** zhubingbing has quit IRC11:27
*** gfidente is now known as gfidente|lunch11:27
Mech422Kdecherf: you sure you don't have some networking issue?  packet loss or something...11:27
Mech422Kdecherf: seems odd it fails so often11:28
*** dwalsh has quit IRC11:30
*** dwalsh_ has quit IRC11:30
*** mewald has quit IRC11:31
*** mewald1 has joined #openstack-kolla11:31
bjolowhen building my own images, is it possible to customize the ceph images as well? i cant see any options for ceph kolla-build.conf11:35
bjolokolla 2 is using ceph hammer, would like to use jewel11:36
bjolorpm packages located here http://download.ceph.com/rpm-jewel/11:36
bdacaok, so looks like keystone is rather ok... can you describe me your scenario, jmccarthy? I can try to reproduce this in my env if you give me some time11:38
jmccarthybdaca: It's to do with when I try to deploy a murano environment - but I am suspicious now of my murano.conf may be off, I'll go and research that :)11:40
Mech422bjolo: hmm - looks like the ceph Dockerfile is hardcoded...11:42
bjolothats what i mean11:42
Mech422bjolo: you can do what I do, and before running the build, patch the dockerfile11:42
bdacajmccarthy: I will try it this evening. what kolla version do you use?11:43
*** britthou_ has quit IRC11:43
bjoloMech422, what do you do it? just adding the ceph repo prior to yum install?11:44
Mech422bjolo: I don't do it for ceph specifically, but yes - I imagine you have to add the repo to the host, and apt-get update, and check the package names in the Dockerfile11:45
bjolohttp://docs.ceph.com/docs/master/install/get-packages/11:45
openstackgerritWill Rouesnel proposed openstack/kolla: Volume specification for ironic-api was invalid (should be a list).  https://review.openstack.org/34477511:45
Mech422bjolo: not sure what the equivalent of 'apt-get update' is for RH systems11:46
bjoloyum update11:46
Mech422bjolo: dont forget to check that the 'ceph.com' package names are the same as the ones in the Dockerfile11:47
bjoloi think i skip this for now11:47
bjolojust need ceph to work for now so i can get the cloud up for testing out customizations (keystone domains with AD auth, multiple external networks, SSL certs)11:49
bjoloplan is to go prod with kolla 3 using external ceph (tnx to mewald1 :) )11:50
Mech422bjolo: I would HIGHLY recommend creating an ansible playbook of the customizatons you do along the way11:50
Mech422bjolo: I actually 'drive' kolla via a set of custom playbooks so I can apply patches, change configs, etc etc11:51
bjolocool11:51
bjolodo you have any playbooks to share to get me started on this approach?11:52
Mech422bjolo: err..not really - it's all pretty site specific11:53
*** cristicalin has joined #openstack-kolla11:55
mewald1bjolo: is everything working ok with the external ceph stuff?11:56
bjolohave not had time to really try it out... frankly i tried to do a basic multinode deploy of kolla 3 but it broke quite early on11:57
bjolosince im on vacation friday i dont have time to dig into it11:57
mewald1ok let me know if there are any problems11:57
bjolooh i will :)11:57
Mech422mewald1: just a heads up - there is an issue with the 'read gpt partition' table stuff in find_disks.py11:58
Mech422mewald1: thats not your patch, but related11:58
bjoloi be back end of august. will try it out then11:58
bjoloand do what i can to help out get kolla 3 out the door11:59
Mech422mewald1: for one thing - it hard codes the location of sgdisk to /usr/sbin but .deb systems put it in /sbin11:59
mewald1Mech422: but we're talking about the code that deploys ceph, right? I didnt code that11:59
Mech422mewald1: right - I said its not your patch :-P12:00
mewald1yeah read it afterwards :D12:00
*** bdaca has quit IRC12:00
mewald1can you file a bug report and propose a patch?12:00
Mech422mewald1: I wrote the patch :-P  but it got 'enhanced' along the way12:00
Mech422mewald1: and like any program, it got enhanced to the point where it doesn't work anymore :-P12:01
openstackgerritDennis Mueller proposed openstack/kolla: Added ceilometer decision in neutron.conf for event notifications  https://review.openstack.org/34478212:02
*** athomas has joined #openstack-kolla12:03
mewald1Mech422: Can you send me a link?12:05
Mech422mewald1:   https://review.openstack.org/32660912:06
Mech422mewald1: I'm trying to see whats up with it now12:07
Mech422mewald1: it should probably 'shell' out to sgisk so search path can locate the binary12:07
*** banix has joined #openstack-kolla12:07
Mech422mewald1: and I have a sneaking suspicion that the udev library needs to be higher then 188012:07
*** banix has quit IRC12:09
*** Liuqing has joined #openstack-kolla12:10
*** jmccarthy has left #openstack-kolla12:13
*** jmccarthy has quit IRC12:13
*** gfidente|lunch is now known as gfidente12:26
*** cristicalin has quit IRC12:41
mewald1Does anyone have any experience with influxdb clustering? I am working on deployment of collectd + influxdb + grafana and was wondering if the effort to cluster influxdb was worth it12:44
*** salv-orlando has joined #openstack-kolla12:44
Mech422mewald1: I was looking at using influx with promethus - how you liking it ?12:45
Mech422mewald1: from what I've read, clustering is a weak spot for influxdb12:46
mewald1what is promethus?12:46
mewald1found it12:46
Mech422mewald1: https://prometheus.io/12:46
mewald1:)12:47
openstackgerritDennis Mueller proposed openstack/kolla: Added neutron-metering-agent  https://review.openstack.org/34480812:47
Mech422mewald1: I'm debating prometheus + grafana vs the TICK stack with influx12:47
mewald1I have not arguments available to debate this :D All I can add is: people a probably more familiar with grafana12:49
Mech422mewald1: yeah - though you can use grafana with TICK too12:49
Kdecherfmewald1: the native clustering on influxdb is now a business option12:49
Mech422Kdecherf: supposedly there some half-asses HA stuff you can do with the OSS version12:50
Mech422Kdecherf: I think its something like DRDB + HAProxy type stuff12:50
mewald1DRBD … I hope not :D12:51
KdecherfMech422: there is indeed a kind-of HA feature on the new community edition which consist of proxifying data and replicate them on two separate influxdb nodes12:52
Kdecherfworse than drbd :D12:52
Mech422Kdecherf: ahh...12:53
Mech422Kdecherf: have you used infuxdb/TICK ?  I just want something relatively painless for my home lab monitoring..12:53
Kdecherf(ha mechanism is completely removed from community edition)12:53
Mech422Kdecherf: cassandra based stuff seems like overkill, and influxdb sounds easy to setup - but I keep hearing its not quite 'stable' yet..12:54
Kdecherfi deployed a TIG stack on my lab env12:54
Kdecherf(telegraf/influxdb without HA/grafana)12:54
Mech422oh - how do you like it ?12:54
Kdecherfit's really simple and works quite out of the box12:54
Mech422Kdecherf: that might be the deciding factor for me...ease of setup12:55
Mech422Kdecherf: how is telegraf compared to collectd/diamond?  It sounds pretty light weight ?12:55
*** rhallisey has joined #openstack-kolla12:56
mewald1one thing I wanna know is: how many plugins are there for telegraf? Will it be pain in the ass to monitor a ceph cluster or openstack metrics or is there something we can use?12:56
KdecherfMech422: my setup is really simple atm but it works and it gives me a good summary of performance of my nodes (10s collect, 7d retention), seems really fast on queries12:56
Kdecherfmewald1: i didn't play with collectd, so I can't tell12:56
mewald1I wanna know for telegraf. For collectd I know it has all I need12:57
Mech422mewald1: I thought I saw a ceph plugin listed for telegraph12:57
Kdecherfbut telegraf seems really light, small footprint and the configuration file is really really short12:57
*** JianqingJiang has joined #openstack-kolla12:58
Kdecherf(which is good, it generates metrics for network, cpu, disks, memory and other stuff out of the box)12:58
Mech422mewald1: https://github.com/influxdata/telegraf/tree/master/plugins/inputs/ceph12:59
Mech422Kdecherf: seems like I should give it a go ...13:00
*** cristicalin has joined #openstack-kolla13:00
KdecherfMech422: yep13:00
*** daneyon has joined #openstack-kolla13:01
Kdecherfanyway i'm interested if anyone has feedback on the Kapacitor/alerting part13:01
Mech422Kdecherf: I actually wasn't gonna bother with that - seemed like grafana or promethus could do it?13:01
Kdecherfi'm not sure that it's the job of grafana13:02
Mech422Kdecherf: probably not13:02
mewald1I guess we should somehow vote to get a majority behind a certain monitoring stack13:03
mewald1before I starting putting time into this13:03
Mech422mewald1: for kolla ?13:03
Mech422mewald1: operators are going to have monitoring in place already, so I'm not sure kolla should setup a 'snowflake'13:04
Mech422mewald1: probably just have the operator install their collector of choice as part of host setup before deploying kolla ?13:05
*** daneyon has quit IRC13:05
Kdecherfwell, as long as openstack has its own influxdb competitor (gnocchi), it could be natural to deploy gnocchi for that13:05
Kdecherfoh, docker killed one of my controllers, again13:06
Mech422Kdecherf: I wasn't aware of gnocchi...13:06
Mech422sooo many openstack projects :-P13:06
KdecherfMech422: it's quite new, and the main goal of gnocchi seems to be a replacement of ceilometer13:06
Kdecherf(another openstack project)13:07
mewald1Mech422: if that was Kolla's approach we wouldnt have elasticsearch + kibana in the stack13:07
Mech422Kdecherf: oh - ceilometer is going away ?13:09
Mech422mewald1: perhaps - I kinda think ELK is there for debugging, not production :-P13:09
KdecherfMech422: not to my knowledge13:09
KdecherfMech422: ELK is production-proof13:09
Mech422mewald1: so you can corrolate all the messages that fly between containers when you make a request13:10
Mech422Kdecherf: no - I mean ELK specifically in Kolla ... it must have been a bitch trying to follow a request thru all the containers and logs without it13:10
Mech422but thats just my guess13:10
*** JianqingJiang has quit IRC13:11
Kdecherfoh, i see13:11
*** salv-orlando has quit IRC13:11
Mech422for some reason, kibana never worked on my kolla cluster... all I get is horizon logs13:13
Mech422that'll be the next thing I work on after I get ceph working again13:13
mewald1I dont believe ELK is there to help developers13:13
Kdecherfhm, the behavior of this controller is really weird13:16
*** jtriley has joined #openstack-kolla13:17
Kdecherftop - 15:17:17 up 5 min,  3 users,  load average: 126.41, 45.97, 16.9313:17
Kdecherfyeah, nice13:17
bjolohas anyone tried to deploy kolla-ansible ontop of coreos or atomic or any other minimal distro?13:17
bjoloKdecherf, ouch13:17
Mech422Kdecherf: Did you forget to put the fire out ? :-)13:18
Mech422Kdecherf: whats thrashing the machine ?13:18
*** JianqingJiang has joined #openstack-kolla13:18
Kdecherfbjolo: i am tempted to replace docker with rkt13:20
*** cristicalin has quit IRC13:21
KdecherfMech422: well, i though we could have more heat in the room, it's pretty cold right now13:21
bjolodont know much about rkt13:21
Mech422Kdecherf: yeah - that'll make a nice lil space heater :-)13:21
*** sdake has joined #openstack-kolla13:21
Mech422Kdecherf: what sort of hosts you running on ?13:21
Kdecherfbjolo: coreos's container tool13:22
bjolowas more wondering if there is anything in the kolla-ansible design that prevents it from working on minial os distros. e.g. net=host, host mounts, etc that kolla is making use of13:22
*** mewald has joined #openstack-kolla13:22
rhalliseybjolo, I've done it on atomic13:22
mewald1how is a question like which perf monitoring stack to use or whether to use one at all solved in a open source community like this?13:23
bjolorhallisey, cool13:23
bjolorhallisey, any workarounds neeeded or worked out of the box?13:23
rhalliseybjolo, worked for me13:25
Kdecherfhttp://i.imgur.com/4nown7C.png well..13:25
rhalliseyany issues I had in the past I added into kolla13:25
rhalliseyso it should work fine13:25
bjolofrom a sysadmin/operator perspective, i see a bright future for minimal distros. (and equally dark future for phat distros like ubuntu, rhel, etc). Devs prefer to use the container approach for development13:25
sdakemewald1 whoever writes the code gets to decide13:25
rhalliseyhttps://github.com/openstack/kolla/blob/cac7be2f3bf5fddd48aa35e83a855681ae93aafb/docker/nova/nova-libvirt/extend_start.sh#L613:25
bjolomeaning that ALL physical hosts will be sort of hypervisor13:26
sdakemewald1 naturally the individual most interested in wrigin the code is most likely to have the best opinions about which choice to make ;)13:26
mewaldsdake: that's good news:D13:27
sdakeKdecherf you have a kernel backtrace13:27
*** mewald1 has quit IRC13:27
Mech422Kdecherf: Ouch!13:28
*** cristicalin has joined #openstack-kolla13:28
*** athomas has quit IRC13:29
Kdecherfsdake: i have a lot of kernel backtraces here13:30
sdakefwiw a system should never backtrace13:30
*** dwalsh has joined #openstack-kolla13:34
*** dwalsh_ has joined #openstack-kolla13:34
*** Jeffrey4l has joined #openstack-kolla13:36
*** nradojevic has left #openstack-kolla13:37
*** diogogmt has quit IRC13:42
*** dwalsh has quit IRC13:43
*** dwalsh_ has quit IRC13:43
*** cristicalin has quit IRC13:43
*** diogogmt has joined #openstack-kolla13:44
*** mgoddard_ has joined #openstack-kolla13:45
*** mgoddard has quit IRC13:49
*** lyncos has joined #openstack-kolla13:52
*** diogogmt has quit IRC13:55
*** athomas has joined #openstack-kolla13:58
*** sdake_ has joined #openstack-kolla14:02
*** shardy has joined #openstack-kolla14:03
*** sdake has quit IRC14:03
*** sdake has joined #openstack-kolla14:05
*** diogogmt has joined #openstack-kolla14:05
*** JianqingJiang has quit IRC14:05
bjolomewald, tnx for the SSL tip about the order to cat them together. I switched around to mycrt, intermediate crt, my key and now it works14:05
Kdecherfit's time to burn this test env14:06
*** rhallisey has quit IRC14:07
*** sdake_ has quit IRC14:07
*** rhallisey has joined #openstack-kolla14:08
*** salv-orlando has joined #openstack-kolla14:08
*** salv-orlando has quit IRC14:09
*** JianqingJiang has joined #openstack-kolla14:09
*** salv-orlando has joined #openstack-kolla14:10
Mech422Kdecherf: yeah - I fully reformat my test machines once a day...14:10
KdecherfMech422: well, i will just pause my eval of kolla14:11
Mech422Kdecherf: yeah - might be good to make sure the hardware/network isn't causing you grief...14:12
Mech422I'm sat here trying to figure out some stupid GSSAPI error on my ldap stuff14:13
coolsvapsdake: we need more milestones for stable branches14:14
*** diogogmt has quit IRC14:14
*** bdaca has joined #openstack-kolla14:14
sdakeagree14:14
sdakei'm goin gback to bed - bbiaf :)14:15
*** unicell1 has quit IRC14:15
coolsvap:)14:15
*** dwalsh has joined #openstack-kolla14:17
*** dwalsh_ has joined #openstack-kolla14:18
*** JianqingJiang has quit IRC14:20
*** salv-orlando has quit IRC14:21
*** unicell has joined #openstack-kolla14:22
*** bjolo has quit IRC14:29
*** britthouser has joined #openstack-kolla14:30
*** dave-mccowan has joined #openstack-kolla14:33
*** lyncos has quit IRC14:34
*** shardy is now known as shardy_mtg14:35
*** huikang has joined #openstack-kolla14:37
*** huikang has quit IRC14:41
*** huikang has joined #openstack-kolla14:42
*** daneyon has joined #openstack-kolla14:49
*** zhurong has quit IRC14:50
*** daneyon has quit IRC14:53
*** jruano has joined #openstack-kolla14:54
*** mgoddard_ has quit IRC14:58
*** mgoddard has joined #openstack-kolla14:59
*** diogogmt has joined #openstack-kolla15:02
*** bootsha has quit IRC15:02
*** b_bezak has quit IRC15:04
*** haplo37_ has joined #openstack-kolla15:04
*** b_bezak has joined #openstack-kolla15:04
*** Liuqing has quit IRC15:07
*** b_bezak has quit IRC15:09
*** banix has joined #openstack-kolla15:09
*** unicell has quit IRC15:16
*** jogam has quit IRC15:17
*** jogam has joined #openstack-kolla15:17
*** belmoreira has quit IRC15:18
*** vhosakot has joined #openstack-kolla15:24
*** jogam has quit IRC15:24
*** jogam has joined #openstack-kolla15:24
*** sdake has quit IRC15:26
*** dave-mccowan has quit IRC15:29
*** sdake has joined #openstack-kolla15:29
bdacaI have a weird question... is there any reason why mitaka uses nova api v2, not v2.1 by default?15:29
*** inc0 has joined #openstack-kolla15:29
inc0good morning15:30
Mech422inc0: Morning15:30
dasmbdaca: it's not in kolla, but johnthetubaguy answered for neutron: https://review.openstack.org/#/c/324232/4//COMMIT_MSG@1015:30
patchbotdasm: patch 324232 - neutron - Update nova api 2.0 to 2.115:31
dasmbdaca: "I just wanted to be clear, Nova still supports the /2.0 API, and has no plans to drop that API. All the same requests that worked before, should continue to work today, and long into the future."15:31
Mech422anyone tried the Quanta 10Gig switches? They're going cheap on Ebay....15:36
*** lyncos has joined #openstack-kolla15:39
*** dmsimard is now known as dmsimard|afk15:39
bdacadasm: so as I understand kolla keeps v2 API because it works, but switching to 2.1 should be probably safe (as v2.1 is suggested in official mitaka deployment guide)15:39
dasmbdaca: but in the same time: "This change will stops neutron working with older versions of Nova that didn't have the v2.1 API. Although moving to v2.1 API is probably a good overall, because you will get some stronger input validation."15:40
dasmbdaca: but i'm not quite ure why kolla keeps v215:41
inc0well, I'd love 2.1 to be up for at least a release15:41
inc0so if we upgrade neutron first nova second, it won't break15:41
inc0however we upgrade nova first now, but just to be on the safe side15:41
*** dave-mccowan has joined #openstack-kolla15:42
*** mewald has quit IRC15:43
harlowjacoolsvap whats up, saw some ping :-P15:44
harlowjaping/mention15:44
*** inc0 has quit IRC15:45
coolsvapharlowja: related to the oslo.context issue related to requirements15:45
coolsvapi saw mention in oslo meeting15:45
*** inc0 has joined #openstack-kolla15:45
coolsvap;)15:45
harlowjaah15:46
*** mbound has quit IRC15:46
harlowjaafaik its not an issue, people are doing things weirdly with tests in there project, lol15:47
Mech422inc0: so, I think I'm working thru my ceph issues.  That patch hard codes the location of '/usr/sbin/sgdisk', which breaks on ubuntu...Any objection to just running it with shell=True and letting the search path find the bin ?15:47
coolsvapharlowja: yes15:47
inc0Mech422, not really15:47
coolsvaptopic for sometime later at #openstack-dev perhaps15:47
inc0I mean shell=true has security issues15:47
coolsvapharlowja: ^^15:48
*** lyncos has quit IRC15:48
harlowjacoolsvap sure15:48
Mech422inc0: yeah - but if someone can mess with your containers during build, you have bigger issues15:48
inc0publish a patch, I'll review15:48
*** daneyon_ has joined #openstack-kolla15:49
Mech422inc0: also, logic was added to try and use that udev library if its 'new enough' - I have a suspicion thats not working ..15:49
inc0Paul, who added it, is on PTO now15:49
inc0so you might want to hold on to that thought till he's back15:49
Mech422yeah - I know15:49
Mech422fair enough15:49
inc0I'm sure he'd like to know too15:50
*** hrito has joined #openstack-kolla15:50
inc0but what is it?15:50
Mech422inc0: what is what ?15:50
inc0the suspicion15:50
inc0what's not working?15:50
Mech422inc0: oh - it says something like 'if udev.version > 1800 then use udev'15:51
*** v1k0d3n has joined #openstack-kolla15:51
sdakeinc0 speculation is the word your looking for ;)15:51
Mech422inc0: my udev is higher then that, but it appears not to be working15:51
inc0sdake, I know I always can count on you ;)15:51
inc0in that matter at least;)15:51
sdakegotta be good at something15:51
sdakegrammar policia ftw15:51
inc0maybe it's version that's needs checking?15:51
inc0or it's not about udev version at all?15:52
Mech422inc0: but I haven't looked to see if the partition table re-write does the 'force udev reload' thing15:52
*** harlowja has quit IRC15:52
Mech422inc0: dunno15:52
inc0sdake, https://www.reddit.com/r/Jokes/comments/2dmirb/sir_were_mining_too_many_useless_minerals/15:52
inc0sdake, I'll miss todays meeting unfortunately:(15:54
*** mbound has joined #openstack-kolla15:54
*** unicell has joined #openstack-kolla15:57
*** mgoddard_ has joined #openstack-kolla16:00
coolsvapmeeting time16:01
*** lyncos has joined #openstack-kolla16:01
coolsvapsdake: ^^16:02
*** mgoddard has quit IRC16:03
Mech422coolsvap: openstack-meeting4 ?16:03
coolsvapyes16:04
coolsvapMech422: yes16:04
Mech422coolsvap: thx!16:04
*** vhosakot has quit IRC16:07
*** vhosakot has joined #openstack-kolla16:08
*** dmueller-evoila has quit IRC16:10
*** vhosakot has quit IRC16:14
*** vhosakot has joined #openstack-kolla16:14
*** mgoddard_ has quit IRC16:14
*** mgoddard has joined #openstack-kolla16:14
*** david-lyle has joined #openstack-kolla16:19
*** mdnadeem_home has joined #openstack-kolla16:20
*** dwalsh has quit IRC16:21
*** dwalsh_ has quit IRC16:21
*** shardy_mtg has quit IRC16:27
*** Serlex has quit IRC16:32
*** athomas has quit IRC16:33
*** vhosakot has quit IRC16:36
*** unicell has quit IRC16:36
*** vhosakot has joined #openstack-kolla16:36
*** daneyon has joined #openstack-kolla16:38
*** ayoung has joined #openstack-kolla16:38
*** lyncos has quit IRC16:39
*** harlowja has joined #openstack-kolla16:40
*** mark-casey has joined #openstack-kolla16:40
*** daneyon has quit IRC16:42
*** david-lyle has quit IRC16:42
*** jruano has quit IRC16:48
openstackgerritPrithiv proposed openstack/kolla: Support Networking-SFC Container  https://review.openstack.org/34493016:53
*** vhosakot has quit IRC16:54
*** bdaca has quit IRC16:55
Mech422inc0: so yeah - forcing the ceph stuff thru the sgdisk code seems to get my cluster back..16:56
*** hrito has left #openstack-kolla16:56
Mech422inc0: I'll try and crank out a patch, and I can just apply it locally till pburke gets back16:56
*** vhosakot has joined #openstack-kolla17:01
*** jascott1_tmp has joined #openstack-kolla17:03
*** gfidente has quit IRC17:04
*** sdake has quit IRC17:07
*** sdake has joined #openstack-kolla17:11
sdakeinc0 around?17:11
*** sdake has quit IRC17:16
*** unicell has joined #openstack-kolla17:18
*** vhosakot has quit IRC17:20
*** mdnadeem_home has quit IRC17:26
*** dave-mccowan has quit IRC17:37
*** ntpttr has quit IRC17:47
*** ntpttr has joined #openstack-kolla17:48
*** lyncos has joined #openstack-kolla17:50
*** Jeffrey4l has quit IRC17:51
*** bootsha has joined #openstack-kolla17:53
*** dave-mccowan has joined #openstack-kolla17:57
*** michauds has joined #openstack-kolla18:01
openstackgerritChristian Berendt proposed openstack/kolla: [ceilometer] use haproxy to access mongodb  https://review.openstack.org/34496118:02
*** mbound has quit IRC18:04
*** jascott1_tmp has quit IRC18:16
*** jruano has joined #openstack-kolla18:17
*** harlowja has quit IRC18:19
*** bootsha has quit IRC18:23
*** bootsha has joined #openstack-kolla18:25
*** dmsimard|afk is now known as dmsimard18:26
*** jruano has quit IRC18:28
v1k0d3nhey all...starting to move my kolla testing phase into a multi-node deployment rather than just straight aio...would it be ok to use a 5 node cluster running all containers (both controllers network and compute)?18:28
rhalliseyv1k0d3n, ya go for it18:29
v1k0d3nok, so no real issues to worry about then or consider?18:29
v1k0d3nalso, i guess do i put all node everywhere...like for glance nova-compute etc?18:29
v1k0d3nor do i need to leave the additional controls in place?18:30
*** bootsha has quit IRC18:30
*** bootsha has joined #openstack-kolla18:30
v1k0d3njust just edit network, compute and control sections?18:30
rhalliseyv1k0d3n, depends what you're after. Check out the multinode inventory file18:31
rhalliseykolla/ansible/inventory/multinode18:31
v1k0d3nthat's what i'm looking at now. lot of inventory groups. i kind of want all boxes to be the same, and share resources/control.18:32
openstackgerritPrithiv proposed openstack/kolla: Support Networking-SFC Container  https://review.openstack.org/34493018:32
rhalliseyok so add the addresses for all machines under all roles18:32
*** jruano has joined #openstack-kolla18:33
*** jruano has quit IRC18:35
*** bootsha has quit IRC18:59
*** unicell1 has joined #openstack-kolla19:03
*** jruano has joined #openstack-kolla19:03
*** jruano has quit IRC19:04
*** unicell has quit IRC19:04
*** mbound has joined #openstack-kolla19:04
*** jruano has joined #openstack-kolla19:05
*** jruano has quit IRC19:05
*** unicell1 has quit IRC19:06
*** mbound has quit IRC19:10
*** unicell has joined #openstack-kolla19:10
*** unicell1 has joined #openstack-kolla19:11
*** unicell has quit IRC19:11
*** unicell1 has quit IRC19:12
*** unicell has joined #openstack-kolla19:12
*** harlowja has joined #openstack-kolla19:19
*** harlowja has quit IRC19:36
*** sdake has joined #openstack-kolla19:38
*** lyncos has quit IRC19:39
v1k0d3nok, that's what i thought.19:39
v1k0d3n@rhallisey is pretty much everything else the same as it would in the main config file?19:40
v1k0d3ni mean same as it would be for an AIO node?19:40
v1k0d3nseems like it really...if it is, that's super straight forward, unless i'm missing something.19:40
*** smartshader has joined #openstack-kolla19:45
smartshaderHi everyone, I am playing a little bit with kolla and have a question regarding nova-compute container and nova-ssh container19:46
smartshaderhow does nova-compute container use the ssh-agent inside the nova-ssh container19:46
*** sdake has quit IRC19:47
*** bwe_ has quit IRC19:52
rhalliseyv1k0d3n, ya if you put all you hosts under each role, you can leave everything the same19:56
*** Liuqing has joined #openstack-kolla19:58
*** aernhart__ has quit IRC20:00
*** prithiv has joined #openstack-kolla20:01
rhalliseysmartshader, Jeffrey4l would know.  He's not around at the moment20:02
*** mewald has joined #openstack-kolla20:02
smartshaderrhallisey, thank you for the prompt response. I will stay around to look for Jeffrey4l20:03
rhalliseysmartshader, he's UTC+8 I think.  He won't be around for a few hours20:04
*** jascott1_tmp has joined #openstack-kolla20:10
smartshaderrhallisey, Thanks but after thinking with my collegues we got our answer20:12
rhalliseyawesome20:13
*** daneyon has joined #openstack-kolla20:14
sean-k-mooneysmartshader: nova or rather libvirt use the nova-ssh container to supprot livemightion20:14
sean-k-mooney/livemightion/live migration/20:15
*** daneyon has quit IRC20:18
*** huikang has quit IRC20:26
*** smartshader has quit IRC20:28
*** salv-orlando has joined #openstack-kolla20:30
*** jtriley_ has joined #openstack-kolla20:31
*** jtriley has quit IRC20:34
*** jtriley_ has quit IRC20:36
*** mbound has joined #openstack-kolla20:51
*** jogam has quit IRC20:52
*** jogam has joined #openstack-kolla20:52
*** salv-orl_ has joined #openstack-kolla20:59
*** salv-orlando has quit IRC20:59
*** jruano has joined #openstack-kolla20:59
*** salv-orlando has joined #openstack-kolla21:03
*** salv-orl_ has quit IRC21:04
*** Liuqing has quit IRC21:12
*** banix has quit IRC21:17
*** prithiv has quit IRC21:21
*** jascott1_tmp has quit IRC21:21
*** jascott1_tmp has joined #openstack-kolla21:22
*** jruano has quit IRC21:26
*** haplo37_ has quit IRC21:27
*** rhallisey has quit IRC21:32
*** aernhart__ has joined #openstack-kolla21:35
*** aernhart__ has quit IRC21:40
*** britthou_ has joined #openstack-kolla21:42
*** britthouser has quit IRC21:44
*** inc0 has quit IRC21:48
*** daneyon has joined #openstack-kolla22:02
*** daneyon has quit IRC22:07
*** britthouser has joined #openstack-kolla22:10
*** harlowja has joined #openstack-kolla22:11
*** rhallisey has joined #openstack-kolla22:11
*** harlowja has quit IRC22:11
*** britthou_ has quit IRC22:12
*** signed8bit has joined #openstack-kolla22:12
*** mark-casey has quit IRC22:44
*** rhallisey has quit IRC22:45
*** jascott1_tmp has quit IRC22:50
*** rhallisey has joined #openstack-kolla22:57
*** harlowja has joined #openstack-kolla23:00
*** signed8bit is now known as signed8bit_Zzz23:03
*** sogabe has joined #openstack-kolla23:03
*** harlowja has quit IRC23:03
*** michauds has quit IRC23:03
*** harlowja has joined #openstack-kolla23:07
*** britthouser has quit IRC23:11
*** britthouser has joined #openstack-kolla23:12
*** harlowja has quit IRC23:12
*** mbound has quit IRC23:37
*** ayoung has quit IRC23:39
*** aernhart has joined #openstack-kolla23:39
*** inc0 has joined #openstack-kolla23:47
*** zhurong has joined #openstack-kolla23:48
openstackgerritDavid Wang proposed openstack/kolla-kubernetes: WIP(do not merge) Add status command for Workflow Engine  https://review.openstack.org/34510223:51
*** daneyon has joined #openstack-kolla23:51
*** daneyon has quit IRC23:55

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!