Saturday, 2017-05-27

*** manheim has joined #openstack-kolla00:07
*** srwilkers has joined #openstack-kolla00:09
*** kbaegis has joined #openstack-kolla00:12
kbaegisHey guys. What was the haproxy buy associated with solum?00:12
*** manheim has quit IRC00:16
*** unicell has quit IRC00:20
*** itlinux has joined #openstack-kolla00:21
*** salv-orlando has joined #openstack-kolla00:23
*** manheim has joined #openstack-kolla00:25
*** salv-orlando has quit IRC00:27
*** manheim has quit IRC00:30
*** manheim has joined #openstack-kolla00:39
*** harlowja has quit IRC00:41
*** manheim has quit IRC00:48
*** emccormick has joined #openstack-kolla00:51
*** srwilkers has quit IRC00:54
*** Pavo has joined #openstack-kolla00:54
*** ddyer2 has quit IRC00:56
*** manheim has joined #openstack-kolla00:57
*** manheim has quit IRC01:02
*** itlinux has quit IRC01:10
*** manheim has joined #openstack-kolla01:11
*** Pavo has quit IRC01:14
*** manheim has quit IRC01:20
*** salv-orlando has joined #openstack-kolla01:24
*** salv-orlando has quit IRC01:28
*** jtriley has joined #openstack-kolla01:29
*** harlowja has joined #openstack-kolla01:29
sdakedmsimard has rdo ocata undergone some kind of change in mariadb?01:32
*** hrw has quit IRC01:32
*** manheim has joined #openstack-kolla01:34
*** hrw has joined #openstack-kolla01:34
*** emccormick has quit IRC01:36
*** manheim has quit IRC01:38
*** emccormick has joined #openstack-kolla01:39
*** stefan_bo has joined #openstack-kolla01:41
*** manheim has joined #openstack-kolla01:47
sdakeinc0 we may be forced to eject xtradbbackup01:49
sdakeinc0 can you confirm if it is needed01:49
*** eanylin has quit IRC01:52
*** rhallisey has joined #openstack-kolla01:53
*** eaguilar has quit IRC01:56
*** manheim has quit IRC02:00
*** harlowja has quit IRC02:01
*** rhallisey has quit IRC02:01
*** hieulq has quit IRC02:04
*** caowei has joined #openstack-kolla02:04
*** manheim has joined #openstack-kolla02:04
*** manheim has quit IRC02:09
*** emccormick has quit IRC02:14
*** eaguilar has joined #openstack-kolla02:17
*** emccormick has joined #openstack-kolla02:18
*** eanylin has joined #openstack-kolla02:22
*** emccormick has quit IRC02:24
*** hieulq has joined #openstack-kolla02:24
*** emccormick has joined #openstack-kolla02:25
*** hieulq has quit IRC02:27
sdakeJeffrey4l if your aroudn coudl use some help02:31
*** kbaegis has quit IRC02:31
*** kbaegis has joined #openstack-kolla02:32
*** manheim has joined #openstack-kolla02:40
*** hieulq has joined #openstack-kolla02:43
*** manheim has quit IRC02:45
*** eaguilar has quit IRC02:48
*** itlinux has joined #openstack-kolla02:56
*** itlinux has quit IRC02:58
*** lucasxu has joined #openstack-kolla03:14
*** manheim has joined #openstack-kolla03:17
*** emccormick has quit IRC03:18
openstackgerritLi Yingjun proposed openstack/kolla master: Add os-brick to glance container  https://review.openstack.org/46860403:19
*** itlinux has joined #openstack-kolla03:20
*** manheim has quit IRC03:21
*** salv-orlando has joined #openstack-kolla03:25
*** salv-orlando has quit IRC03:30
*** unicell has joined #openstack-kolla03:33
*** stefan_bo has quit IRC03:33
*** lucasxu has quit IRC03:35
sdakesbezverk that cell integration is broken03:41
sdakesbezverk placement must register prior to cells03:41
sdakecell0 enters restart loop03:42
openstackgerritshaofeng cheng proposed openstack/kolla-ansible master: Enabled pxe_ssh driver in ironic  https://review.openstack.org/46860603:45
*** manheim has joined #openstack-kolla03:53
*** manheim has quit IRC03:57
*** stefan_bo has joined #openstack-kolla04:15
*** salv-orlando has joined #openstack-kolla04:26
*** manheim has joined #openstack-kolla04:29
*** salv-orlando has quit IRC04:30
*** stefan_bo has quit IRC04:31
*** salv-orlando has joined #openstack-kolla04:32
*** manheim has quit IRC04:34
*** kbaegis has quit IRC04:34
*** harlowja has joined #openstack-kolla04:35
openstackgerritSteven Dake proposed openstack/kolla master: Remove sudo operations that are no longer necessary  https://review.openstack.org/46860704:40
*** mewald has quit IRC04:49
*** manheim has joined #openstack-kolla05:05
*** salv-orl_ has joined #openstack-kolla05:07
*** salv-orlando has quit IRC05:07
*** manheim has quit IRC05:09
*** salv-orl_ has quit IRC05:10
*** salv-orlando has joined #openstack-kolla05:10
*** itlinux has quit IRC05:14
*** manheim has joined #openstack-kolla05:14
*** salv-orlando has quit IRC05:15
*** manheim has quit IRC05:23
*** ntpttr has quit IRC05:29
*** ntpttr has joined #openstack-kolla05:29
*** stefan_bo has joined #openstack-kolla05:31
*** manheim has joined #openstack-kolla05:32
*** harlowja has quit IRC05:32
*** manheim has quit IRC05:37
Jeffrey4lsdake, still around? sorry busy recently.05:41
sdakeJeffrey4l cany ou reiew https://review.openstack.org/46860705:41
sdakeJeffrey4l no reason to apologize for RL :)05:41
*** manheim has joined #openstack-kolla05:42
sdakealso stable/ocata mariadb does not build properly05:42
*** hieulq has quit IRC05:42
Jeffrey4lwill we really moved to set_config configuration, which is configurable by deploy tool?05:43
sdakeright - which is the rationale behidn that change05:43
Jeffrey4li prefer to save these in image side. ( move the configuration from config.json  to image ).05:44
Jeffrey4lthe benefit of this is deploy tool do not care about the permissions.05:44
sdakedeploy tool often care sabout permissions and the containers don't have the knoweldge to make the right call re premissions05:44
sdakethat is why we added that work in the fist place05:45
*** janki has joined #openstack-kolla05:45
Jeffrey4lok.05:46
Jeffrey4lsdake, base on that, should we move " mkdir -p /var/log/kolla/elasticsearch" such command into deploy tool part?05:47
sdakecan we maek dirctories with config.json?05:47
Jeffrey4lno right now.05:48
sdakeprobably good followup work05:48
Jeffrey4lbut we can add such feature.05:48
sdakeeach deploy tool will want slightly differnet layout on stuff and permissions05:48
sdakethat is why config.json exists05:48
Jeffrey4lon the other hand, we are re-implement ansible in set_config ( config.json ) rignt now.05:48
Jeffrey4lhow about use ansible or re-use ansible module?05:49
sdakei need a backport of this change for ocata05:49
sdakehence needs to go in master first05:49
sdakeansibleilziing that is a feature imo :)05:49
Jeffrey4lknow that. beside that, what do u thin of my idea?  use ansible to init the container images.05:50
sdakeseems reasonble minus the heavyweight content of ansible in all the containers05:50
sdakeset_configs.py works well05:50
sdakehate ot change what isn't broken :)05:50
*** manheim has quit IRC05:50
*** manheim has joined #openstack-kolla05:51
sdakej think a more pressing problem is stable/ocata of kolla repo doesn't build05:51
Jeffrey4li'd like use what ever we already have. when we trying to add more funcation into set_config.py, we can re-use what we have right now.05:52
Jeffrey4lok. got.05:52
Jeffrey4liirc, master is fix. may backport works.05:52
sdakeyou fixed master for this rpoblem?05:52
sdake(the mariadb galera interaction)05:52
Jeffrey4lno.05:53
Jeffrey4llet me search for that.05:53
sdakei have a patch which builds mariadb05:53
sdakealthough it ejects percona in the process05:53
Jeffrey4lsdake, https://review.openstack.org/467360  i think.05:53
sdakeya - i dont htink that really fixes it05:54
sdakeit may slow down the pain level a little05:55
*** manheim has quit IRC05:55
sdakeJeffrey4l look this patch: https://da.gd/x40sx05:57
sdakethe package names have changed - somewhere05:57
sdake-        'MariaDB-Galera-server',05:57
sdake-        'MariaDB-client',05:57
sdake+        'mariadb-server-galera',05:57
sdake+        'mariadb-client',05:57
sdakeatm, there is 3 copies of mariadb-server-galera in our repos05:57
sdakeone  from percona, one from mariadb, and one from centos-ocata05:57
sdakethis is a recent change, i built this mornign without problem05:58
sdakeMariaDB-Galea-server was named "Mariadb-server-galera"05:58
sdakei think we really need to be on redhat's version of mariadb05:58
sdakethey have a team of 20+ people maintaing it05:58
Jeffrey4lok.  sound good05:58
sdakethe problem is xtradb05:59
sdakei dont know what it does05:59
Jeffrey4l iirc, we depends on that tools for data sync05:59
sdakebut it only comes from the percona repo05:59
sdakei get the impression it is not mandatory05:59
sdakebut was chosen early on05:59
sdakewithout alot of good rationale05:59
sdakeunfortunately I dont know how to ge trid of it because i dont really knwo what it does06:00
Jeffrey4lsdake, this line https://github.com/openstack/kolla-ansible/blob/master/ansible/roles/mariadb/templates/galera.cnf.j2#L4006:00
Jeffrey4li  will dig this issue more and try to get more info about it.06:00
sdakei think one optoin is just to install the RPM dirctly rather then laod in the repos06:01
sdakeif we really need it06:01
sdakeand its hard to break compatiblity in a zstream06:01
sdakeso we are stuck with it for ocata06:01
Jeffrey4lid like that if it work properly as now.06:01
sdakethat patch works - without the galera clustering part06:01
sdakeits got some netapp cruft in it06:02
sdakealthough atm, stable/ocata is unbildable06:02
sdakei think all ocata is unbuildable06:02
Jeffrey4lguess so.06:02
sdakewhich means in essence we have a DOA deliverable06:02
sdakerather it builds, but mariadb doesnt' work06:03
Jeffrey4lthis is hard to avoid. we have lots of dependency.06:03
Jeffrey4lif any repo or rpm is bust, kolla may be affected.06:04
sdakeyes06:04
sdakewe need to fix this asap tho and cut a new zstream06:04
sdakei wa planning to build 4.0.1 and I can't do that06:04
sdakeas it would be a DOA tag06:04
Jeffrey4lanyway, remove percona should be a good direct if it works.06:04
Jeffrey4lagree.06:04
sdakewe can't remove that xtradb crap in a zstream06:05
sdakeso we are stuck wit htaht at a minimum06:05
*** manheim has joined #openstack-kolla06:05
sdakelast i looked at ejecting it, we only use one package06:05
sdakethe questio nis is that package compatible with rdos' mariadb06:05
Jeffrey4lnot sure right now.06:06
*** hieulq has joined #openstack-kolla06:09
Jeffrey4lsdake, check this method06:12
Jeffrey4lsdake, check this http://galeracluster.com/documentation-webpages/mysqlwsrepoptions.html#wsrep-sst-method06:12
Jeffrey4lxtrabackup This option is a fast and practically non-blocking state transfer method based on the Percona xtrabackup tool. If you want to use it, the following settings must be present in the my.cnf configuration file on all nodes:06:12
sdakeya06:13
*** manheim has quit IRC06:14
sdakerabbitmq is also in centos ocata repo06:15
sdakemariadb + rabbitmq - i think we should drop the external repos and just use centos for centos derived things06:15
Jeffrey4lyes. we moved it from rabbitmq repo to centos's06:16
Jeffrey4lagree. that will be more stable06:16
sdakemvoed when?06:16
sdakestable/ocata i think doesn't have this change?06:16
Jeffrey4lit has06:16
Jeffrey4lcheck it06:16
sdakefianlly woudl liek to drop epel from master ;)06:17
sdakethe rdo guys complain loudly that we still use epel06:18
Jeffrey4lthat may a little hard .06:18
sdakeand for the most part epel is not needed06:18
Jeffrey4lya, i heard that.06:18
sdakealthough i took epel out and the build craters06:18
sdakeanyway we can tackle that later, mostly interested in sorting out making stable/ocata build :)06:19
sdakei can do it single node, i dont really ahve a way to test multinode06:19
Jeffrey4lxtrabackup tools should be the key point.06:21
sdakethats the part i can't test06:21
sdakei can do the implementation06:21
sdakebut our gates dont etst06:21
Jeffrey4lah06:21
sdakeand i dont hae a 3 node setu pof kolla-ansile06:21
sdakekolla-kubernetes does not use galera06:21
sdakewe use single node database, database dies, we just rechedule it and have a short outage06:22
*** manheim has joined #openstack-kolla06:23
Jeffrey4lok. i will check it06:24
*** manheim has quit IRC06:28
*** janki has quit IRC06:32
*** manheim has joined #openstack-kolla06:37
*** salv-orlando has joined #openstack-kolla06:41
sdakeoen of hte problems is mariadb-lib comes from epel..06:42
sdake(and percona and mariadb and ocata-release...) iirc06:42
*** salv-orlando has quit IRC06:45
*** manheim has quit IRC06:45
*** manheim has joined #openstack-kolla06:46
sdakeJeffrey4l thanks for the pointer about that pin06:46
sdakeJeffrey4l removing the pin (reverting that change) I get:06:47
sdakeINFO:kolla.image.build.mariadb:Error: Package: 3:mariadb-server-galera-10.1.20-1.el7.x86_64 (delorean-pike-testing)06:47
sdakeINFO:kolla.image.build.mariadb:           Requires: mariadb-server(x86-64) = 3:10.1.20-1.el706:47
sdakeINFO:kolla.image.build.mariadb:           Available: MariaDB-server-10.0.31-1.el7.centos.x86_64 (mariadb)06:47
sdakeINFO:kolla.image.build.mariadb:               mariadb-server(x86-64) = 1:10.0.31-1.el7.centos06:47
sdakeINFO:kolla.image.build.mariadb:           Available: 1:mariadb-server-5.5.52-1.el7.x86_64 (base)06:47
sdakeINFO:kolla.image.build.mariadb:               mariadb-server(x86-64) = 1:5.5.52-1.el706:47
sdakeINFO:kolla.image.build.mariadb:           Available: 3:mariadb-server-10.1.20-1.el7.x86_64 (delorean-pike-testing)06:47
sdakeINFO:kolla.image.build.mariadb:               mariadb-server(x86-64) = 3:10.1.20-1.el706:47
Jeffrey4lok. chaotic world06:49
*** stefan_bo has quit IRC06:52
*** manheim has quit IRC06:55
sdakei think if I could solve this:07:01
sdakeINFO:kolla.image.build.mariadb:Error: Package: 3:mariadb-10.1.20-1.el7.x86_64 (delorean-pike-testing)07:01
sdakeINFO:kolla.image.build.mariadb:           Requires: mariadb-libs(x86-64) = 3:10.1.20-1.el707:01
sdakeINFO:kolla.image.build.mariadb:           Available: 1:mariadb-libs-5.5.52-1.el7.x86_64 (base)07:01
sdakeINFO:kolla.image.build.mariadb:               mariadb-libs(x86-64) = 1:5.5.52-1.el707:01
sdakeINFO:kolla.image.build.mariadb:           Available: 3:mariadb-libs-10.1.20-1.el7.x86_64 (delorean-pike-testing)07:01
sdakeINFO:kolla.image.build.mariadb:               mariadb-libs(x86-64) = 3:10.1.20-1.el707:01
sdakewe are good to go07:01
sdakei dont understand why yum ins't pulling the dependency it requires07:01
*** manheim has joined #openstack-kolla07:08
*** manheim has quit IRC07:18
*** dciabrin has quit IRC07:18
*** manheim has joined #openstack-kolla07:19
*** manheim has quit IRC07:29
*** manheim has joined #openstack-kolla07:29
*** caoyuan has joined #openstack-kolla07:32
*** dciabrin has joined #openstack-kolla07:35
*** shardy has quit IRC07:39
*** salv-orlando has joined #openstack-kolla07:42
*** salv-orlando has quit IRC07:46
*** jascott1 has quit IRC07:47
*** jascott1 has joined #openstack-kolla07:48
*** jascott1 has quit IRC07:52
*** tvignaud_pto has quit IRC08:02
sdakei have found the problem - percona obsoltes mariadb-lib08:04
sdakebunch of fial tha tis08:04
*** manheim has quit IRC08:10
*** tvignaud_pto has joined #openstack-kolla08:15
*** manheim has joined #openstack-kolla08:15
*** manheim has quit IRC08:19
*** mewald has joined #openstack-kolla08:19
Jeffrey4lsdake, i just test ocata branch ,which works ( re mariadb issue )08:23
sdakeyou modified the code?08:24
sdakemariadb doesn't install..08:24
Jeffrey4lno. just use stable/ocata codes08:28
Jeffrey4lwhat issue did u  encounter?08:30
Jeffrey4lbuild or deployment?08:30
*** pbourke has quit IRC08:31
*** pbourke has joined #openstack-kolla08:31
*** salv-orlando has joined #openstack-kolla08:33
sdakedeploy fails08:44
sdakebuild also fails if your paying attention08:44
sdakei've got an install technique that works08:45
*** pcaruana has joined #openstack-kolla08:50
*** manheim has joined #openstack-kolla08:51
openstackgerritSteven Dake proposed openstack/kolla master: Consume centos-pike-testing mariadb version  https://review.openstack.org/46863208:54
*** manheim has quit IRC08:56
sdakeprobably not completely correct08:57
sdakebut pretty close08:57
sdakeok bedtim e:)08:57
*** mewald has quit IRC09:02
*** trozet has quit IRC09:07
*** trozet has joined #openstack-kolla09:08
*** pcaruana has quit IRC09:24
*** manheim has joined #openstack-kolla09:27
*** yingjun has quit IRC09:30
*** manheim has quit IRC09:32
caoyuanplease help review https://review.openstack.org/#/c/435236/09:41
*** manheim has joined #openstack-kolla10:04
*** manheim has quit IRC10:08
*** salv-orlando has quit IRC10:11
*** salv-orlando has joined #openstack-kolla10:13
*** caowei has quit IRC10:36
*** manheim has joined #openstack-kolla10:40
*** manheim has quit IRC10:44
*** jascott1 has joined #openstack-kolla11:03
*** manheim has joined #openstack-kolla11:15
*** manheim has quit IRC11:20
*** manheim has joined #openstack-kolla11:25
*** manheim has quit IRC11:34
*** janki has joined #openstack-kolla11:42
*** janki has quit IRC11:43
*** manheim has joined #openstack-kolla11:43
*** manheim has quit IRC11:47
*** matrohon has joined #openstack-kolla11:54
*** manheim has joined #openstack-kolla11:57
*** salv-orlando has quit IRC11:58
*** salv-orlando has joined #openstack-kolla11:59
*** manheim has quit IRC12:01
*** openstackgerrit has quit IRC12:02
*** salv-orlando has quit IRC12:03
-openstackstatus- NOTICE: The Gerrit service on review.openstack.org is restarting to clear some hung API connections and should return to service momentarily.12:03
*** manheim has joined #openstack-kolla12:11
*** manheim has quit IRC12:15
*** manheim has joined #openstack-kolla12:20
*** manheim has quit IRC12:28
*** manheim has joined #openstack-kolla12:30
*** pcaruana has joined #openstack-kolla12:34
*** iniazi has quit IRC12:45
*** iniazi has joined #openstack-kolla12:50
*** jamesPR has joined #openstack-kolla12:57
*** openstackgerrit has joined #openstack-kolla13:11
openstackgerritBernard Maltais proposed openstack/kolla-ansible master: Updated style of proposed changes for better readability.  https://review.openstack.org/46688113:11
*** matrohon has quit IRC13:17
*** srwilkers has joined #openstack-kolla13:20
*** manheim has quit IRC13:24
*** manheim has joined #openstack-kolla13:25
*** Pavo has joined #openstack-kolla13:28
*** salv-orlando has joined #openstack-kolla13:29
*** srwilkers has quit IRC13:31
*** jamesPR has quit IRC13:32
*** manheim has quit IRC13:33
*** salv-orlando has quit IRC13:34
*** manheim has joined #openstack-kolla13:38
*** kbaegis has joined #openstack-kolla13:41
*** manheim has quit IRC13:42
*** manheim has joined #openstack-kolla13:47
*** manheim has quit IRC13:51
*** manheim has joined #openstack-kolla13:52
openstackgerritSteven Dake proposed openstack/kolla master: Consume centos-pike-testing mariadb version  https://review.openstack.org/46863213:56
*** manheim has quit IRC13:56
*** jgriffith has joined #openstack-kolla14:18
*** manheim has joined #openstack-kolla14:21
*** sbezverk_ has joined #openstack-kolla14:23
openstackgerritSteven Dake proposed openstack/kolla master: Consume centos-pike-testing mariadb version  https://review.openstack.org/46863214:23
sdakeberendt you around?14:24
sdakepbourke ?14:25
*** sbezverk has quit IRC14:26
*** salv-orlando has joined #openstack-kolla14:30
*** salv-orlando has quit IRC14:35
*** manheim has quit IRC14:40
openstackgerritSteven Dake proposed openstack/kolla master: Consume centos-pike-testing mariadb version  https://review.openstack.org/46863214:48
kbaegisHey sdake? If I wanted to prevent the ovs-vswitchd daemon from insmodding15:02
kbaegisHow would I patch that?15:03
kbaegis(I have ovs built-in, not as a module)15:03
*** Pavo has quit IRC15:12
*** manheim has joined #openstack-kolla15:16
*** manheim has quit IRC15:21
sdakekbaegis no idea15:21
sdakekbaegis would require custom code i think15:21
kbaegissdake: Thanks15:23
*** salv-orlando has joined #openstack-kolla15:31
*** salv-orlando has quit IRC15:35
inc0good morning15:37
inc0sdake: what's up with mariadb?15:37
sdakethere are 3 repositories with mariadb in it15:37
sdakepercona, mariadb, and rdo15:38
sdakethat change chooses rdo15:38
sdakecurrently stable/ocata doesn't work15:38
inc0careful with that, last time we did that for ubuntu we broke multinode15:38
sdakei understand15:38
sdakehowever everything is broken atm15:39
inc0for centos?15:40
sdakeinc0 can you review this plz: https://review.openstack.org/#/c/468607/15:41
sdakeyes centos mariadb doesn't ubild15:42
sdakemy change builds and works (for kolla-kubernetes)15:42
sdakenot sure about ansible galera15:42
inc0it already has 2*+2, someone forgot to tag +w;)15:42
inc0I'll review it15:42
inc0sdake: what are steps to reproduce? just build stable/ocata and deploy?15:43
inc0with centos?15:43
sdakebuild stable/ocata15:43
sdakewatch for the error "package mariadb can't be installed"15:43
sdakeor somethign like that15:43
sdakepercona has a deprecation on mariadb-libs15:44
sdakeyum can't resolve with percona in the repo mix15:44
sdakethe solution is to exclude percona during mariadb galera install, then to install xtradb after galera is installed from rdo15:45
sdakextradbbackup is just a slew of shell scripts and stuff that get called externally15:45
sdakethe bad result is we can't tag 4.0.115:46
sdakeas 4.0.1 is unbuildable15:46
sdake(tag in dockerhub)15:47
*** manheim has joined #openstack-kolla15:52
*** manheim has quit IRC15:57
openstackgerritSteven Dake proposed openstack/kolla stable/ocata: Remove sudo operations that are no longer necessary  https://review.openstack.org/46865515:59
inc0sdake: I just ran pip install kolla and build16:05
inc0and worked16:06
sdakeit doesn't error16:06
sdakeit just doesn't work16:06
sdakeread the fine rpint :)16:06
inc0ok, so deploy will break right?16:06
sdakepip install kolla will install 4.0.116:06
sdakeyou need  to git clone - git checkout stable/ocata, sudo pip install .16:06
sdakethen build16:07
inc0well, if it's upstream change (repo messed up) that shouldn't matter16:07
inc0if it's something we did, we should revert that16:07
kbaegisDo either of you remember what I had done months back to solve this? https://hastebin.com/benoveteji.py16:07
kbaegisI had to disable qos or fwaas or something?16:08
sdakeinc0 https://da.gd/rqJmH -> https://paste.fedoraproject.org/paste/fwE5xdH4cJrGeDOIMZGDxF5M1UNdIGYhyRLivL9gydE=/16:12
sdakeINFO:kolla.image.build.mariadb:No package MariaDB-Galera-server available.16:13
sdakeMariaDB-Galera-server was renamed mariadb-server-galera16:13
sdakewhich is the same name of the same package in 3 different repositories16:14
ReepicheepI'm still having trouble with my override of interfaces in the inventory file, can have help verifying my settings?16:18
Reepicheephttp://paste.openstack.org/show/610788/16:18
Reepicheepit completes the bootstrap-servers and prechecks fine but is failing on ceph16:19
kbaegisReepicheep: Huh, curious. bond0/team0.821?16:19
Reepicheepon the two servers that don't have team0.821 as interface_storage16:19
Reepicheepall the other nodes use team0.821 so that is what is set in the globals16:20
kbaegisReepicheep: inc0 covered this yesterday, globals.yml overrides the node-specific. It has precedence16:20
kbaegisReepicheep: I don't like that, but it's how it's set up16:21
Reepicheephmm.  I read that backwards then yesterday16:21
*** caoyuan has quit IRC16:21
*** eaguilar has joined #openstack-kolla16:21
kbaegisReepicheep: I'd use vim and visual block edit to just copy and alter the config per set of nodes16:22
kbaegisBut unset it in globals.yml and set it per node in inventory16:22
inc0kbaegis: you can also create new group of nodes and use vars16:23
Reepicheepinterestingly it does use the interfaces file for some because the bootstrap-servers would not even run until I set each of the *_interfaces in the inventory file16:23
inc0kbaegis: http://docs.ansible.com/ansible/intro_inventory.html#groups-of-groups-and-group-variables16:23
inc0yeah, bootstrap servers needs api_interface to prep hostfile16:24
*** mewald has joined #openstack-kolla16:26
Reepicheepi guess yesterday i was thinking the issue was with I was just setting the network_interface in the inventory which is what the globals overwrote with it value for the api with 'api_interface: "{{ network_interface }}'16:27
kbaegisReepicheep: That would be intuitive, but it won't work16:28
inc0I think that if you set up network interface in globals and override api_interface in inventory that works16:28
kbaegisYeah, ideally it would follow the most-specific-config principle with overrides16:29
*** manheim has joined #openstack-kolla16:29
Reepicheepinc0: I believe that is how I have it set .. unless I'm missing something? http://paste.openstack.org/show/610789/16:32
sdakeinc0 did you see the problem in the past eabove16:33
inc0here's caveat16:33
inc0Reepicheep:16:33
*** manheim has quit IRC16:33
inc0globals override inventory, so you *can't* set up iface name in globals if you do it in inventory16:34
inc0I'd suggest setting all of interfaces in inventory16:34
inc0you don't need to do it inline16:35
inc0you can create host groups with grouped with same iface naming16:35
Reepicheepinc0 I think I got that now,  I will do that and learn to use the host groups ;)16:35
*** Pavo has joined #openstack-kolla16:35
inc0and then use [control:children] group1 kind of stuff16:35
Reepicheepsorry I miss understood the scope of that comment yesterday..16:36
inc0yeah it's hard to convey on irc I guess16:36
inc0and not really intuitive16:36
inc0anyhow, globals.yml can be entirely replaced with inventory if you know what you're doing16:36
inc0another thing - if you run kolla-ansible deploy16:37
inc0first line of output is ansible-playbook ... command16:37
inc0these are equivalent16:37
inc0so if you really want to modify how your ansible runs, just use ansible native CLI16:38
Reepicheepyeah.. and I have years of overwriting node variables in puppet where that is how it works.. the more precise the scope gets precedence16:38
*** Pavo has quit IRC16:38
inc0that's kind of case with ansible too16:38
inc0reason it's not like that is with kolla16:38
*** manheim has joined #openstack-kolla16:38
inc0we use -e @/etc/kolla/globals.yml16:38
*** Pavo has joined #openstack-kolla16:39
inc0as in - top level overrides provided manually16:39
inc0so it's our fault rather than ansible16:39
Reepicheepoh.. that is helpful and now I know why we call kolla-ansible instead of ansible directly16:39
inc0kolla-ansible is glorified alias16:40
Reepicheepsweet.. well at least for this issue I can quit pulling my hair out.. if I know what's going I can work around it ;)16:41
Reepicheepthanks again16:41
*** manheim has quit IRC16:43
inc0sdake: I'll work on it later today16:43
sdakeinc0 i already have a patch up16:43
sdakeinc0 although could use multinode testing16:43
inc0well I want to see the issue for myself, it's important16:43
inc0can do that16:44
inc0and yes, we need multinode testing for that16:44
sdakei've spent about 12 hours working on it thus far16:44
sdakei dont think it is necessary to duplicate that effort16:45
sdakewhat woudl help is if someone else could   do multinode testing - as I don't have a way to do that16:45
inc0I'll do multinode testing16:46
sdakehttps://review.openstack.org/#/c/468632/16:46
sdakecool16:46
*** openstackgerrit has quit IRC16:47
*** openstackgerrit has joined #openstack-kolla16:48
openstackgerritMerged openstack/kolla master: Remove sudo operations that are no longer necessary  https://review.openstack.org/46860716:48
sdakeJeffrey4l around?16:51
sdakeJeffrey4l inc0 can you ack this backport please: https://review.openstack.org/46865516:51
*** manheim has joined #openstack-kolla16:52
*** jascott1 has quit IRC16:56
*** jascott1 has joined #openstack-kolla16:56
*** jascott1 has quit IRC17:01
*** manheim has quit IRC17:01
Reepicheepkbaegis: inc0 I now have no interfaces defined in globals.yml and two host groups [team0] and [bond0] to allow me to use the two odd ball nodes that I'm using a "bond" interface on instead of a "team" to do link aggregation.. and I'm holding my tongue in that right place .. so it's going to work17:09
*** manheim has joined #openstack-kolla17:10
openstackgerritSteven Dake proposed openstack/kolla master: Consume centos-pike-testing mariadb version  https://review.openstack.org/46863217:12
*** manheim has quit IRC17:15
*** manheim has joined #openstack-kolla17:24
*** salv-orlando has joined #openstack-kolla17:32
*** manheim has quit IRC17:33
*** salv-orlando has quit IRC17:37
*** manheim has joined #openstack-kolla17:38
*** manheim has quit IRC17:42
*** manheim has joined #openstack-kolla17:56
kbaegisReepicheep: Sure. There are also a lot of bond/teamed packet schedulers worth investigating17:57
*** eaguilar has quit IRC17:58
*** manheim has quit IRC18:00
ReepicheepI like to use the teamed interfaces over the old bonded interfaces but of course I'm having an issue with two old nodes so it's simpler for me just to leave them as bonded interfaces18:02
ReepicheepI probably could figure out how to make it work on the system level and make it use team0 as the interface but I really need to understand the ansible magic to over ride stuff from the inventories so I am attempting to solve it there18:04
Reepicheepbtw, my trick of holding my tongue just right didn't work so I'm still learning :|18:05
*** jamesPR has joined #openstack-kolla18:05
*** Pavo has quit IRC18:12
*** manheim has joined #openstack-kolla18:14
*** manheim has quit IRC18:18
kbaegisReepicheep: I'm a fan of lacp, but there are good alternatives. round robin is perfectly acceptable18:20
kbaegisI only have a 2 node deployment :)18:20
kbaegisWell, technically 4, but two of those are atom based routers18:21
*** kbyrne has quit IRC18:22
*** Pavo has joined #openstack-kolla18:31
*** manheim has joined #openstack-kolla18:32
*** salv-orlando has joined #openstack-kolla18:33
kbaegisCan you specify more than one virt_type  in nova?18:35
*** Pavo has quit IRC18:35
*** manheim has quit IRC18:36
*** salv-orlando has quit IRC18:38
*** Pavo has joined #openstack-kolla18:38
*** manheim has joined #openstack-kolla18:50
*** manheim has quit IRC18:55
Reepicheepkbaegis: part of my issue is probably my own complexity.. my aggregated interfaces are actually mlag to a pair of tor switches which does add great resistance but at the expense of complexity19:09
*** manheim has joined #openstack-kolla19:09
Reepicheepespecially I have heterogeneous nodes19:09
ReepicheepIt's always nice to have consistent hardware.. but sometimes we don't get that option ;)19:10
kbaegisReepicheep: Best way to do it in a lab environment :) Get to learn more19:12
kbaegisBut then, I'm just the crazy gentoo guy19:13
*** manheim has quit IRC19:13
kbaegisomg! The ovs firewall works, even though the poorly scripted hybrid iptables/ovs one doesn't :DD19:15
*** kbaegis has quit IRC19:17
*** manheim has joined #openstack-kolla19:27
*** kbyrne has joined #openstack-kolla19:30
*** manheim has quit IRC19:31
*** salv-orlando has joined #openstack-kolla19:34
*** iniazi has quit IRC19:37
*** glycerine has joined #openstack-kolla19:37
*** jamesPR has quit IRC19:38
*** salv-orlando has quit IRC19:39
openstackgerritSteven Dake proposed openstack/kolla master: Consume centos-pike-testing mariadb version  https://review.openstack.org/46863219:39
*** pcaruana has quit IRC19:40
*** manheim has joined #openstack-kolla19:45
*** powerd has joined #openstack-kolla19:49
*** manheim has quit IRC19:50
*** iniazi has joined #openstack-kolla19:52
*** manheim has joined #openstack-kolla20:04
*** manheim has quit IRC20:08
sdakeinc0 kfox1111 sbezverk_ can one of you take a look at this easy review please: https://review.openstack.org/#/c/468335/220:12
openstackgerritSteven Dake proposed openstack/kolla-kubernetes master: Conditionalize pv creation  https://review.openstack.org/46826420:14
sdakeand also that easy review ^^20:14
sdakeTIA20:14
openstackgerritSteven Dake proposed openstack/kolla-kubernetes master: Conditionalize pv creation  https://review.openstack.org/46826420:16
openstackgerritMerged openstack/kolla-kubernetes master: Add option to enable ironic inspector LLDP collection  https://review.openstack.org/46833520:20
*** kbaegis has joined #openstack-kolla20:21
*** manheim has joined #openstack-kolla20:22
openstackgerritSteven Dake proposed openstack/kolla master: Consume centos-pike-testing mariadb version  https://review.openstack.org/46863220:24
*** manheim has quit IRC20:26
sdakekfox1111 re search path - i can do that - not sure what you mean by it tho20:28
openstackgerritMerged openstack/kolla-kubernetes master: Script to Deploy Kolla Kubernetes AIO  https://review.openstack.org/46431220:29
kolla-slack1<kfox1111> Related to my earlier comment. Make it overridable at the more individual levels too. See the helm docs on conditional reqs20:30
*** manheim has joined #openstack-kolla20:35
sdakekfox1111 thanks20:36
kbaegisAnyone know what services collide with iscsid?20:38
kbaegisgetting iscsid: can not create NETLINK_ISCSI socket from the iscsi container20:38
openstackgerritSteven Dake proposed openstack/kolla-kubernetes master: Simple configmap to raw kubectl  https://review.openstack.org/44926220:38
*** manheim has quit IRC20:40
openstackgerritMerged openstack/kolla-kubernetes master: Remove useless modules in ansible/library  https://review.openstack.org/45066220:42
openstackgerritSteven Dake proposed openstack/kolla-kubernetes master: Remove pathfinder from call path of secret-generator  https://review.openstack.org/46427320:42
kbaegisNot typically an issue, but when I create a volume from an image I get ImageCopyFailure: Failed to copy image to volume: Could not login to any iSCSI portal20:49
sdakeJeffrey4l any idea what triggers this ansible error: http://logs.openstack.org/31/464031/40/check/gate-kolla-kubernetes-deploy-centos-source-4-ironic-nv/55495a1/console.html in review: https://review.openstack.org/#/c/464031/4020:49
sdakekbaegis no idea what would cause that - other then you may have iscsi running on your host20:50
sdakekbaegis or iscsi is not enabled in your kernel is another possibility20:50
kbaegissdake: I'll double check the kernel20:54
kbaegis"iscsid: can not create NETLINK_ISCSI socket"20:54
*** manheim has joined #openstack-kolla20:54
kbaegisYup, good call sdake!20:54
kbaegisTARGET_CORE was hiding the setting from me20:55
*** jamesPR has joined #openstack-kolla20:56
*** eaguilar has joined #openstack-kolla20:58
*** caoyuan has joined #openstack-kolla20:58
*** caoyuan has quit IRC20:58
*** manheim has quit IRC20:59
*** manheim has joined #openstack-kolla21:07
sdakekbaegis so it was a lack of kernel config then?21:07
kbaegissdake: Yeah, one of the things necessary for gentoo is to document all the necessary kernel flags. iscsid was the last one21:10
kbaegisHere's the config I have working on gentoo: https://hastebin.com/gaxosevuyu.coffeescript21:11
kbaegisTurns out it still doesn't like it when I create a volume from image on an encrypted drive21:12
kbaegisNot working on my setup. But they work independently, copy from image and encrypt21:12
kbaegisAlso, the hybridovs driver decidedly doesn't work. Don't know why21:12
openstackgerritSteven Dake proposed openstack/kolla-kubernetes master: Add a destroy workflow for kolla-kubernetes  https://review.openstack.org/46403121:17
openstackgerritSteven Dake proposed openstack/kolla-kubernetes master: Remove pin on ansible<2.1  https://review.openstack.org/46867221:17
openstackgerritSteven Dake proposed openstack/kolla-kubernetes master: Add a destroy workflow for kolla-kubernetes  https://review.openstack.org/46403121:25
*** powerd has quit IRC21:28
*** mewald has quit IRC21:31
*** kbyrne has quit IRC21:34
*** Pavo has quit IRC21:37
Reepicheepkbaegis: I would not call you the crazy gentoo guy.. I can relate ;)21:39
*** manheim has quit IRC21:40
ReepicheepI used to strip everything out of my kernels except the bare minimum but then I found my self recompiling the kernel every time I wanted stuff like iSCSI ;)21:40
*** manheim has joined #openstack-kolla21:41
*** eaguilar has quit IRC21:42
ReepicheepI finally got tired of it and decided to take the route of "you know those Red Hat guys, they probably know what they are doing, they may have most of the kitchen in there but at least it doesn't include the kitchen sink"21:42
*** kbyrne has joined #openstack-kolla21:43
*** Pavo has joined #openstack-kolla21:44
Reepicheepbut when people ask what my favorite distro is after I say "that depends" usually Gentoo is near the top of the list.. using gentoo always seems to may me happy21:45
*** manheim has quit IRC21:46
Reepicheepand because you have shared .. when I get time I will try it.. so thanks ;)21:47
*** Pavo has quit IRC21:53
*** manheim has joined #openstack-kolla21:53
*** manheim has quit IRC21:56
*** manheim has joined #openstack-kolla21:56
sdakekfox1111 quick Q - trying to get a list of all the pvc names in the kolla namespace and using this:21:57
sdakekubectl get pvc -n kolla -o jsonpath={.metadata.name}21:57
sdakeError executing template: name is not found. Printing more information for debugging the template:21:57
sdakehwoever kuectl get pvc -n kolla -o yaml prints em out21:57
*** rcrit has quit IRC21:58
*** manheim has quit IRC21:58
*** manheim has joined #openstack-kolla21:59
sbezverk_sdake:check carefully returned json22:00
sbezverk_it gives you back list22:01
sbezverk_and you try to access individual element of item of that list22:01
sdakeis there a way to print out al items in the lsit?22:01
*** powerd has joined #openstack-kolla22:01
sdakerelated to that jsonpath?22:02
*** kbyrne has quit IRC22:03
sbezverk_sdake: kubectl get pvc -n kolla -o jsonpath={.items[*].metadata.name}22:05
sdakethanks i'll try that22:05
openstackgerritSteven Dake proposed openstack/kolla-kubernetes master: Remove pin on ansible<2.1  https://review.openstack.org/46867222:15
openstackgerritSteven Dake proposed openstack/kolla-kubernetes master: Add a destroy workflow for kolla-kubernetes  https://review.openstack.org/46403122:15
sdakesbezverk_ thanks dude !22:16
*** manheim has quit IRC22:21
openstackgerritSteven Dake proposed openstack/kolla-kubernetes master: Add a destroy workflow for kolla-kubernetes  https://review.openstack.org/46403122:23
kbaegisReepicheep: :) Well, I'll be writing a community guide to kolla on gentoo22:25
kbaegisReepicheep: kolla-ansible only so far. I'll let you know if I get around to kolla-mesos or kolla-k8s22:25
kbaegisSeen zero decent guides for k8s on gentoo22:26
*** manheim has joined #openstack-kolla22:26
kbaegisReally makes more sense for 5+ node clusters22:26
kbaegisimo22:26
Reepicheepkbaegis: lol.. since it will be there and it's gentoo and kolla I will probably end up trying just so I can say I tried22:29
Reepicheepthanks for eating up my future time ;)22:30
kbaegisReepicheep: That's what gentoo is good for! ;)22:30
*** manheim has quit IRC22:31
*** powerd has quit IRC22:32
*** powerd has joined #openstack-kolla22:35
jamesPRnice to see ppl using gentoo ;) was my second linux OS back in 200122:36
*** salv-orlando has joined #openstack-kolla22:36
*** salv-orlando has quit IRC22:40
*** glycerine has quit IRC22:51
*** SaMnCo has quit IRC22:54
*** SaMnCo has joined #openstack-kolla22:54
*** ediardo has quit IRC22:54
*** nanliu has quit IRC22:54
*** harmw_ has quit IRC22:55
*** wirehead_ has quit IRC22:55
*** nanliu has joined #openstack-kolla22:56
*** ediardo has joined #openstack-kolla22:57
*** dgonzalez has quit IRC22:57
*** harmw has joined #openstack-kolla22:58
*** kbaegis has quit IRC22:58
*** caoyuan has joined #openstack-kolla23:00
*** dgonzalez has joined #openstack-kolla23:00
*** wirehead_ has joined #openstack-kolla23:01
*** manheim has joined #openstack-kolla23:02
*** manheim has quit IRC23:06
*** jamesPR has quit IRC23:12
*** caoyuan has quit IRC23:16
openstackgerritSteven Dake proposed openstack/kolla-kubernetes master: Add a destroy workflow for kolla-kubernetes  https://review.openstack.org/46403123:19
*** kbaegis has joined #openstack-kolla23:23
*** kbyrne has joined #openstack-kolla23:25
sdakekfox1111 what do you maek of htis: http://logs.openstack.org/72/468672/2/check/gate-kolla-kubernetes-deploy-centos-binary-2-helm-entrypoint-nv/4a0e541/console.html#_2017-05-27_22_38_10_49015123:27
sdakeit looks to me like there is a new cloud infra 2017-05-27 22:16:04.075501 | [Zuul] Building remotely on centos-7-citycloud-sto2-9023263 in workspace /home/jenkins/workspace/gate-kolla-kubernetes-deploy-centos-binary-2-helm-entrypoint-nv23:27
sdakei'm curious if it works with our gating23:27
*** powerd has quit IRC23:33
*** salv-orlando has joined #openstack-kolla23:37
*** manheim has joined #openstack-kolla23:38
*** salv-orlando has quit IRC23:42
*** manheim has quit IRC23:42
sdakegate implodes even on a gatecheck23:52
sdakehttp://logs.openstack.org/80/422380/1/check/gate-kolla-kubernetes-deploy-centos-binary-2-ceph-nv/7023dc3/console.html#_2017-05-27_20_58_07_04965523:52
sdakeselenium among other things23:53
kbaegisanyone else having issues with murano?23:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!