Monday, 2017-09-18

*** manheim has quit IRC00:03
*** magicboiz has quit IRC00:03
*** magicboiz has joined #openstack-kolla00:04
*** jemcevoy has quit IRC00:07
*** huzhengchuan has quit IRC00:14
*** lrensing has joined #openstack-kolla00:28
*** yangyapeng has quit IRC00:29
*** yangyapeng has joined #openstack-kolla00:29
*** Long has quit IRC00:33
*** yangyapeng has quit IRC00:34
*** yangyapeng has joined #openstack-kolla00:41
*** yangyapeng has quit IRC00:46
*** tovin07_ has joined #openstack-kolla00:49
*** hieulq has joined #openstack-kolla00:50
*** caowei has joined #openstack-kolla00:53
*** magicboiz has quit IRC00:53
*** magicboiz has joined #openstack-kolla00:54
*** yangyapeng has joined #openstack-kolla00:58
*** yangyapeng has quit IRC01:03
*** yangyapeng has joined #openstack-kolla01:10
*** huzhengchuan has joined #openstack-kolla01:11
*** itlinux has joined #openstack-kolla01:18
*** yangyapeng has quit IRC01:19
*** magicboiz has quit IRC01:24
*** magicboiz has joined #openstack-kolla01:24
*** itlinux has quit IRC01:30
*** lrensing has quit IRC01:41
*** yangyapeng has joined #openstack-kolla01:51
openstackgerritzhongshengping proposed openstack/kolla-ansible master: Deprecate bind_host , bind_port and api_handler options in octavia  https://review.openstack.org/50398201:52
*** yangyapeng has quit IRC01:52
*** yangyape_ has joined #openstack-kolla01:52
*** itlinux has joined #openstack-kolla01:54
*** dave-mccowan has joined #openstack-kolla01:55
*** dave-mcc_ has joined #openstack-kolla01:58
*** dave-mccowan has quit IRC02:00
*** yangyape_ has quit IRC02:02
*** yangyapeng has joined #openstack-kolla02:02
openstackgerritDebo Zhang proposed openstack/kolla-kubernetes master: Update links in README  https://review.openstack.org/49036602:04
*** yangyapeng has quit IRC02:07
*** yangyapeng has joined #openstack-kolla02:07
*** yangyapeng has quit IRC02:08
*** yangyapeng has joined #openstack-kolla02:08
*** yangyapeng has quit IRC02:09
*** yangyapeng has joined #openstack-kolla02:09
*** yangyapeng has quit IRC02:10
*** yangyape_ has joined #openstack-kolla02:10
*** jmorgan1_ has quit IRC02:11
*** jmorgan1 has joined #openstack-kolla02:11
*** yangyapeng has joined #openstack-kolla02:12
*** yangyape_ has quit IRC02:12
*** zhangfei has joined #openstack-kolla02:13
*** zhangfei has quit IRC02:20
*** schwicht has quit IRC02:29
*** zhangfei has joined #openstack-kolla02:33
*** dave-mcc_ has quit IRC02:39
*** zhubingbing has joined #openstack-kolla02:44
*** virus_ has quit IRC02:49
*** yangyape_ has joined #openstack-kolla02:51
*** yangyapeng has quit IRC02:51
*** magicboiz has quit IRC02:56
*** zhubingbing has quit IRC02:56
*** magicboiz has joined #openstack-kolla02:56
*** zhubingbing has joined #openstack-kolla02:56
*** spsurya has joined #openstack-kolla03:00
*** jianghuaw has joined #openstack-kolla03:01
spsuryamorning guys03:15
*** virus_ has joined #openstack-kolla03:20
*** yangyapeng has joined #openstack-kolla03:36
*** yangyape_ has quit IRC03:36
*** magicboiz has quit IRC03:42
*** magicboiz has joined #openstack-kolla03:43
*** mtaylor22 has joined #openstack-kolla03:47
*** virus_ has quit IRC04:04
*** janki has joined #openstack-kolla04:19
openstackgerritcaoyuan proposed openstack/kolla-ansible master: Make the pg and pgp value configurable  https://review.openstack.org/50484504:28
*** yangyapeng has quit IRC04:30
*** yangyapeng has joined #openstack-kolla04:30
*** yangyapeng has quit IRC04:32
*** yangyapeng has joined #openstack-kolla04:33
*** yangyapeng has quit IRC04:37
iniazihello.  anyone see any issues building packages recently?  sources were bulding fine a few weeks ago, but I tried couple of days ago, and many services aren't starting up.  libvirt, kuryr04:38
iniazithis is with 4.0.2/ocata on centos.  recently I've tried source build and binary builds (but binary builds, require multple packages to be built in source, and then have to override some variables... and that one i think libvirt wasn't working at the end.04:39
iniazii'm trying to use a different server to build on, but shouldn't matter, as I think it uses a standard centos image to build the packages, so shouldn't matter.  and atleast source wise, I think I'm getting the same errors.  https://pastebin.com/7kLMG00U04:40
iniaziThe pastebin, https://pastebin.com/7kLMG00U, i see some error on what may be cert related.  It looks related to the error I got on libvirt container on deploy: error : daemonSetupNetworking:554 : unsupported configuration: No server certificate path set to match server key04:42
*** itlinux has quit IRC04:48
*** caowei has quit IRC04:53
*** skramaja has joined #openstack-kolla04:56
*** yangyapeng has joined #openstack-kolla05:01
*** yangyapeng has quit IRC05:01
*** yangyapeng has joined #openstack-kolla05:02
*** yangyapeng has quit IRC05:02
*** yangyapeng has joined #openstack-kolla05:03
*** yangyapeng has quit IRC05:03
*** yangyapeng has joined #openstack-kolla05:04
*** yangyapeng has quit IRC05:04
*** yangyapeng has joined #openstack-kolla05:05
*** yangyapeng has quit IRC05:05
*** yangyapeng has joined #openstack-kolla05:06
*** yangyapeng has quit IRC05:06
*** yangyapeng has joined #openstack-kolla05:07
*** yangyapeng has quit IRC05:07
*** yangyapeng has joined #openstack-kolla05:08
*** yangyapeng has quit IRC05:13
*** caowei has joined #openstack-kolla05:21
*** jaosorior has joined #openstack-kolla05:22
*** yangyapeng has joined #openstack-kolla05:22
*** jbadiapa has joined #openstack-kolla05:23
*** yangyapeng has quit IRC05:27
*** david-lyle has quit IRC05:30
*** yangyapeng has joined #openstack-kolla05:31
*** yangyapeng has quit IRC05:34
*** yangyapeng has joined #openstack-kolla05:35
*** yangyapeng has quit IRC05:35
*** yangyapeng has joined #openstack-kolla05:35
*** yangyapeng has quit IRC05:36
*** hoaile has left #openstack-kolla05:39
*** lpetrut has joined #openstack-kolla05:39
*** lpetrut has quit IRC05:40
*** yangyape_ has joined #openstack-kolla05:40
*** yangyape_ has quit IRC05:44
*** yangyapeng has joined #openstack-kolla05:44
*** yangyapeng has quit IRC05:44
*** mdnadeem has joined #openstack-kolla05:45
*** yangyapeng has joined #openstack-kolla05:51
*** yangyapeng has quit IRC05:52
*** yangyapeng has joined #openstack-kolla05:52
*** yangyapeng has quit IRC05:52
*** yangyapeng has joined #openstack-kolla05:53
*** yangyapeng has quit IRC05:53
*** yangyapeng has joined #openstack-kolla05:54
*** yangyapeng has quit IRC05:54
*** yangyapeng has joined #openstack-kolla05:55
*** yangyapeng has quit IRC06:00
*** yangyapeng has joined #openstack-kolla06:00
*** yangyapeng has quit IRC06:00
*** yangyapeng has joined #openstack-kolla06:01
*** yangyapeng has quit IRC06:02
*** hoaile has joined #openstack-kolla06:02
*** yangyapeng has joined #openstack-kolla06:02
*** yangyapeng has quit IRC06:02
*** yangyapeng has joined #openstack-kolla06:04
*** yangyapeng has quit IRC06:04
*** yangyapeng has joined #openstack-kolla06:04
*** yangyapeng has quit IRC06:05
*** yangyapeng has joined #openstack-kolla06:06
*** magicboiz has quit IRC06:06
*** magicboiz has joined #openstack-kolla06:06
*** yangyapeng has quit IRC06:06
*** yangyapeng has joined #openstack-kolla06:07
*** yangyapeng has quit IRC06:07
*** yangyapeng has joined #openstack-kolla06:08
*** hoaile has quit IRC06:09
*** yangyapeng has quit IRC06:09
*** hoaile has joined #openstack-kolla06:09
*** yangyapeng has joined #openstack-kolla06:10
*** yangyapeng has quit IRC06:12
*** yangyape_ has joined #openstack-kolla06:12
*** magicboiz has quit IRC06:13
*** yangyape_ has quit IRC06:14
*** rwsu has joined #openstack-kolla06:14
*** yangyapeng has joined #openstack-kolla06:14
*** yangyapeng has quit IRC06:15
*** yangyapeng has joined #openstack-kolla06:15
*** yangyapeng has quit IRC06:16
*** yangyapeng has joined #openstack-kolla06:17
*** yangyapeng has quit IRC06:17
*** yangyapeng has joined #openstack-kolla06:18
*** yangyapeng has quit IRC06:18
*** yangyapeng has joined #openstack-kolla06:19
*** yangyapeng has quit IRC06:19
*** yangyapeng has joined #openstack-kolla06:20
*** yangyapeng has quit IRC06:20
*** manheim has joined #openstack-kolla06:20
*** yangyapeng has joined #openstack-kolla06:20
*** yangyapeng has quit IRC06:21
*** yangyapeng has joined #openstack-kolla06:22
*** manheim has quit IRC06:24
*** kollian has joined #openstack-kolla06:37
*** vharsh has joined #openstack-kolla06:38
*** manheim has joined #openstack-kolla06:40
*** yangyapeng has quit IRC06:42
*** kollian_ has joined #openstack-kolla06:42
*** yangyapeng has joined #openstack-kolla06:42
*** yangyapeng has quit IRC06:42
*** yangyapeng has joined #openstack-kolla06:43
*** yangyapeng has quit IRC06:43
*** kollian has quit IRC06:44
*** yangyapeng has joined #openstack-kolla06:44
*** yangyapeng has quit IRC06:44
*** manheim has quit IRC06:45
*** yangyapeng has joined #openstack-kolla06:45
kollian_iniazi: http://paste.openstack.org/show/621305/06:46
kollian_have you faced this before ?06:46
*** mnasiadka has joined #openstack-kolla06:50
*** shardy_afk is now known as shardy06:52
openstackgerritMatthias Runge proposed openstack/kolla master: Add more collectd plugins to the container  https://review.openstack.org/50295606:53
*** yangyapeng has quit IRC06:53
*** yangyapeng has joined #openstack-kolla06:54
*** yangyapeng has quit IRC06:54
*** yangyapeng has joined #openstack-kolla06:55
*** gaoyan has joined #openstack-kolla06:56
*** gaoyan has quit IRC06:56
*** gaoyan has joined #openstack-kolla06:56
*** gaoyan_ has joined #openstack-kolla07:00
*** gaoyan_ has quit IRC07:00
*** jmccarthy has joined #openstack-kolla07:01
*** yangyapeng has quit IRC07:02
*** yangyapeng has joined #openstack-kolla07:02
*** yangyapeng has quit IRC07:02
*** gaoyan has quit IRC07:03
*** yangyapeng has joined #openstack-kolla07:03
*** yangyapeng has quit IRC07:03
*** yangyapeng has joined #openstack-kolla07:04
*** yangyapeng has quit IRC07:04
openstackgerritMerged openstack/kolla-ansible master: Add missing document start "---" to yaml files  https://review.openstack.org/50290607:05
*** yangyapeng has joined #openstack-kolla07:05
*** kollian_ has quit IRC07:07
*** yangyapeng has quit IRC07:08
*** mewald has joined #openstack-kolla07:08
*** yangyapeng has joined #openstack-kolla07:09
*** yangyapeng has quit IRC07:09
mewaldmorning07:09
*** yangyapeng has joined #openstack-kolla07:10
*** yangyapeng has quit IRC07:10
*** yangyapeng has joined #openstack-kolla07:11
*** athomas has joined #openstack-kolla07:13
*** gaoyan has joined #openstack-kolla07:13
*** yangyapeng has quit IRC07:15
*** serlex has joined #openstack-kolla07:17
*** gaoyan has quit IRC07:18
*** gaoyan has joined #openstack-kolla07:19
*** pcaruana has joined #openstack-kolla07:21
openstackgerritEduardo Gonzalez proposed openstack/kolla master: Fix dind image build and oraclelinux build  https://review.openstack.org/50371207:23
*** yangyapeng has joined #openstack-kolla07:24
*** gaoyan has quit IRC07:24
openstackgerritEduardo Gonzalez proposed openstack/kolla master: Fix dind image build and oraclelinux build  https://review.openstack.org/50371207:26
*** yangyapeng has quit IRC07:26
*** shardy is now known as shardy_afk07:28
*** magicboiz has joined #openstack-kolla07:29
*** magicboiz has quit IRC07:36
*** robbbe has quit IRC07:37
*** janki is now known as janki|lunch07:42
*** magicboiz has joined #openstack-kolla07:43
*** kollian has joined #openstack-kolla07:48
*** gaoyan has joined #openstack-kolla07:48
kolliananyone http://paste.openstack.org/show/621305/07:49
kollianjust finding this error07:49
kollianin image build07:49
kollian^^07:49
*** gaoyan has quit IRC07:49
kollianiniazi: ??07:49
*** gaoyan has joined #openstack-kolla07:52
*** shardy_afk is now known as shardy07:53
*** gaoyan_ has joined #openstack-kolla07:53
*** janki|lunch is now known as janki07:55
*** gaoyan_ has quit IRC07:56
*** gaoyan has quit IRC07:56
*** gaoyan has joined #openstack-kolla07:56
*** gaoyan is now known as gaoyanami08:01
*** gaoyanami has quit IRC08:01
*** gaoyan has joined #openstack-kolla08:01
*** gaoyan is now known as gaoyanami08:02
*** gaoyanami has quit IRC08:03
*** gaoyanami has joined #openstack-kolla08:03
openstackgerritMatthias Runge proposed openstack/kolla master: Add more collectd plugins to the container  https://review.openstack.org/50295608:03
vharshkollian, I wish it was more readable.08:05
*** shardy is now known as shardy_afk08:05
*** gaoyan has joined #openstack-kolla08:06
kollianvharsh: do you figure out the problem08:06
kollian?08:06
vharshkollian, Nope08:06
vharshkollian, I faced a similar problem08:07
kollianvharsh: np08:07
*** mewald has left #openstack-kolla08:07
vharshkollian, I am a newbie.08:07
kollianvharsh: same here08:07
vharshkollian, I was trying to follow kolla's quickstart setup.08:07
kolliani am also following the same08:07
kollianhttps://docs.openstack.org/project-deploy-guide/kolla-ansible/pike/quickstart.html08:08
vharshkollian, And in the globals.yaml file it was configured to get the `auto` build tag.08:08
kollianvharsh:08:08
vharshkollian, Change that to 4.0.0 or something else.08:08
kollianvharsh: i am trying with master08:09
vharshkollian, Check if it exists here https://hub.docker.com/u/kolla/08:09
vharshkollian, I recently finished doing that, where(which step) are you stuck btw?08:09
kolliani am trying again08:10
kolliani was stuck at http://paste.openstack.org/show/621305/08:10
vharshkollian, It worked for me after I set, `openstack_release: "4.0.0"`08:11
vharsh* in globals.yaml08:11
kollianvharsh: ok i will try08:11
kollianthanks08:11
kollianlets see08:11
vharshkollian, I had gotten the binaries :| So I don't know much about building it from source.08:12
*** egonzalez has joined #openstack-kolla08:12
egonzalezmorning o/08:12
vharshkollian, It is set to `#kolla_install_type: "binary"`08:12
kollianvharsh: `kolla-build main -b centos -t source`08:12
kollianvharsh: i am running this08:13
kollianfor source installation08:13
vharshDid you pass the prechecks?08:13
kolliani think bulding the image need to be run before precheck08:14
vharshkollian, Did you do `kolla-ansible pull` ?08:14
kollianvharsh: i am in kolla part not reached to kolla-ansible part08:14
*** zhubingbing_ has joined #openstack-kolla08:15
*** gaoyan_ has joined #openstack-kolla08:16
*** gaoyan_ has quit IRC08:17
*** zhubingbing has quit IRC08:18
*** vharsh has quit IRC08:18
jmccarthyegonzalez: Hrm I'm failing with reno .. how is the xml file meant to be laid out ?08:24
openstackgerritEduardo Gonzalez proposed openstack/kolla master: Fix dind and zun images and oraclelinux build  https://review.openstack.org/50371208:25
egonzalezjmccarthy, is rst iirc08:25
egonzalezcan I get reviews in https://review.openstack.org/50371208:26
egonzalezfixes many build errors in one commit08:26
*** gaoyan has quit IRC08:29
*** gaoyan has joined #openstack-kolla08:30
*** vharsh has joined #openstack-kolla08:31
*** gaoyan has quit IRC08:34
*** gaoyan has joined #openstack-kolla08:34
*** gaoyan has quit IRC08:34
openstackgerritLong Zhang proposed openstack/kolla-ansible master: Add supplement in high-availability.rst about cinder  https://review.openstack.org/50488008:34
*** gaoyan has joined #openstack-kolla08:35
*** manheim has joined #openstack-kolla08:37
*** shardy_afk is now known as shardy08:40
*** hoaile is now known as fuegoel08:47
fuegoelhey guys, I hit this error ERROR (IOError): Could not find a suitable TLS CA certificate bundle, invalid path when trying to get the self-signed certs work. Any idea?08:50
manheimhi guys08:51
manheimPTG over?08:51
spsuryamanheim: yes08:53
manheimnice, how was it?08:54
serlexAny links to etherpads or conclusions or something similar?08:54
*** manheim is now known as dardelean08:54
dardeleanegonzalez, will the hyperv pathches be present on pike/stable also?08:55
spsuryadardelean serlex https://etherpad.openstack.org/p/kolla-queens-ptg-planning08:56
egonzalezdardelean, pike doesnt work without the change?08:56
dardeleannope08:56
dardeleani thought pike was going to be released this week for some reason so I was a bit late with them08:57
egonzalezdardelean, you got the answer then ;) , i triaged into pike too08:57
dardeleankolla-ansible stable/pike I mean08:57
serlexthanks!08:57
dardeleanI sent one also 2 weeks ago but cannot find it in pike08:58
dardeleanhttps://review.openstack.org/#/c/500835/08:58
egonzalezdardelean, backport should be done once master change is merged, it can differs from merged version08:59
dardeleanI see08:59
dardeleananything I have to do?08:59
egonzalezdardelean, sorry, is a cherrypick of a different change, though was the new fix09:00
vharshegonzalez, How can I start looking at the failed logs of some tests?09:03
egonzalezvharsh, in gates?09:03
vharshegonzalez, I see many tests failed here https://review.openstack.org/#/c/503693/409:03
egonzalezvharsh, some things are broken in gates now :(09:04
vharshegonzalez, Sorry, what are gates?09:04
vharshOh, gate-tests :|09:04
vharshI am an openstack newbie, can I try fixing some ?09:04
egonzalezbuilding images fails due some changes in other projects, and deploy fails for some libvirt error (no idea why, but i think will be fixed once image fix is merged)09:05
spsuryavharsh: https://docs.openstack.org/infra/system-config/devstack-gate.html09:05
vharshThanks spsurya :) I'll take a look09:05
egonzalezvharsh, fix for the image building https://review.openstack.org/#/c/503712/09:05
egonzalezvharsh, for logs, you can check current gate-tests at http://status.openstack.org/zuul/ , here can search for change ID09:06
vharshThanks egonzalez :)09:07
vharshThis was my first Pull Request to openstack.09:07
*** vkhanna has quit IRC09:08
*** vkhanna has joined #openstack-kolla09:08
* vharsh will try to learn to see diffs on Gerrit09:08
egonzalezvharsh, once gates finish, logs for build in logs/build/ of each job, but better look for what images failed in console.log(only CRITICAL: failures http://logs.openstack.org/56/502956/3/check/gate-kolla-dsvm-build-centos-source-centos-7/984bcfb/console.html#_2017-09-18_08_52_51_924168)09:09
vharshegonzalez, So you mean, I should look out for *CRITICAL* before *ERROR*09:10
egonzalezvharsh, for deploy jobs, can see logs in console.log, logs/ansible (ansible playbook executions), docker_logs (container restarting/failures),  and kolla (openstack logs)09:11
egonzalezvharsh, yes, many images are expected to fail, only critical are images that are not expected09:11
openstackgerritBertrand Lallau proposed openstack/kolla-ansible master: Fluentd: reduce CPU and I/O consumption  https://review.openstack.org/50186409:11
*** mgoddard has joined #openstack-kolla09:11
*** igordc_ has quit IRC09:13
*** dardelea_ has joined #openstack-kolla09:30
*** dardelean has quit IRC09:33
*** magicboiz has quit IRC09:43
*** magicboiz has joined #openstack-kolla09:49
*** dgonzalez has left #openstack-kolla09:49
*** magicboiz has quit IRC09:53
*** gaoyanami has quit IRC09:57
*** magicboiz has joined #openstack-kolla10:00
kollianegonzalez: http://paste.openstack.org/show/621305/10:01
kollianwhat is the problem ?10:01
kolliancan you please help10:01
kollianmy globbal.rml is http://paste.openstack.org/show/621320/10:03
kollianglobal.yml*10:03
egonzalezkollian, whats the main in the command for? just tools/build.py -t source -b centos is enough10:03
egonzalezkollian, globals do nothing during build phase, only to deploy10:04
egonzalezbuild reads kolla-build.conf but needs to be generated10:04
*** tovin07_ has quit IRC10:04
kollianegonzalez: yes did the same10:04
kolliani wanted to build only main images10:05
kollianso i fire the command10:05
kolliansudo tools/build.py main -t source -b centos10:05
kollianis it correct ?egonzalez10:05
egonzalezkollian,  tools/build.py  -t source -b centos -p main10:06
egonzalezto build main need to set -p (--profile) to use main10:06
kollian egonzalez thanks10:06
egonzalezkollian, but that dont build infra images (rabbitmq, mariadb) better use gate profile -p gate10:07
kollianegonzalez: i am trying set up dev env for kolla10:07
kollianwhat can be the best way to do10:07
kollian?10:08
egonzalezkollian, at least will need databases and rabbitmq, I would use gate or default profiles10:08
*** fengyc has joined #openstack-kolla10:09
egonzalezkollian, those are the images build by each profile https://github.com/openstack/kolla/blob/master/kolla/common/config.py#L133,#L17110:09
egonzalezchoose the one that fits better to your env10:09
*** fengyc has quit IRC10:10
kollianegonzalez: thanks10:12
kollianegonzalez: this is only for kolla right ?10:12
kollianwhat about kolla-ansible ?10:12
kolliandev env10:12
egonzalezkollian, kolla-ansible deploy enabled services in globals.yml, by default are https://github.com/openstack/kolla-ansible/blob/stable/ocata/ansible/group_vars/all.yml#L280 enabled10:13
kollianegonzalez: great thanks10:15
openstackgerritJames McCarthy proposed openstack/kolla-ansible master: Update designate to allow use of external bind9 dns servers.  https://review.openstack.org/50175810:18
*** jiriprox has joined #openstack-kolla10:20
*** jchhatbar has joined #openstack-kolla10:21
*** jchhatbar has quit IRC10:22
*** jchhatbar has joined #openstack-kolla10:22
*** igordc has joined #openstack-kolla10:23
*** janki has quit IRC10:24
*** jchhatbar is now known as janki10:25
*** caowei has quit IRC10:29
*** egonzalez has quit IRC10:29
*** jchhatbar has joined #openstack-kolla10:33
*** jchhatbar has quit IRC10:33
*** jchhatbar has joined #openstack-kolla10:33
*** janki has quit IRC10:35
kollianINFO:kolla.image.build.base:Cannot open: http://repo.percona.com/release/7/RPMS/x86_64/percona-release-0.1-4.noarch.rpm. Skipping. INFO:kolla.image.build.base:Error: Nothing to do INFO:kolla.image.build.base:10:37
Walexso yesterday I had thought I had found a workaround, but after a reboot no more:10:42
Walexwhen I do 'deploy' the first container to start is 'mariadb' and not the 3 common services, so I get:10:43
WalexTASK [mariadb : Creating haproxy mysql user]10:43
Walexfatal: [localhost]: FAILED! => {"changed": false, "failed": true, "msg": "kolla_toolbox container is not running."}10:43
*** rhallisey has joined #openstack-kolla10:43
Walexhow can I start the containers in a specific order if that's the problem? Looks like a race condition to me.10:44
Walexthis is with 5.0.0 and CentOS10:44
*** egonzalez has joined #openstack-kolla10:45
Walexand the host runs Scientific Linux (same as CentOS) with Docker 1.12.610:45
Walexthis is all-in-one deploy10:48
Walexand sometimes with 'deploy' I get "no hosts matched" and nothing happends10:55
Walexuhmmm using 'tools/cleanup-countainers' helped, but curious that it was needed at all, I did a clean 'stop' yesterday.11:03
*** jchhatbar is now known as janki11:04
egonzalezWalex, stop only stop containers, does not remove config, volumes, etc11:05
egonzalezthere is a bug in kolla_docker module that not start containers during deploy if are stopped, is work in progress11:06
Walexegonzalez: yes, but why is it needed after a clean shutdown and reboot? weird11:06
kollianegonzalez: INFO:kolla.image.build.base:Cannot open: http://repo.percona.com/release/7/RPMS/x86_64/percona-release-0.1-4.noarch.rpm. Skipping. INFO:kolla.image.build.base:Error: Nothing to do INFO:kolla.image.build.base11:07
egonzalezhttps://review.openstack.org/#/c/499205/11:07
kollianfacing this issue11:07
Walexegonzalez: ahhhhh!11:07
Walexegonzalez: thanks for the information.11:07
Walexegonzalez: so now I know that 'tools/cleanup-countainers' is temporarily necessary11:08
egonzalezkollian, i can retrieve that file from my host, maybe have some network issue in your side, or the cdn in percona repos on your region11:08
kollianegonzalez: couldnt parse cdn in percona repos on your region11:09
kollian:(11:10
*** gaoyan has quit IRC11:15
kolla-slack<egonzalez> kollian, try accessing tohttp://repo.percona.com/release/7/RPMS/x86_64/percona-release-0.1-4.noarch.rpm from your computer11:25
*** egonzalez has quit IRC11:26
kolla-slack<egonzalez> If that works,  is an issue in your docker network11:26
Walexgood, 'bash init-runonce' succeeded. But Horizon/web UI can be connected to but does not reply, will have a look11:27
*** zhangfei has quit IRC11:28
*** kollian has quit IRC11:31
*** dardelean has joined #openstack-kolla11:31
*** dardelea_ has quit IRC11:34
Walexfor Horizon I am hitting this known bug: https://bugs.launchpad.net/horizon/+bug/1573488/comments/6 I hope there is a corresponding Kolla bug11:41
openstackLaunchpad bug 1573488 in OpenStack Dashboard (Horizon) "Openstack Mitaka: can not access dashboard(internal server 500)" [Undecided,Expired]11:41
Walexcomment #1 says "This is a setup issue, not a Horizon issue. Please file a bug with your distributor."11:43
*** sambetts_ is now known as sambetts11:53
*** zhubingbing_ has quit IRC11:56
Walexindeed manually editing the HTTPD config files and adding the line suggested makes Horizon work.11:58
WalexAhhhh just had a look at the templates and it is really funny...12:03
Walexthe HTTPD templates for several services have "WSGIApplicationGroup %{GLOBAL}" *except* that for Horizon :-)#12:05
*** mattmceuen has quit IRC12:05
-openstackstatus- NOTICE: Gerrit will be offline for the upgrade to 2.13 starting at 15:00 UTC (in roughly 3 hours) and is expected to probably be down/unusable for 8+ hours while an offline reindex is performed: http://lists.openstack.org/pipermail/openstack-dev/2017-August/120533.html12:06
*** dave-mccowan has joined #openstack-kolla12:06
WalexIndeed all config files except for the sole case of Horizon.12:07
*** egonzalez has joined #openstack-kolla12:07
Walexall config files that have a WSGIProcessGroup12:07
*** dave-mcc_ has joined #openstack-kolla12:08
*** dave-mccowan has quit IRC12:11
*** lucasxu has joined #openstack-kolla12:12
*** mewald has joined #openstack-kolla12:16
mewaldcan I get some reviews on https://review.openstack.org/#/c/485150/ please? :)12:16
chandankumarmandre: Hello12:25
mandrechandankumar: hi!12:26
chandankumarmandre: regarding this review https://review.openstack.org/#/c/502460/,12:27
chandankumarmandre: from queens release, all the projects having bundled tempest plugin moving to a seperate repo12:27
*** janki is now known as janki|afk12:28
*** awidders_ has joined #openstack-kolla12:28
chandankumarmandre: all the tempest plugins depends on tempest. On packaging level, I have packaged it in two ways, each of the plugins in a seperate package and one meta package having all the tempest plugins12:28
chandankumarmandre: using meta package is too much risky as it might screw other tempest plugins12:28
chandankumarmandre: so i have decided, to create seperate kolla images for each plugins having base tempest container and one meta container having all tempest plugins12:29
mandrechandankumar: ack, I understand that you may want to package the plugins in different packages, but in our case we could still install all the packages in the same image, right?12:29
chandankumarso that if an user needs 2 tempest plugins, they just pull it12:30
mandrechandankumar: what's the point of the meta-package if the plugins can be conflicting?12:30
*** awiddersheim has quit IRC12:31
chandankumarmandre: if somebody wants all the plugins they can pull it meta container, it is on user how to debug it12:31
chandankumarmandre: what is what i am doing in RDO at packaging level12:31
*** palexster has joined #openstack-kolla12:32
chandankumarmandre: in ci, we are installing seperate plugins packages based on service availability12:32
Walexso how do I report a bug in a 'kolla-ansible' template?12:32
chandankumarmandre: https://github.com/rdo-packages/tempest-distgit/blob/rpm-master/openstack-tempest.spec#L9812:32
mandrehmm ok, what bothers me is that we now have a multitude of tempest images12:32
mandreI suppose this will have to go in a blueprint12:32
*** schwicht has joined #openstack-kolla12:33
chandankumarmandre: yup i will put a blueprint on that.12:33
chandankumarmandre: one more question, are we running tempest in kolla gates?12:34
mandrechandankumar: k, thx12:34
chandankumarfrom project-config jobs i didnot find a tempest.12:34
openstackgerritJames McCarthy proposed openstack/kolla-ansible master: Update designate to allow use of external bind9 dns servers.  https://review.openstack.org/50175812:35
egonzalezi dont see the point of having multiple images per plugin, I understand in a packager point of view, but not in container images12:35
mandrechandankumar: no we're not running tempest in the gates12:36
openstackgerritJames McCarthy proposed openstack/kolla-ansible master: Update designate to allow use of external bind9 dns servers.  https://review.openstack.org/50175812:36
*** schwicht has quit IRC12:37
chandankumarmandre: egonzalez i think let me enable tempest testing for kolla, then add meta container for all plugins, if not workaround move to another solution through BP?12:39
egonzalezchandankumar, awesome, that was something we discussed during PTG12:41
*** vharsh has quit IRC12:41
chandankumaregonzalez: do we have some transcripts regarding the discussion apart from this https://etherpad.openstack.org/p/kolla-queens-ptg-ansible ?12:43
egonzalezchandankumar, nope, will need to translate that into blueprints12:45
*** schwicht has joined #openstack-kolla12:46
mandrechandankumar: perfect, thanks!12:46
egonzalezjmccarthy, release note tests failed due templating errors, this passed tox -e releasenotes locally http://paste.openstack.org/show/621334/12:47
jmccarthyegonzalez: Ok - I'll update it !12:49
openstackgerritJames McCarthy proposed openstack/kolla-ansible master: Update designate to allow use of external bind9 dns servers.  https://review.openstack.org/50175812:51
openstackgerritMathias Ewald proposed openstack/kolla master: Add sensu images  https://review.openstack.org/48637912:55
*** skramaja has quit IRC12:58
*** dave-mcc_ has quit IRC12:59
*** schwicht has quit IRC12:59
egonzalezjmccarthy, much better :) http://docs-draft.openstack.org/58/501758/17/check/gate-kolla-ansible-releasenotes/69a941a//releasenotes/build/html/unreleased.html13:05
*** mnasiadka has quit IRC13:09
jmccarthyegonzalez: Cool ! Thanks for helping sort that out :)13:10
*** goldyfruit has joined #openstack-kolla13:11
openstackgerritEduardo Gonzalez proposed openstack/kolla master: Add sanity check for kolla deploy gates  https://review.openstack.org/50494113:11
jmccarthyegonzalez: Actually I had a question about rndc.conf13:11
jmccarthyegonzalez: With the default setup, on each control node the rndc.conf has the ip of itself in there13:12
jmccarthyegonzalez: So even though the pool.yaml involves all the bind9 servers, the rndc.conf only point to 1 - but one each control node they point to different ones13:13
jmccarthyegonzalez: What I mean is if it's an external setup, and this configuration file is overridden, the rndc.conf will have the address of the same server in all control nodes ?13:14
jmccarthyegonzalez:I know the pool.yaml will stuff have them, but wasn't sure if this rndc.conf setup might cause an issue of some sort ?13:14
egonzalezjmccarthy, this is because bind and mdns are deployed in same host and designate writes to all of them same records13:15
egonzalezjmccarthy, yeah, probably if bind is down in same node, mdns wont be able to connect13:15
*** schwicht has joined #openstack-kolla13:16
jmccarthyegonzalez:Yea so maybe for external, if the *one server is unavailable, they will all have problems is it ?13:16
egonzalezjmccarthy, is rndc connecting through udp or tcp? if tcp we could try with haproxy vip13:16
jmccarthyegonzalez: Maybe this should be fixed up in a separate patchset13:17
egonzalezjmccarthy, sure, if that is the case will need backport13:17
jmccarthyegonzalez:I think for the default (non-external) in stable/pike it's ok ? Since on one of the control nodes, the mdns should be able to reach it's local bind9, and then things are ok ?13:19
jmccarthy(not 100% on that)13:19
egonzalezjmccarthy, not really sure if will work in multinode since mdns was moved to network group13:22
egonzalezIt worked in my env, because some nodes are shared between network and controller groups13:23
jmccarthyegonzalez: Hmm I see, ok yea maybe some adjustment in this area does need to be backported ..13:24
*** huzhengchuan has quit IRC13:24
egonzalezjmccarthy, not sure about nothing atm xD , really I dont know which control the host mdns points to13:25
jmccarthyegonzalez: There must be some config missing - in my setup I have similar (some nodes/roles shared)13:25
egonzalezi dont know if rndc is used to connect itself, or to connect bind9 hosts13:25
jmccarthyegonzalez: So I mean by sharing in this way is possibly masking some other issue ?13:26
jmccarthyegonzalez: Hmm me either really, new to this designate stuff lol13:27
egonzalezjmccarthy, just deploying with 1 control and one network and see what happens13:28
jmccarthyegonzalez: I think I tried that and it didn't work (dunno why exactly) will see if I can rejig my setup soon13:29
*** dave-mccowan has joined #openstack-kolla13:32
*** gfidente has joined #openstack-kolla13:35
*** gfidente has quit IRC13:35
*** gfidente has joined #openstack-kolla13:35
*** lucasxu has quit IRC13:35
*** jtriley has joined #openstack-kolla13:35
-openstackstatus- NOTICE: Gerrit will be offline for the upgrade to 2.13 starting at 15:00 UTC (in roughly 1.5 hours) and is expected to probably be down/unusable for 8+ hours while an offline reindex is performed: http://lists.openstack.org/pipermail/openstack-dev/2017-August/120533.html13:37
egonzalezjmccarthy, as oracle guy you know why mitaka buildlogs was used in master build? https://github.com/openstack/kolla/blob/master/docker/base/Dockerfile.j2#L18913:38
egonzalezis failing to build due that, removing that repo succeed13:39
jmccarthyegonzalez: Oh ? Hrm I'm afraid I dunno13:40
egonzalezjmccarthy, could you confirm before +w https://review.openstack.org/#/c/50371213:40
*** jiriprox has quit IRC13:40
*** jiriprox has joined #openstack-kolla13:40
*** kollian has joined #openstack-kolla13:41
jmccarthyegonzalez: Not sure I follow, you mean I should see if I can build these ?13:43
jmccarthyegonzalez: I'll see if I can do some builds later today, might not be till evening13:45
*** yangyapeng has joined #openstack-kolla13:47
sdakeegonzalez https://github.com/openstack/kolla/commit/0537a7f9ca5e045cca1b6e083674705fab523bd913:48
*** ansmith has joined #openstack-kolla13:51
dardeleanquestion, in the Dockerfiles of the services, in horizon for example, all horizon_packages are isntalled via pip, right?13:54
egonzalezdardelean, nope, binary with packages and source from tarball13:56
*** trozet has quit IRC13:56
egonzalezsdake, https://bugs.launchpad.net/kolla/+bug/1576238 is there a reason to maintain mitaka in there? oraclelinux builds ok now without build logs13:57
openstackLaunchpad bug 1576238 in kolla mitaka "Base image build broken for oraclelinux" [Critical,Fix committed] - Assigned to Jeffrey Zhang (jeffrey4l)13:57
dardeleanfor example ubuntu binary from apt?13:57
sdakeegonzalez not that I'm awaare of13:57
egonzalezdardelean, yep13:57
*** erlon has joined #openstack-kolla13:57
dardelean@egonzalez, I have ubuntu source and I have a .dev with a hotizon dashboard, how can I install it?13:58
dardeleanif i just dpkg -i, it is missing the openstack-dashboard package13:58
dardeleanin this case I have to do it from binary, right?13:58
egonzalezdardelean, in source openstack-dashboard is installed from tarball into a virtual environment13:59
dardeleani see the packages in pip, this means I can install pip packages on them, right?14:00
egonzalezdardelean, yep, first ``source /var/lib/kolla/venv/bin/activate``, however is not recommended install directly in containers, better try to build you own14:02
hrwmorning14:04
hrwhow went PTG?14:04
dardeleanhi14:04
*** yangyapeng has quit IRC14:05
kollianegonzalez: seems like in docker network14:06
hrwwill "--docker-dir" argument allow me to provide 'overlay' on top of kolla's docker/ directory?14:06
kollianas curl -i http://repo.percona.com/release/7/RPMS/x86_64/percona-release-0.1-4.noarch.rpm working14:06
*** huzhengchuan has joined #openstack-kolla14:09
*** mgoddard has quit IRC14:10
egonzalezkollian, are you behind a proxy or something?14:10
kollianegonzalez: no14:11
egonzalezkollian, could you try again skipping current image cache? --nocache in build command14:11
*** mgoddard has joined #openstack-kolla14:12
dardelean@egonzales thank you14:12
kollianegonzalez: ok trying14:12
kollianegonzalez: not helping14:14
kolliangot the same errro14:14
*** jtriley has quit IRC14:20
*** yangyapeng has joined #openstack-kolla14:20
*** mgoddard has quit IRC14:21
*** k_mouza has joined #openstack-kolla14:23
*** kiennt has joined #openstack-kolla14:24
k_mouzagood day all! First of all, egonzalez thanks a lot for your help with Murano the other day. It finally worked!14:27
k_mouzaI was also wondering if you have any idea of when the Centos Binaries for Monasca will be ready to build?14:27
*** zhubingbing has joined #openstack-kolla14:28
egonzalezk_mouza, only monasca python clients are available in RDO https://trunk.rdoproject.org/centos7-master/current/14:31
-openstackstatus- NOTICE: Gerrit will be offline for the upgrade to 2.13 starting at 15:00 UTC (in roughly 30 minutes) and is expected to probably be down/unusable for 8+ hours while an offline reindex is performed: http://lists.openstack.org/pipermail/openstack-dev/2017-August/120533.html14:31
*** hongbin has joined #openstack-kolla14:32
*** david-lyle has joined #openstack-kolla14:34
*** krtaylor has joined #openstack-kolla14:37
*** hrw has quit IRC14:48
*** lakerzhou has joined #openstack-kolla14:49
*** hrw has joined #openstack-kolla14:50
*** yangyapeng has quit IRC14:51
*** k_mouza has quit IRC14:54
*** k_mouza has joined #openstack-kolla14:54
*** livelace has joined #openstack-kolla14:55
*** jamesbenson has joined #openstack-kolla14:55
*** jiriprox has quit IRC14:58
openstackgerritMerged openstack/kolla master: Fix dind and zun images and oraclelinux build  https://review.openstack.org/50371214:59
egonzalezbuild gates are fixed, can recheck failing changes15:00
-openstackstatus- NOTICE: The Gerrit service at https://review.openstack.org/ is offline, upgrading to 2.13, for an indeterminate period of time hopefully not to exceed 23:59 UTC today: http://lists.openstack.org/pipermail/openstack-dev/2017-August/120533.html15:02
*** huzhengchuan has quit IRC15:03
*** lucasxu has joined #openstack-kolla15:12
*** yangyapeng has joined #openstack-kolla15:12
*** zhubingbing has quit IRC15:14
*** janki|afk is now known as janki15:14
*** jtriley has joined #openstack-kolla15:15
*** mgoddard has joined #openstack-kolla15:23
*** serlex has quit IRC15:24
*** dardelean has quit IRC15:26
*** vhosakot has joined #openstack-kolla15:33
*** itlinux has joined #openstack-kolla15:35
*** yangyapeng has quit IRC15:36
*** itlinux has quit IRC15:36
*** trozet has joined #openstack-kolla15:38
*** mdnadeem has quit IRC15:42
*** lucasxu has quit IRC15:50
*** kiennt has quit IRC15:53
*** sbezverk has quit IRC16:00
*** jamesbenson has quit IRC16:11
vhosakothttps://review.openstack.org is down16:11
egonzalezvhosakot, is under maintenance for a few hours16:12
vhosakotyeah, saw the notice.16:12
vhosakotegonzalez: did you have a good flight back home?16:12
sdakesup egonzalez - notsure if i missed yoru response - although it has  scrolled16:13
egonzalezvhosakot, yeah, long trip16:13
vhosakotegonzalez: cool16:13
*** Slower has joined #openstack-kolla16:14
*** egonzalez has quit IRC16:16
*** pcaruana has quit IRC16:16
*** lucasxu has joined #openstack-kolla16:22
*** sbezverk has joined #openstack-kolla16:22
*** dave-mccowan has quit IRC16:22
*** gfidente has quit IRC16:23
*** itlinux has joined #openstack-kolla16:25
*** gfidente has joined #openstack-kolla16:26
*** unicell has quit IRC16:28
-openstackstatus- NOTICE: The Gerrit service at https://review.openstack.org/ is offline, upgrading to 2.13, for an indeterminate period of time hopefully not to exceed 23:59 UTC today: http://lists.openstack.org/pipermail/openstack-dev/2017-August/120533.html16:37
*** ChanServ changes topic to "The Gerrit service at https://review.openstack.org/ is offline, upgrading to 2.13, for an indeterminate period of time hopefully not to exceed 23:59 UTC today: http://lists.openstack.org/pipermail/openstack-dev/2017-August/120533.html"16:37
inc0good morngin everyone16:40
*** k_mouza has quit IRC16:41
*** trozet has quit IRC16:43
vhosakotmorning inc0, we're still in station 26 lol16:44
inc0lol16:45
inc0I thought you were coming back yesterday?16:46
*** itlinux has quit IRC16:46
inc0this ptg was cool16:46
*** jamesbenson has joined #openstack-kolla16:48
*** janki has quit IRC16:49
*** jamesbenson has quit IRC16:52
*** dave-mccowan has joined #openstack-kolla16:58
*** itlinux has joined #openstack-kolla16:58
vhosakotinc0: I was kidding, came back home yesterday.16:58
vhosakotyeah, the ptg was awesome.16:58
inc0right, it's raining in PDX finally, but that means my brain went into "just lay in bed wrapped tightly with blanets" mode16:59
inc0sense of humor is slow16:59
*** shardy has quit IRC17:00
*** aagate has joined #openstack-kolla17:01
rwellumMorning guys.17:03
inc0hey Rich, welcome back to land of written text;)17:03
rwellumWay less efficient...17:03
inc0well that's where code is made tho17:04
inc0now we all turn back into worky ants and implement what we discussed17:04
inc0:)17:04
rwellumYeah was looking at the kolla-k8s 1.0 requirements this morning17:05
vhosakotto the land of "text"  from the land of "mostly free beer, lots of amazing face-to-face interaction/debugging, late night drinking talking the ideas of the future technology"17:05
inc0if it happens to be in Colorado, add weed to mix17:05
vhosakothahaha.. true17:06
rwellumYeah people happy to sit down and debug ovs switching with me for hours. Fantastic!17:06
inc0did you get it working?17:07
inc0to work*17:07
rwellumWe did... There's a little mini-pc sitting on my desk running OpenStack with full access to VM's right now.17:07
inc0awesome17:07
rwellumsean-k-mooney: and vhosakot and kfox1111 worked a miracle.17:08
inc0haha17:08
* inc0 was lazy at that very moment17:08
rwellumYou had a lot going on...17:08
inc0well, we worked on replacing docker build;)17:08
inc0so that was fun17:08
rwellumWas going to ask was there any conclusion with Triple-o interlock?17:08
inc0https://etherpad.openstack.org/p/tripleo-ptg-queens-kolla-collaboration17:09
inc0there is plenty of collab space there17:09
inc0yay I got permission for Sydney! \o/17:10
vhosakotnice inc0! :)17:10
rwellumI saw that but then you were talking with the team again on Monday?17:10
rwellumNice inc017:10
inc0rwellum: no, we didn't17:11
vhosakotinc0: yeah, testing build without docker build was a a very cool hack.17:11
rwelluminc0: you and kfox1111 and a big bunch of the red hat guys were discussing - I sat and listened for a while... Maybe not triple-o?17:13
vhosakotThe main takeaway I got from the tripleO meeting was cross-project gates and they may use the config-gen part of kolla17:13
kfox1111morning.17:14
inc0rwellum: well, we did talk about dockerhub publishing too17:14
inc0mostly with mad hatters17:14
vhosakotyeah, that was on Fri with infra guys.. I liked that chat17:14
kfox1111they want to do ansible templates so probably wont be sharing our helm objects. :/17:14
inc0but it's really only matter of switching few tasks one with another17:15
kfox1111they had an idea to share openstack-ansible config gen17:15
inc0not a bad idea too17:15
rwellumOverall seemed positive to me.17:15
vhosakotyeah, they talks about sharing OSA's config-gen17:19
vhosakottalked*17:19
*** dardelean has joined #openstack-kolla17:20
*** itlinux has quit IRC17:22
ulf`inc0: ping17:23
ulf`inc0: got some time now?17:23
inc0yup17:23
inc0I'm at my desk17:23
ulf`inc0: let me swing by your cube17:23
* ulf` just set the whole thing back up in the lab17:23
*** trozet has joined #openstack-kolla17:24
* rwellum wants to come by inc0 cube too...17:27
*** rhallisey has quit IRC17:27
*** rhallisey has joined #openstack-kolla17:28
dardeleanhei17:32
*** lucasxu has quit IRC17:35
*** itlinux has joined #openstack-kolla17:36
*** dave-mcc_ has joined #openstack-kolla17:38
*** dave-mccowan has quit IRC17:39
dardeleaninc0 that a witcher cosplay on twitter?17:43
inc0almost;)17:43
inc0we had piano party on ptg17:43
inc0and there were costumes involved17:43
dardeleannice17:43
inc0and I became creative17:44
dardeleani'm reading the books and replaying the witcher 3 atm, i'm absolutely blown away17:44
inc0haha17:44
*** livelace has quit IRC17:44
inc0try 1 and 2 too17:44
dardeleanwork and withcher on repeat17:44
inc0a bit more "rigit" in gameplay departament17:44
inc0but good games nonetheless17:44
inc0I just replayed both and started w317:45
*** jbrooks has quit IRC17:45
dardelean1 is unplayable for me sadly :(17:45
dardeleani will do 2 after 317:45
inc0why it's unplayable?17:45
inc0besides looking like..uhh17:45
dardeleanthe combat system is bad, stuff like that17:45
inc0(it wasn't bad when it was new)17:46
inc0yeah17:46
dardeleantried it like 2 times, no...17:46
inc0story is good tho17:46
inc0and you'll get lots of references to it in 317:46
*** jbrooks has joined #openstack-kolla17:46
dardeleanoh, yes, I know17:46
*** sambetts is now known as sambetts|afk17:47
dardeleanvernon roche is my fav char17:48
inc0well, that's one of main chars in 217:48
dardeleanyes yes17:48
dardeleanthere were some news some years back that they were doing a movie, dunno what came of it17:49
inc0netflix is doing TV series17:50
inc0http://www.ign.com/articles/2017/05/17/netflix-to-produce-the-witcher-tv-series17:51
dardeleanoh, let's hope they won;t cast Michał Żebrowski17:52
inc0hahaha17:52
inc0hear hear17:52
inc0I hated polish adaptation17:52
dardeleanhaha, I can relate17:53
inc0and by hate I mean pure, fiery form of hate, one that stays with you17:53
iniazihas anyone tried kolla on centos 7.4?17:53
dardeleaniniazi not yet, just curious what kernel varsion does it have?17:55
iniazi 3.10.0-693.2.2.el7.x86_6417:55
*** jmccarthy has quit IRC17:55
dardeleangod, still not on 417:56
iniazii think same kernel just newer revision17:56
iniaziwhat version is on ubuntu?17:57
iniazii mean 4.x or 3.x?17:58
iniazibut centos/rhel 7.x should stay on the same major kernel... only newer major rhel/centos would do a major kernel upgrade, I think17:59
dardeleanubuntu is on 4.417:59
iniazii think maybe I should upgrade my servers to ubuntu... or atleast the ones for openstack/kolla?  running into a lot of issues18:00
dardeleanneed 4 for I use HyperV a lot as hypervisor, 4 kernel contains stuff that makes it faster18:00
iniaziwell not upgrade, but rather reinstall18:00
iniaziwhats the general feel for ubuntu vs centos (latest versions) for kolla/openstack?  i see several docs/packages come out for ubuntu (openstack anyway)18:01
iniazicome out for ubuntu first, i mean, it seems18:01
*** egonzalez has joined #openstack-kolla18:02
dardeleaniniazi from what I see thay are maintained kind of the same, think it's safe to go with either, i prefer ubuntu18:05
iniaziok.  well centos 7.4 upgrade came out recently, and that has broken kolla.  can't build, and of course some launched services don't start up18:08
*** itlinux has quit IRC18:12
Sloweriniazi: oooo.. I just installed centos 7.4..18:12
rwelluminiazi: I don't see much difference between the two tbh18:17
rwellumMaybe ubuntu a bit more stable18:17
*** itlinux has joined #openstack-kolla18:18
dardeleanyep, i feel the same18:18
*** jmccarthy has joined #openstack-kolla18:22
*** ntpttr_laptop has joined #openstack-kolla18:22
kolla-slack<britthouser> Good job @rwellum on getting that cluster up!18:22
iniazistability wise, I think both are good.  sometimes comes down to certain apps.18:22
iniaziSlower, have you deployed kolla-ansible yet?  had any input on using it?18:23
egonzaleziniazi build issues has  been fixed recently, deploy on centos 7.4 still trying to figure out what is failing18:27
*** ntpttr_laptop has quit IRC18:27
egonzalezas far i found libvirt version changed from  2.3 to 3.218:27
iniaziegonzalez, fixed recently, like past few hours?  it didn't about 10 hours ago.18:28
*** jmccarthy has quit IRC18:28
iniaziin my build it was failing on: kolla.image.build.nova-compute:  Downloading libvirt-python-2.5.0.tar.gz18:28
egonzaleziniazi around 4 hours18:28
iniazicool.  I'll try to build now18:28
egonzaleziniazi different issue then, failure was dind in ubuntu, and some oraclelinux repos18:29
iniaziegonzalez, oh, so different issue.  i'm getting ERROR: failed virConnectSecretEventRegisterAny and ERROR: failed virConnectSecretEventRegisterAny followed by error: command '/var/lib/kolla/venv/bin/python' failed with exit status 118:30
iniazimaybe its innocous, and my libvirt container isn't starting for another reason, although sounds similiar.  the error on libvirt bootstrap is:18:30
egonzaleziniazi nova compute error when creating instance?18:30
iniazino, can't deploy18:31
iniazirunning kolla-ansible deploy...18:31
iniazierrors i see in nova-libvirt container when deploy fails: error : daemonSetupNetworking:554 : unsupported configuration: No server certificate path set to match server key18:32
egonzaleziniazi i've not  seen such error yet18:33
iniazialmost everything in centos 7.3 was working fine when running deploy except that mariadb won't form a cluster on deploy.  timeouts waiting for mariadb start.  normally I have to run deploy over and over again until it starts to work.  usually takes 2 to 4 tries.18:34
*** itlinux has quit IRC18:34
vhosakotegonzalez: do you run multinode kolla?18:35
vhosakotkolla-ansible I meant18:35
egonzalezvhosakot yep, always18:35
iniazibut late last week, after about 12 to 14 tries, it won't coalesce.  i realized there were updates for centos.. and i figured it would be good.  didn't realize it was major release upgrade18:35
iniazi(I also had just fixed a yum priorities issue, so thought it was giving me a package list related to that).  anyway, that was my fault.  immediately after, lots of packages wouldn't deploy18:37
egonzalezyeah, started to  fail  arount thursday when  centos 7.4 was pushed to docker hub18:37
iniazitried to rebuild images using source, nova-compute wouldn't build etc.18:37
vhosakotegonzalez: cool, I'll ask you help later today/tomorrow about the inventory file example18:37
iniazisource build was working great a few weeks ago.  Anyway I tried binary, and whatever fails, i built as source, updated globals.yml and overloaded the image variables to signify 'source' as the install type18:38
Sloweriniazi: just getting things set up18:38
Slowerso no I haven't18:39
egonzalezvhosakot current one http://paste.openstack.org/show/621352/ , other groups are default18:39
*** Suicky has joined #openstack-kolla18:41
iniazibuild failed for nova-compute, so that was a diff issue.  (this is centos image on centos 7.4) https://pastebin.com/n170J6JM18:47
*** Suicky has left #openstack-kolla18:48
egonzaleziniazi i'm building  atm, however my host still in centos 7.318:53
*** cduchesne has quit IRC18:53
*** cduchesne has joined #openstack-kolla18:53
*** cduchesne has joined #openstack-kolla18:54
egonzalezgates build succeed with centos  7.4 host, i dont think is a host issue during building18:54
*** cduchesne has quit IRC18:54
*** cduchesne has joined #openstack-kolla18:54
*** cduchesne has quit IRC18:54
*** cduchesne has joined #openstack-kolla18:55
*** cduchesne has quit IRC18:57
*** cduchesne has joined #openstack-kolla18:57
*** mgoddard has quit IRC18:58
*** cduchesne has quit IRC18:58
*** cduchesne has joined #openstack-kolla18:58
*** robbbe has joined #openstack-kolla19:09
*** dardelean_ has joined #openstack-kolla19:17
*** mgoddard has joined #openstack-kolla19:19
*** jamesbenson has joined #openstack-kolla19:24
*** jamesbenson has quit IRC19:29
*** dardelean has quit IRC19:31
*** dardelean has joined #openstack-kolla19:31
*** gfidente has quit IRC19:32
mandreegonzalez: I'm seeing the same issue with centos 7.4 based containers in my local tripleo deployment19:33
mandreegonzalez: same as http://logs.openstack.org/40/456140/19/check/gate-kolla-ansible-dsvm-deploy-centos-binary-centos-7-nv/529537c/logs/kolla/nova/nova-compute.txt.gz#_2017-09-18_14_54_37_22019:33
*** gfidente has joined #openstack-kolla19:34
*** gfidente has quit IRC19:34
*** gfidente has joined #openstack-kolla19:34
*** dave-mcc_ has quit IRC19:35
*** dardelean has quit IRC19:35
*** awidders_ has quit IRC19:37
*** awiddersheim has joined #openstack-kolla19:38
*** kollian has quit IRC19:38
vhosakotthanks for the link egonzalez!19:43
*** mewald has left #openstack-kolla19:44
egonzalezmandre any idea?19:44
iniaziegonzalez, ok.  i'm guessing docker & containers will all be using parent os binaries, right, so if I build something on 7.3 but try to deploy on 7.4?19:44
mandreegonzalez: not at all :D I'll try debugging tomorrow19:45
*** jamesbenson has joined #openstack-kolla19:45
iniazibtw, I'm trying to deploy ocata/4.0.2.  i forgot to mention again19:46
mandreiniazi: it should work if you base your containers on centos 7.3, by using FROM: centos:7.3.161119:47
iniazimandre how/where would I set the FROM?19:48
mandreif this is a centos 7.4 issue that is... it's not yet clear what the issue is19:48
egonzaleziniazi --base-tag 7.3.1611 in kolla-build19:49
*** awiddersheim has quit IRC19:49
*** dave-mccowan has joined #openstack-kolla19:50
egonzalezmandre thanks for taking a look  :)19:50
iniazimandre, yeah, sorry.  trying to deploy on 7.4 using ocata/4.0.2 kolla.  issue is that source packages don't build completely (nova-compute getting an error on libvirt).  tried binary, which does build most, and rest I built using source, but i get a cert error with libvirt container19:50
*** awiddersheim has joined #openstack-kolla19:50
*** jamesbenson has quit IRC19:50
iniazii have to run... going to let the build run...19:55
iniazido I need to set the base-tag on deploy as well?19:55
egonzaleziniazi nope, image will be tagged with kolla version19:56
mandresorry iniazi, in a meeting now19:56
iniazino problem mandre, thanks guys.  i just finished my 4 or 5 hours of meetings... :)  going to go to work for a few, and try to get some work done19:57
mandreiniazi: in testing kolla containers from master branch with centos 7.4 containers, the deployment works fine, but then it fails when trying to boot a VM19:57
mandreiniazi: nova-compute fails to communicate with libvirt19:58
iniazimandre: interesting.  haven't gotten that far with 7.4.  but I do see an error when deploying in 7.4, and the error is "unsupported configuration: No server certificate path set to match server key" sounds similar issue19:58
iniazithats the nova-libvirt container. i'm hoping anchoring 7.3 will solve that.19:59
mandreiniazi: worth a try :)19:59
goldyfruitHey guys, any of you have issue trying to redeploy glance ?20:04
goldyfruitglance_api container is never recreated20:05
goldyfruithttps://paste.api-zulu.com/ceqakezime.sql20:05
goldyfruitthe handler is not trigger20:05
goldyfruitit worked for glance_registry20:05
iniaziand general question, if I deploy with 1 network node, can I add another network node in the future, without redeploy openstack?  i.e. with kolla-ansible20:07
egonzalezgoldyfruit pike with glance backend file?20:11
goldyfruitegonzalez: pike with glance backend external ceph20:12
goldyfruitI found the issue20:12
goldyfruitthe glance_api handler has this condition :  - inventory_hostname in glance_api_hosts20:12
*** itlinux has joined #openstack-kolla20:12
goldyfruitbut should be :  - inventory_hostname in groups[service.group]20:12
egonzalezgoldyfruit that is to avoid multile glance-api when file backend is used20:13
egonzalezs/multiple20:13
egonzalezhttps://github.com/openstack/kolla-ansible/blob/9c43de8955860f74cbd0680cd2e40699622a4ea3/ansible/roles/glance/defaults/main.yml#L8020:15
*** jamesbenson has joined #openstack-kolla20:27
*** unicell has joined #openstack-kolla20:30
*** jamesbenson has quit IRC20:31
*** gfidente has quit IRC20:32
*** robbbe has quit IRC20:34
bkeroHi there. We're working on a pike deployment with kolla. One of our compute hosts has /dev/kvm set to the wrong group. For some reason it's owned by crontab.20:38
bkeroBut only on one host. Has anyone experienced something like this?20:39
*** k_mouza has joined #openstack-kolla20:39
*** jiriprox has joined #openstack-kolla20:40
*** k_mouza has quit IRC20:42
bkerodocker/nova/nova-libvirt/Dockerfile.j2:{{ macros.configure_user(name='nova', groups='qemu') }}20:45
bkerodocker/nova/nova-libvirt/extend_start.sh:    chown root:kvm /dev/kvm20:45
bkeroThat...doesn't seem right20:45
bkeroOur nova user isn't part of the qemu group20:46
*** jamesbenson has joined #openstack-kolla20:48
*** jamesbenson has quit IRC20:52
bkeroinc0: ^ any idea?20:57
*** ntpttr_laptop has joined #openstack-kolla20:57
bkeroSamYaple: Why the 'qemu' group? {{ macros.configure_user(name='nova', groups='qemu') }}20:59
SamYaplebkero: are you using centos or ubuntu?21:00
*** jtriley_ has joined #openstack-kolla21:00
bkeroSamYaple: ubuntu21:00
SamYapleall the /dev/kvm stuff is for compatiblity with centos. it just happens to work for ubuntu21:00
SamYapleunless you are saying its broken?21:01
bkeroI'm thinking it's broken21:01
SamYapledo you have an error somewhere?21:01
bkeroI do21:01
bkerodocker/nova/nova-libvirt/extend_start.sh:    chown root:kvm /dev/kvm21:01
bkerodocker/nova/nova-libvirt/Dockerfile.j2:{{ macros.configure_user(name='nova', groups='qemu') }}21:01
bkerohttp://paste.openstack.org/show/621356/21:02
bkeropermission denied for /dev/kvm21:02
*** ansmith has quit IRC21:02
bkerokvm is 660 root:kvm, but the nova user is not in the 'kvm' group, but is in 'qemu'21:02
*** jtriley has quit IRC21:03
SamYaplebkero: meh that doesnt matter so much21:03
SamYaplebecause libvirt (technically qemu) is the one talking to /dev/kvm21:04
SamYapleso nova-conductor is telling you this21:04
goldyfruitbkero: https://bugs.launchpad.net/kolla/+bug/168146121:04
openstackLaunchpad bug 1681461 in kolla "Instance fails to boot: Could not access KVM kernel module: Permission denied" [Undecided,Confirmed] - Assigned to Mohammed Naser (mnaser)21:04
SamYapleis nova-conductor trying do somethign with kvm that it wasnt before?21:04
*** jtriley_ has quit IRC21:05
goldyfruitegonzalez: ok, thanks21:05
SamYaplebkero: ah so it seems upstream libvirt issue. there are work arounds in that bug21:06
bkeroSo it'll either be starting them as libvirt or nova21:06
bkeroneither are in the kvm group21:06
bkerobleh21:06
bkeroYeah, I know I can just chown it, but I wanted to solve the general case :/21:06
SamYaplewell bkero since this is an *upstream* libvirt change, not much we can do other than workaround it however we see fit21:07
bkeroright21:07
bkeroThe chowning seemed to be working around it already21:09
bkeroDoes libvirt launch the VMs as libvirt, or as nova? I see running processes as nova.21:12
bkerothe nova-libvirt Dockerfile is adjusting the 'nova' user, not libvirt21:13
bkerohttps://github.com/openstack/kolla-ansible/blob/master/ansible/roles/nova/templates/qemu.conf.j221:18
*** lakerzhou has quit IRC21:29
*** mgoddard has quit IRC21:29
*** jamesbenson has joined #openstack-kolla21:29
*** jamesbenson has quit IRC21:34
*** schwicht has quit IRC21:40
inc0bkero: I think libvirt runs vms as libvirt, or even root21:41
inc0libvirt is priviledged container21:41
bkeroinc0: but that github link ^21:42
inc0ahh21:43
inc0my bad21:43
bkeroI have a patch to s/qemu/kvm/ for the nova groupadd21:43
bkerobut it's waiting on gerrit :)21:43
inc0right...I fixed it, fun21:43
inc0kk21:43
bkeroI'm going to chown it on my server for now so I can go find all the other pike deployment bugs21:44
inc0thanks bkero21:44
bkerobut I'll submit this for review later and file a backport to stable/pike21:44
inc0makes sense, thanks21:45
bkeroOf course :) I need this to work21:49
*** dave-mccowan has quit IRC21:58
*** dardelean_ has quit IRC21:59
*** schwicht has joined #openstack-kolla22:01
*** dardelean has joined #openstack-kolla22:03
egonzalezmandre tried with host centos 7.3  and container 7.4 and worked, same for host and containers 7.4 (qemu and kvm both), fresh images from 1 hour ago, weird22:04
*** dave-mccowan has joined #openstack-kolla22:05
bkeroegonzalez: this is xenial22:07
bkeroalso centos has 666 permissions on /dev/kvm AIUI22:07
egonzalezbkero different issue with nova broken due new centos 7.4, but now worked for me locally22:08
*** jamesbenson has joined #openstack-kolla22:11
*** dave-mccowan has quit IRC22:13
egonzalezbkero although i'd not change nothing in libvirt group, at least for rhel distros because are working and likely will cause some upgrade issue22:13
bkeroegonzalez: Yeah. I added a conditional to the nova-libvirt dockerfile to add the 'kvm' group to the nova user22:13
bkeroand left rhel/centos/oracle unchanged22:14
bkerohttp://paste.openstack.org/show/621366/22:14
egonzalezbkero there is a review with a discussion about this, gerrit is down but here is the bug with some context https://bugs.launchpad.net/kolla/+bug/171535622:15
openstackLaunchpad bug 1715356 in kolla pike "libvirt raise Could not access KVM kernel module error" [Undecided,Triaged]22:15
*** jamesbenson has quit IRC22:15
bkeroegonzalez: yeah, the chmod 666 patch got backed out and nobody's moved on this for a week22:16
bkeroas long as ubuntu nova-libvirt is executing as the 'nova' user, this is going to be a problem22:17
egonzalezbkero gates works (with qemu) guess is something specific to kvm22:17
bkeroyes22:18
bkeroqemu doesn't access /dev/kvm so it doesn't get the permission denied error22:18
bkerogate is software because nested can't be trusted on infra clouds22:18
*** schwicht has quit IRC22:30
*** schwicht has joined #openstack-kolla22:32
*** egonzalez has quit IRC22:39
*** schwicht has quit IRC22:40
*** itlinux has quit IRC22:45
*** dave-mccowan has joined #openstack-kolla22:46
*** imcsk8 has joined #openstack-kolla22:49
*** inc0 has quit IRC22:53
*** inc0 has joined #openstack-kolla22:54
*** bmace has quit IRC23:02
*** bmace has joined #openstack-kolla23:02
*** jamesbenson has joined #openstack-kolla23:13
*** jamesbenson has quit IRC23:18
*** dardelean has quit IRC23:19
*** goldyfruit has quit IRC23:26
*** erlon has quit IRC23:27
*** reddydodda has joined #openstack-kolla23:32
*** hongbin has quit IRC23:34
*** reddydodda has quit IRC23:40
openstackgerritManjunath Ranganathaiah proposed openstack/kolla master: Adding ODL source install for kolla.  https://review.openstack.org/50497523:42
*** david-lyle has quit IRC23:43
*** ChanServ changes topic to "support: ask.openstack.org | New to Kolla: docs @ https://docs.openstack.org/kolla/latest/ | | Kolla IRC meetngs on Wednesdays @ 16:00 UTC - see agenda @ https://goo.gl/OXB0DL - IRC channel is *LOGGED* @ http://goo.gl/3mzZ7b"23:45
-openstackstatus- NOTICE: review.openstack.org Gerrit 2.13 upgrade is functionally complete. The Infra team will be cleaning up bookkeeping items over the next couple days. If you have any questions please let us know23:45
*** jmorgan1 has left #openstack-kolla23:49
*** jamesbenson has joined #openstack-kolla23:55
*** lakerzhou has joined #openstack-kolla23:56
*** jamesbenson has quit IRC23:59

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!