| *** fergus has joined #openstack-ansible | 00:36 | |
| *** SerenaFeng has joined #openstack-ansible | 01:03 | |
| *** SerenaFeng has quit IRC | 01:24 | |
| *** SerenaFeng has joined #openstack-ansible | 01:24 | |
| *** dxiri has joined #openstack-ansible | 01:34 | |
| *** cuongnv has joined #openstack-ansible | 01:37 | |
| *** SerenaFeng has quit IRC | 01:43 | |
| *** SerenaFeng has joined #openstack-ansible | 01:44 | |
| *** woodard has joined #openstack-ansible | 01:48 | |
| *** woodard has quit IRC | 01:52 | |
| *** charcol has quit IRC | 01:54 | |
| *** dxiri_ has joined #openstack-ansible | 01:57 | |
| *** charcol has joined #openstack-ansible | 01:57 | |
| *** dxiri has quit IRC | 01:58 | |
| *** SerenaFeng has quit IRC | 01:58 | |
| *** SerenaFeng has joined #openstack-ansible | 01:58 | |
| *** SerenaFeng has quit IRC | 02:10 | |
| *** gouthamr has joined #openstack-ansible | 02:14 | |
| *** SerenaFeng has joined #openstack-ansible | 02:19 | |
| *** dxiri_ has quit IRC | 02:23 | |
| *** dxiri has joined #openstack-ansible | 02:26 | |
| *** hybridpollo has quit IRC | 02:39 | |
| *** hybridpollo has joined #openstack-ansible | 02:44 | |
| *** dxiri has quit IRC | 02:46 | |
| *** dxiri has joined #openstack-ansible | 02:48 | |
| *** dxiri has quit IRC | 02:54 | |
| *** dxiri has joined #openstack-ansible | 02:56 | |
| *** strattao has quit IRC | 02:59 | |
| *** strattao has joined #openstack-ansible | 02:59 | |
| *** charcol has quit IRC | 03:02 | |
| openstackgerrit | zhongshengping proposed openstack/openstack-ansible-os_aodh master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477320 | 03:14 |
|---|---|---|
| *** gkadam has joined #openstack-ansible | 03:15 | |
| *** woodard has joined #openstack-ansible | 03:15 | |
| *** woodard has quit IRC | 03:16 | |
| *** woodard has joined #openstack-ansible | 03:16 | |
| *** woodard has quit IRC | 03:17 | |
| *** woodard has joined #openstack-ansible | 03:17 | |
| *** woodard has quit IRC | 03:17 | |
| *** woodard has joined #openstack-ansible | 03:18 | |
| *** woodard has quit IRC | 03:18 | |
| *** woodard has joined #openstack-ansible | 03:19 | |
| *** woodard has quit IRC | 03:19 | |
| *** woodard has joined #openstack-ansible | 03:19 | |
| *** woodard has quit IRC | 03:20 | |
| *** gouthamr has quit IRC | 03:22 | |
| *** dxiri has quit IRC | 03:24 | |
| *** charcol has joined #openstack-ansible | 03:27 | |
| openstackgerrit | zhongshengping proposed openstack/openstack-ansible-os_barbican master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477325 | 03:28 |
| openstackgerrit | zhongshengping proposed openstack/openstack-ansible-os_barbican master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477325 | 03:29 |
| openstackgerrit | zhongshengping proposed openstack/openstack-ansible-os_ceilometer master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477326 | 03:30 |
| openstackgerrit | zhongshengping proposed openstack/openstack-ansible-os_cinder master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477328 | 03:31 |
| openstackgerrit | zhongshengping proposed openstack/openstack-ansible-os_cloudkitty master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477329 | 03:31 |
| openstackgerrit | zhongshengping proposed openstack/openstack-ansible-os_designate master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477330 | 03:32 |
| openstackgerrit | zhongshengping proposed openstack/openstack-ansible-os_glance master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477332 | 03:33 |
| openstackgerrit | zhongshengping proposed openstack/openstack-ansible-os_gnocchi master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477333 | 03:33 |
| openstackgerrit | zhongshengping proposed openstack/openstack-ansible-os_heat master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477334 | 03:34 |
| openstackgerrit | zhongshengping proposed openstack/openstack-ansible-os_ironic master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477335 | 03:34 |
| openstackgerrit | zhongshengping proposed openstack/openstack-ansible-os_keystone master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477336 | 03:35 |
| openstackgerrit | zhongshengping proposed openstack/openstack-ansible master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477337 | 03:35 |
| openstackgerrit | zhongshengping proposed openstack/openstack-ansible-os_magnum master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477338 | 03:35 |
| openstackgerrit | zhongshengping proposed openstack/openstack-ansible-os_neutron master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477339 | 03:36 |
| openstackgerrit | zhongshengping proposed openstack/openstack-ansible-os_monasca master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477340 | 03:37 |
| openstackgerrit | zhongshengping proposed openstack/openstack-ansible-os_octavia master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477341 | 03:37 |
| openstackgerrit | zhongshengping proposed openstack/openstack-ansible-os_rally master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477343 | 03:38 |
| openstackgerrit | zhongshengping proposed openstack/openstack-ansible-os_sahara master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477344 | 03:38 |
| openstackgerrit | zhongshengping proposed openstack/openstack-ansible-os_nova master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477345 | 03:38 |
| openstackgerrit | zhongshengping proposed openstack/openstack-ansible-os_searchlight master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477346 | 03:38 |
| openstackgerrit | zhongshengping proposed openstack/openstack-ansible-os_swift master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477347 | 03:39 |
| openstackgerrit | zhongshengping proposed openstack/openstack-ansible-os_tempest master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477348 | 03:39 |
| openstackgerrit | zhongshengping proposed openstack/openstack-ansible-os_trove master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477349 | 03:39 |
| openstackgerrit | zhongshengping proposed openstack/openstack-ansible-os_watcher master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477350 | 03:40 |
| openstackgerrit | zhongshengping proposed openstack/openstack-ansible-os_zaqar master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477351 | 03:41 |
| *** dxiri has joined #openstack-ansible | 03:50 | |
| *** dxiri has quit IRC | 03:55 | |
| *** rstarmer has joined #openstack-ansible | 03:57 | |
| *** SerenaFeng has quit IRC | 04:00 | |
| *** rstarmer has quit IRC | 04:05 | |
| *** cNilesh has joined #openstack-ansible | 04:11 | |
| *** hybridpollo has quit IRC | 04:19 | |
| *** cNilesh has quit IRC | 04:24 | |
| *** cNilesh has joined #openstack-ansible | 04:24 | |
| *** cNilesh has quit IRC | 04:24 | |
| *** cNilesh has joined #openstack-ansible | 04:24 | |
| *** cNilesh has quit IRC | 04:27 | |
| *** cNilesh has joined #openstack-ansible | 04:27 | |
| *** charcol-v2 has joined #openstack-ansible | 04:31 | |
| *** charcol has quit IRC | 04:34 | |
| *** ccollett has joined #openstack-ansible | 04:38 | |
| *** charcol-v2 has quit IRC | 04:40 | |
| *** ccollett has quit IRC | 04:45 | |
| *** dxiri has joined #openstack-ansible | 05:04 | |
| *** dxiri has quit IRC | 05:08 | |
| *** udesale has joined #openstack-ansible | 05:24 | |
| *** udesale has quit IRC | 05:24 | |
| *** SerenaFeng has joined #openstack-ansible | 05:27 | |
| *** MasterOfBugs has joined #openstack-ansible | 05:40 | |
| *** tobberydberg has joined #openstack-ansible | 05:46 | |
| *** jvidal has joined #openstack-ansible | 05:48 | |
| openstackgerrit | zhongshengping proposed openstack/openstack-ansible-haproxy_server master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477383 | 05:51 |
| openstackgerrit | zhongshengping proposed openstack/openstack-ansible-openstack_hosts master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477387 | 05:56 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_keystone stable/newton: Correct behaviour for serialised execution https://review.openstack.org/477083 | 05:57 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_keystone stable/newton: Split user create and ssh key generation https://review.openstack.org/476717 | 05:58 |
| openstackgerrit | zhongshengping proposed openstack/openstack-ansible-os_freezer master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477388 | 06:01 |
| openstackgerrit | zhongshengping proposed openstack/openstack-ansible-os_molteniron master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477389 | 06:02 |
| openstackgerrit | zhongshengping proposed openstack/openstack-ansible-os_monasca-ui master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477390 | 06:03 |
| openstackgerrit | zhongshengping proposed openstack/openstack-ansible-os_monasca-agent master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477391 | 06:05 |
| openstackgerrit | zhongshengping proposed openstack/openstack-ansible-tests master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477392 | 06:05 |
| openstackgerrit | Merged openstack/openstack-ansible-ops master: Fixed ip confliction. https://review.openstack.org/476954 | 06:07 |
| *** charcol has joined #openstack-ansible | 06:07 | |
| *** vnogin has joined #openstack-ansible | 06:10 | |
| *** Oku_OS-away is now known as Oku_OS | 06:12 | |
| *** MasterOfBugs has quit IRC | 06:13 | |
| *** MasterOfBugs has joined #openstack-ansible | 06:13 | |
| *** vnogin has quit IRC | 06:14 | |
| openstackgerrit | zhongshengping proposed openstack/openstack-ansible-os_freezer master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477388 | 06:19 |
| *** chigang__ has joined #openstack-ansible | 06:22 | |
| openstackgerrit | zhongshengping proposed openstack/openstack-ansible-apt_package_pinning master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477397 | 06:29 |
| openstackgerrit | zhongshengping proposed openstack/openstack-ansible-ceph_client master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477398 | 06:30 |
| openstackgerrit | zhongshengping proposed openstack/openstack-ansible-galera_client master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477399 | 06:30 |
| openstackgerrit | zhongshengping proposed openstack/openstack-ansible-galera_server master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477400 | 06:31 |
| openstackgerrit | zhongshengping proposed openstack/openstack-ansible-lxc_container_create master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477403 | 06:31 |
| openstackgerrit | zhongshengping proposed openstack/openstack-ansible-lxc_hosts master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477404 | 06:32 |
| openstackgerrit | zhongshengping proposed openstack/openstack-ansible-openstack_openrc master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477405 | 06:32 |
| openstackgerrit | zhongshengping proposed openstack/openstack-ansible-ops master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477406 | 06:33 |
| openstackgerrit | zhongshengping proposed openstack/openstack-ansible-os_almanach master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477407 | 06:33 |
| *** charcol has quit IRC | 06:35 | |
| openstackgerrit | zhongshengping proposed openstack/openstack-ansible-pip_install master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477408 | 06:36 |
| *** markus_z has joined #openstack-ansible | 06:38 | |
| openstackgerrit | zhongshengping proposed openstack/openstack-ansible-pip_lock_down master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477411 | 06:42 |
| openstackgerrit | zhongshengping proposed openstack/openstack-ansible-repo_build master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477412 | 06:43 |
| openstackgerrit | zhongshengping proposed openstack/openstack-ansible-plugins master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477413 | 06:43 |
| openstackgerrit | zhongshengping proposed openstack/openstack-ansible-repo_server master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477414 | 06:43 |
| openstackgerrit | zhongshengping proposed openstack/openstack-ansible-rsyslog_client master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477415 | 06:44 |
| openstackgerrit | zhongshengping proposed openstack/openstack-ansible-rsyslog_server master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477416 | 06:44 |
| *** pcaruana has joined #openstack-ansible | 06:45 | |
| openstackgerrit | zhongshengping proposed openstack/openstack-ansible-pip_lock_down master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477411 | 06:46 |
| *** SerenaFeng has quit IRC | 06:47 | |
| *** drifterza has joined #openstack-ansible | 06:47 | |
| *** Neepu has quit IRC | 06:50 | |
| openstackgerrit | zhongshengping proposed openstack/openstack-ansible-plugins master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477413 | 06:52 |
| *** shardy has joined #openstack-ansible | 06:54 | |
| *** shardy has quit IRC | 06:54 | |
| *** HarryHuang has joined #openstack-ansible | 07:01 | |
| *** charcol has joined #openstack-ansible | 07:01 | |
| *** yifei has joined #openstack-ansible | 07:01 | |
| *** hw_wutianwei has joined #openstack-ansible | 07:01 | |
| *** dxiri has joined #openstack-ansible | 07:05 | |
| *** shardy has joined #openstack-ansible | 07:07 | |
| *** dxiri has quit IRC | 07:10 | |
| *** pbandark has joined #openstack-ansible | 07:19 | |
| *** yifei1 has joined #openstack-ansible | 07:39 | |
| *** charcol has quit IRC | 07:39 | |
| *** yifei has quit IRC | 07:39 | |
| *** yifei1 is now known as yifei | 07:39 | |
| *** manheim has joined #openstack-ansible | 07:45 | |
| *** manheim has left #openstack-ansible | 07:45 | |
| *** lkoranda has quit IRC | 07:52 | |
| *** pmannidi has quit IRC | 07:56 | |
| *** lkoranda has joined #openstack-ansible | 07:58 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible stable/ocata: Implement rolling upgrades for keystone https://review.openstack.org/477165 | 07:58 |
| *** pcaruana has quit IRC | 07:58 | |
| *** pmannidi_2 has joined #openstack-ansible | 07:59 | |
| *** pmannidi_2 is now known as pmannidi | 07:59 | |
| *** dxiri has joined #openstack-ansible | 08:07 | |
| *** dxiri has quit IRC | 08:11 | |
| *** pcaruana has joined #openstack-ansible | 08:12 | |
| *** armaan has joined #openstack-ansible | 08:34 | |
| *** pmannidi has quit IRC | 08:40 | |
| *** pmannidi has joined #openstack-ansible | 08:42 | |
| *** rstarmer has joined #openstack-ansible | 08:46 | |
| *** rstarmer has quit IRC | 08:47 | |
| *** MasterOfBugs has quit IRC | 08:47 | |
| *** pmannidi has quit IRC | 08:47 | |
| hw_wutianwei | odyssey4me: hi, there are an issue when run setup-openstack | 08:53 |
| hw_wutianwei | odyssey4me: this is the log: "failed": true, "reason": "the file_name '/opt/openstack-ansible/playbooks/common-tasks/haproxy-endpoint-manage.yml' does not exist, or is not readable"} | 08:54 |
| hw_wutianwei | I use stable/ocata | 08:55 |
| hw_wutianwei | odyssey4me: I either don't find this file in https://github.com/openstack/openstack-ansible/tree/stable/ocata/playbooks/common-tasks | 08:58 |
| *** Martin__ has joined #openstack-ansible | 09:09 | |
| *** tobberyd_ has joined #openstack-ansible | 09:11 | |
| *** tobberydberg has quit IRC | 09:15 | |
| *** electrofelix has joined #openstack-ansible | 09:20 | |
| *** vnogin has joined #openstack-ansible | 09:22 | |
| *** pbandark1 has joined #openstack-ansible | 09:26 | |
| *** pbandark has quit IRC | 09:27 | |
| *** pbandark1 is now known as pbandark | 09:27 | |
| *** Guest58 has joined #openstack-ansible | 09:28 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_nova stable/newton: Perform an atomic policy file change https://review.openstack.org/477467 | 09:30 |
| *** SerenaFeng has joined #openstack-ansible | 09:44 | |
| *** jamesden_ has joined #openstack-ansible | 09:46 | |
| *** jamesdenton has quit IRC | 09:47 | |
| odyssey4me | hw_wutianwei oh dear, that managed to slip through without the required file present | 09:49 |
| odyssey4me | :/ | 09:49 |
| odyssey4me | let me get a patch up for that quickly | 09:49 |
| hw_wutianwei | odyssey4me: thanks, I found this file in master branch so I copy that file to the directory, but I get another issue: please refer to http://paste.openstack.org/show/613694/ | 09:52 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible stable/ocata: Add haproxy-endpoint-manage common task https://review.openstack.org/477470 | 09:53 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible stable/newton: Implement rolling upgrades for cinder https://review.openstack.org/476256 | 09:56 |
| *** tobberyd_ has quit IRC | 09:57 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible stable/ocata: Implement rolling upgrades for keystone https://review.openstack.org/477165 | 09:57 |
| *** tobberydberg has joined #openstack-ansible | 09:57 | |
| *** SerenaFeng has quit IRC | 09:58 | |
| *** SerenaFeng has joined #openstack-ansible | 09:58 | |
| *** tobberydberg has quit IRC | 09:59 | |
| *** tobberydberg has joined #openstack-ansible | 09:59 | |
| odyssey4me | hw_wutianwei the dynamic address fact find method was not changed - have you executed the current configuration successfully before? | 09:59 |
| odyssey4me | do you have an ip address on your storage bridge? | 09:59 |
| hw_wutianwei | odyssey4me: I have assigned ip to br-storage | 10:02 |
| hw_wutianwei | odyssey4me: | 10:02 |
| hw_wutianwei | [root@host4 ~]# ifconfig br-storage | 10:02 |
| hw_wutianwei | br-storage: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 | 10:02 |
| hw_wutianwei | inet 172.16.2.4 netmask 255.255.255.0 broadcast 172.16.2.255 | 10:02 |
| odyssey4me | hw_wutianwei was that recently assigned, or was that assigned before you executed any ansible? | 10:02 |
| hw_wutianwei | before I run any ansible | 10:03 |
| *** Guest58 is now known as totos | 10:03 | |
| totos | hello on ubuntu 16.04 i downgraded my kernel to 3.19 and on first boot i see the containers get no ip , restarting lxcbr0 made no difference. do i need to rerun parts of the role or something else i need to run | 10:04 |
| odyssey4me | hw_wutianwei it appears that you have 3 haproxy hosts, but one cinder service host - are you sure that your host/group mappings are right? can you pastebin them? | 10:06 |
| odyssey4me | totos unlikely - you'll need to look at logs to figure out what's going wrong in the containers to figure out what's next | 10:07 |
| odyssey4me | check the lxc logs on the host first, that may help | 10:07 |
| totos | ok thanks | 10:07 |
| hw_wutianwei | odyssey4me: I do do you mean I paste haproxy configuration, or openstack_user_config.yml | 10:10 |
| hw_wutianwei | odyssey4me: do you mean I paste haproxy configuration, or openstack_user_config.yml? | 10:10 |
| odyssey4me | hw_wutianwei openstack_user_config and any conf.d file entries | 10:11 |
| hw_wutianwei | odyssey4me: this is openstack_user_config http://paste.openstack.org/show/613696, I don't change anything in conf.d | 10:18 |
| *** rstarmer has joined #openstack-ansible | 10:27 | |
| *** stuartgr has joined #openstack-ansible | 10:29 | |
| *** cathrichardson has quit IRC | 10:40 | |
| *** cathrichardson has joined #openstack-ansible | 10:41 | |
| *** rstarmer has quit IRC | 10:42 | |
| *** fergus has quit IRC | 10:43 | |
| *** dixiaoli has joined #openstack-ansible | 10:46 | |
| *** SerenaFeng has quit IRC | 10:51 | |
| *** lkoranda has quit IRC | 10:51 | |
| *** pcaruana has quit IRC | 10:53 | |
| *** SerenaFeng has joined #openstack-ansible | 10:55 | |
| *** HarryHuang has quit IRC | 10:58 | |
| *** lkoranda has joined #openstack-ansible | 10:59 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_nova stable/newton: Perform an atomic policy file change https://review.openstack.org/477467 | 10:59 |
| *** cuongnv has quit IRC | 11:01 | |
| *** dixiaoli has quit IRC | 11:01 | |
| *** drifterza has quit IRC | 11:03 | |
| *** tobberydberg has quit IRC | 11:03 | |
| *** tobberydberg has joined #openstack-ansible | 11:04 | |
| *** pcaruana has joined #openstack-ansible | 11:05 | |
| *** armaan has quit IRC | 11:08 | |
| *** manheim has joined #openstack-ansible | 11:08 | |
| *** tobberydberg has quit IRC | 11:09 | |
| *** tobberydberg has joined #openstack-ansible | 11:10 | |
| *** tobberydberg has quit IRC | 11:11 | |
| *** tobberydberg has joined #openstack-ansible | 11:11 | |
| *** manheim has quit IRC | 11:12 | |
| *** pomac has joined #openstack-ansible | 11:14 | |
| *** tobberydberg has quit IRC | 11:16 | |
| *** tobberydberg has joined #openstack-ansible | 11:16 | |
| *** manheim has joined #openstack-ansible | 11:18 | |
| hw_wutianwei | 2 | 11:18 |
| *** smatzek has joined #openstack-ansible | 11:25 | |
| *** coolkil has joined #openstack-ansible | 11:26 | |
| *** d9k_ has joined #openstack-ansible | 11:26 | |
| *** d9k has quit IRC | 11:28 | |
| *** manheim has left #openstack-ansible | 11:34 | |
| *** hw_wutianwei has quit IRC | 11:38 | |
| *** hw_wutianwei has joined #openstack-ansible | 11:39 | |
| hwoarang | mhayden: hello. do you have 5 min to talk about ansible-hardening? | 11:47 |
| markus_z | coolkil: I would start to push a few patches for s390x support, wanna be the "co-author"? | 11:55 |
| coolkil | yeah wanna help anyway i can | 11:56 |
| coolkil | have been working on serial console support unfortunatly no luck yet | 11:56 |
| markus_z | coolkil: Cool, I'll add you as co-author of the patches I have ATM. | 11:57 |
| markus_z | coolkil: Ah, any issue I might be helpful? | 11:57 |
| coolkil | well wednesday I will be giving it another active attempt | 11:59 |
| markus_z | coolkil: OK, let me know when I can help, I know the feature well(-ish). | 11:59 |
| markus_z | coolkil: Which name + email should I use for the co-author keyword in the commit messages? | 12:00 |
| coolkil | noticed saw youre name in a lot of articles and pages around that feature | 12:00 |
| Martin__ | Hi! I had some problems with haproxy during setup-infrastructure.yml and someone recommended to destroy the containers and also remove /var/cache/lxc. I did so but now setup-hosts.yml fails during [lxc_hosts : Prepare cached image setup commands]. Error message: Temporary failure resolving 'de.archive.ubuntu.com' (and similar for the other servers). | 12:01 |
| Martin__ | The executed command seems to be `chroot /var/lib/lxc/LXC_NAME/rootfs /usr/local/bin/cache-prep-commands.sh` which fails on the control node because /root/repo.keys is not found. But it exists... What is going wrong here? | 12:02 |
| mgariepy | good morning everyone | 12:03 |
| mgariepy | hw_wutianwei, did you find a solution for mariadb issue ? | 12:05 |
| mgariepy | hw_wutianwei, I think you had this one: https://bugs.launchpad.net/openstack-ansible/+bug/1697531 | 12:06 |
| openstack | Launchpad bug 1697531 in openstack-ansible "Galera server fail to start on CentOS." [High,Fix released] - Assigned to Major Hayden (rackerhacker) | 12:06 |
| odyssey4me | Martin__ you'll need to clear out the apt-cacher-proxy config file - see https://docs.openstack.org/developer/openstack-ansible/developer-docs/quickstart-aio.html#rebuilding-an-aio | 12:15 |
| *** woodard has joined #openstack-ansible | 12:16 | |
| odyssey4me | if any cores have some time available, I'd appreciate reviews for https://review.openstack.org/#/q/owner:jesse-pretorius+status:open+branch:stable/ocata and https://review.openstack.org/#/q/owner:jesse-pretorius+status:open+branch:stable/newton | 12:19 |
| *** dxiri has joined #openstack-ansible | 12:20 | |
| Martin__ | odyssey4me I do not have that file. For 00, there are only 00aptitude and 00trustcdrom. | 12:20 |
| odyssey4me | Martin__ no other files with proxy configurations? | 12:21 |
| *** woodard has quit IRC | 12:21 | |
| Martin__ | odyssey4me and this is not an aio, just in case this makes a difference | 12:21 |
| odyssey4me | oh, also /root/repo.keys is copied by the playbook from the host into the container rootfs | 12:22 |
| odyssey4me | did you delete /var/cache/lxc or only its contents? | 12:22 |
| odyssey4me | the same principle applies to cleaning up a multi-node deployment | 12:23 |
| Martin__ | odyssey4me nope, nothing named like cache or similar, and I deleted taht directory | 12:23 |
| odyssey4me | although the best is always to rebuild from the ground up | 12:23 |
| mhayden | morning | 12:23 |
| mhayden | hwoarang: yessir! (just sat down with coffee_ | 12:23 |
| odyssey4me | Martin__ ok, try recreating /var/cache/lxc ? | 12:23 |
| hwoarang | mhayden: ok great. so a few questions starding from the Vagrantfile | 12:24 |
| hwoarang | why do you use a separate vagrant.yml instead of test.yml which is what the CI uses? | 12:25 |
| *** dxiri has quit IRC | 12:25 | |
| odyssey4me | Martin__ hmm, no we recreate /var/cache/lxc/download it seems, so that won't make a difference | 12:25 |
| Martin__ | odyssey4me it is recreated by the playbook | 12:25 |
| mhayden | hwoarang: that's a good question -- i don't use vagrant ;) | 12:25 |
| odyssey4me | ok, then I have no idea - without more debug informaiton it's hard to tell what's going wrong | 12:25 |
| mhayden | might want to do a 'git blame' on that file and see why it might be that way | 12:26 |
| hwoarang | mhayden: ok then i will rework the vagrant thing then since i had some other questions about it. | 12:26 |
| odyssey4me | mhayden whothat was some sort of legacy reason which may no longer be required | 12:26 |
| Martin__ | odyssey4me Want a log? | 12:26 |
| odyssey4me | Martin__ sure, a log of the lxc-hosts playbook with -vvv | 12:26 |
| odyssey4me | pastebin (or similar) | 12:27 |
| hwoarang | mhayden: any objections to replace ansible_os_family with ansible_pkg_mgr for redhat? because ansible_os_family is a bit messed up on suse on old ansible versions | 12:27 |
| hwoarang | like 'ansible_pkg_mgr in ['yum', 'zypper']' on the tasks that are common to both distros | 12:27 |
| Martin__ | odyssey4me can I run that separately from setup-hosts? | 12:28 |
| odyssey4me | Martin__ yes | 12:28 |
| odyssey4me | setup-hosts is just a meta-playbook which includes a series of others | 12:28 |
| mhayden | hwoarang: i'm not opposed to that -- we might need to do tinkering when RHEL 8 comes out, but we should be okay with that for a while | 12:29 |
| hwoarang | ok | 12:29 |
| hwoarang | now, 'aide' | 12:29 |
| mhayden | yay, aide! | 12:29 |
| hwoarang | you use 'aideinit' to init the db. but this command is only available on ubuntu | 12:29 |
| Martin__ | odyssey4me takes a moment | 12:29 |
| hwoarang | so i guess this task may never pass on rhel | 12:29 |
| hwoarang | am i right? | 12:29 |
| hwoarang | at least, i can't see this executable being present in the centos rpm | 12:29 |
| mhayden | right, ubuntu has a special script for that | 12:30 |
| hwoarang | ok will fix that too then for rhel then | 12:30 |
| mhayden | hmm, there was a reason why i did it that way | 12:30 |
| * mhayden tries to remember | 12:30 | |
| hwoarang | i tried to understand how would that work on rhel but i couldn't find if the db is initialized anywhere else | 12:31 |
| hwoarang | so i presume that it's just broken ;p | 12:31 |
| mhayden | ah, for RHEL it's aide --init | 12:31 |
| hwoarang | yes | 12:32 |
| mhayden | hwoarang: that's a bug! | 12:32 |
| hwoarang | true | 12:32 |
| hwoarang | ;p | 12:32 |
| mhayden | i'll open a bug -- want me to assign it to you? | 12:32 |
| mhayden | the RHEL task is just totally missing | 12:32 |
| hwoarang | yeah ok i already have a fix in my branch | 12:32 |
| hwoarang | good | 12:32 |
| hwoarang | that's all i can remember for now... | 12:32 |
| hwoarang | will push that a bit later today for review | 12:33 |
| mhayden | https://bugs.launchpad.net/openstack-ansible/+bug/1700524 | 12:33 |
| openstack | Launchpad bug 1700524 in openstack-ansible "AIDE isn't getting initialized on RHEL/CentOS/Fedora" [High,Confirmed] | 12:33 |
| mhayden | feel free to grab it | 12:34 |
| mhayden | thanks for doing that, too | 12:34 |
| *** tobberydberg has quit IRC | 12:34 | |
| *** tobberydberg has joined #openstack-ansible | 12:35 | |
| *** tobberydberg has quit IRC | 12:36 | |
| *** tobberydberg has joined #openstack-ansible | 12:37 | |
| *** jamesdenton has joined #openstack-ansible | 12:42 | |
| *** jamesden_ has quit IRC | 12:43 | |
| *** SerenaFeng has quit IRC | 12:43 | |
| hwoarang | no problem | 12:45 |
| *** cpuga has joined #openstack-ansible | 12:45 | |
| *** cpuga_ has joined #openstack-ansible | 12:48 | |
| openstackgerrit | Merged openstack/openstack-ansible-apt_package_pinning master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477397 | 12:50 |
| openstackgerrit | Merged openstack/openstack-ansible-ceph_client master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477398 | 12:50 |
| openstackgerrit | Merged openstack/openstack-ansible-lxc_container_create master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477403 | 12:51 |
| *** cpuga has quit IRC | 12:51 | |
| openstackgerrit | Merged openstack/openstack-ansible-ops master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477406 | 12:51 |
| openstackgerrit | Merged openstack/openstack-ansible-galera_client master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477399 | 12:51 |
| openstackgerrit | Merged openstack/openstack-ansible-plugins master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477413 | 12:52 |
| openstackgerrit | Merged openstack/openstack-ansible-galera_server master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477400 | 12:52 |
| openstackgerrit | Merged openstack/openstack-ansible-lxc_hosts master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477404 | 12:52 |
| openstackgerrit | Merged openstack/openstack-ansible-openstack_openrc master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477405 | 12:52 |
| openstackgerrit | Merged openstack/openstack-ansible-os_almanach master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477407 | 12:52 |
| openstackgerrit | Merged openstack/openstack-ansible-pip_install master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477408 | 12:52 |
| openstackgerrit | Merged openstack/openstack-ansible-repo_server master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477414 | 12:52 |
| openstackgerrit | Merged openstack/openstack-ansible-rsyslog_client master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477415 | 12:52 |
| openstackgerrit | Merged openstack/openstack-ansible-rsyslog_server master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477416 | 12:53 |
| openstackgerrit | Merged openstack/openstack-ansible-repo_build master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477412 | 12:53 |
| openstackgerrit | Merged openstack/openstack-ansible-os_aodh master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477320 | 12:54 |
| openstackgerrit | Merged openstack/openstack-ansible-os_ceilometer master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477326 | 12:54 |
| openstackgerrit | Merged openstack/openstack-ansible-os_barbican master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477325 | 12:54 |
| openstackgerrit | Merged openstack/openstack-ansible-os_cloudkitty master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477329 | 12:54 |
| openstackgerrit | Merged openstack/openstack-ansible-os_cinder master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477328 | 12:54 |
| Martin__ | odyssey4me There you go: https://pastebin.com/tJjH3TwU | 12:56 |
| Martin__ | odyssey4me Thanks for having a look :) | 12:56 |
| openstackgerrit | Merged openstack/openstack-ansible-haproxy_server master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477383 | 12:56 |
| openstackgerrit | Merged openstack/openstack-ansible-os_keystone master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477336 | 12:56 |
| openstackgerrit | Merged openstack/openstack-ansible-os_octavia master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477341 | 12:57 |
| openstackgerrit | Merged openstack/openstack-ansible-tests master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477392 | 12:57 |
| openstackgerrit | Merged openstack/openstack-ansible-openstack_hosts master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477387 | 12:57 |
| openstackgerrit | Merged openstack/openstack-ansible-os_designate master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477330 | 12:57 |
| openstackgerrit | Merged openstack/openstack-ansible-os_glance master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477332 | 12:57 |
| openstackgerrit | Merged openstack/openstack-ansible-os_heat master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477334 | 12:57 |
| openstackgerrit | Merged openstack/openstack-ansible-os_magnum master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477338 | 12:57 |
| openstackgerrit | Merged openstack/openstack-ansible-os_monasca master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477340 | 12:58 |
| openstackgerrit | Merged openstack/openstack-ansible-os_monasca-agent master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477391 | 12:58 |
| openstackgerrit | Merged openstack/openstack-ansible-os_nova master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477345 | 12:58 |
| openstackgerrit | Merged openstack/openstack-ansible-os_sahara master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477344 | 12:58 |
| openstackgerrit | Merged openstack/openstack-ansible-os_searchlight master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477346 | 12:58 |
| openstackgerrit | Merged openstack/openstack-ansible-os_swift master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477347 | 12:58 |
| openstackgerrit | Merged openstack/openstack-ansible-os_tempest master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477348 | 12:58 |
| *** lostRhino has joined #openstack-ansible | 12:58 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_gnocchi master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477333 | 12:58 |
| openstackgerrit | Merged openstack/openstack-ansible-os_molteniron master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477389 | 12:59 |
| openstackgerrit | Merged openstack/openstack-ansible-os_monasca-ui master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477390 | 12:59 |
| lostRhino | good morning | 12:59 |
| openstackgerrit | Merged openstack/openstack-ansible-os_neutron master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477339 | 12:59 |
| openstackgerrit | Merged openstack/openstack-ansible-os_rally master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477343 | 12:59 |
| openstackgerrit | Merged openstack/openstack-ansible-os_trove master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477349 | 12:59 |
| openstackgerrit | Merged openstack/openstack-ansible-os_watcher master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477350 | 13:00 |
| odyssey4me | Martin__ Can you check the content of /var/lib/lxc/LXC_NAME/rootfs/etc/resolv.conf ? It should match your host, and should be valid. | 13:00 |
| openstackgerrit | Merged openstack/openstack-ansible-openstack_hosts master: Improve Python 3 compatibility https://review.openstack.org/477021 | 13:00 |
| openstackgerrit | Merged openstack/openstack-ansible-os_zaqar master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477351 | 13:00 |
| lostRhino | under repo_build - while creating the openstack-ansible requirement wheels - is there a way to bypass errors ? I’ve done some googling and cant seem to find a way to get past checking out a detached head state… we’re working off the newton/stable branch … | 13:00 |
| odyssey4me | lostRhino Can you share which errors you're seeing? | 13:01 |
| odyssey4me | maybe a pastebin or something? | 13:01 |
| lostRhino | yes I have to go there…one moment please… loading.... | 13:02 |
| openstackgerrit | Merged openstack/openstack-ansible master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477337 | 13:02 |
| lostRhino | http://paste.openstack.org/show/613718/ | 13:02 |
| Martin__ | odyssey4me nameserver 10.0.3.1, that sound right to me | 13:03 |
| openstackgerrit | Markus Zoeller (markus_z) proposed openstack/openstack-ansible-lxc_hosts master: Add s390x as supported architecture https://review.openstack.org/477526 | 13:03 |
| odyssey4me | Martin__ it should be the host's address on the lxcbr0 interface | 13:03 |
| Martin__ | however the container does not have an IP and is stopped according to lxc-ls -f: LXC_NAME STOPPED 0 - - - | 13:04 |
| odyssey4me | Martin__ and you should be able to communicate through it. | 13:04 |
| odyssey4me | if you can't, then that's the issue | 13:04 |
| openstackgerrit | Markus Zoeller (markus_z) proposed openstack/openstack-ansible-lxc_container_create master: Add s390x as supported architecture https://review.openstack.org/477527 | 13:04 |
| odyssey4me | lostRhino I think https://github.com/openstack/openstack-ansible-repo_build/commit/be660fa134a95ed36fe1c933df5f587c4bf3f298 will actually stop all those horrible log outputs | 13:05 |
| odyssey4me | I'll push a SHA bump for that role to include it in the next release. | 13:05 |
| Martin__ | odyssey4me but yes, thats the problem, ifconfig in container statest that the interface lacks an ip address, and the network is unreachable | 13:05 |
| Martin__ | so back to lxc-config, yeah... | 13:05 |
| *** gkadam has quit IRC | 13:06 | |
| odyssey4me | Martin__ the container is not supposed to be running | 13:06 |
| odyssey4me | the cache prep process chroots into it | 13:06 |
| *** drifterza has joined #openstack-ansible | 13:06 | |
| lostRhino | but will it continue … it just hangs | 13:06 |
| lostRhino | for me | 13:06 |
| odyssey4me | Martin__ if you can just refresh the hosts, do that - it'll save you time | 13:06 |
| odyssey4me | lostRhino yes, at that time it's building packages | 13:07 |
| odyssey4me | those are not errors, just notifications | 13:07 |
| lostRhino | so I need to backport? | 13:07 |
| lostRhino | or can I override now? | 13:07 |
| *** acormier has joined #openstack-ansible | 13:08 | |
| *** cNilesh has quit IRC | 13:11 | |
| lostRhino | odyssey4me: actually…. I was not reading for newton/stable - ok thanks - I’ll see if this fixes our issue | 13:11 |
| *** cNilesh has joined #openstack-ansible | 13:11 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible stable/ocata: Update repo_build SHA to include idempotent wheel build https://review.openstack.org/477528 | 13:12 |
| *** mrch has joined #openstack-ansible | 13:13 | |
| *** lucasxu has joined #openstack-ansible | 13:13 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible stable/newton: Update repo_build SHA to include idempotent wheel build https://review.openstack.org/477531 | 13:14 |
| odyssey4me | lostRhino ^ that's the backport to the integrated repo to get it included in the next release | 13:16 |
| lostRhino | would an offline build be affected by this? | 13:16 |
| *** gouthamr has joined #openstack-ansible | 13:16 | |
| *** jheroux has joined #openstack-ansible | 13:19 | |
| Martin__ | odyssey4me Thanks a million. I will have a look if I can pt the control host in a VM, at least for testing | 13:20 |
| openstackgerrit | Merged openstack/openstack-ansible-repo_build master: Add protection against no local facts being defined https://review.openstack.org/477065 | 13:20 |
| odyssey4me | lostRhino with that patch, a repo_build will not bother doing any build unless requirements have changed | 13:21 |
| *** dxiri has joined #openstack-ansible | 13:21 | |
| odyssey4me | it won't reach out for git updates, for wheels or for venvs unless there's a new tag (and the build has never been done) or requirements have changed | 13:22 |
| *** electrofelix has quit IRC | 13:23 | |
| openstackgerrit | Merged openstack/openstack-ansible-tests stable/newton: Update common repo files https://review.openstack.org/477096 | 13:24 |
| openstackgerrit | Merged openstack/openstack-ansible-tests stable/ocata: Update common repo files https://review.openstack.org/477093 | 13:24 |
| *** gouthamr has quit IRC | 13:26 | |
| *** esberglu has joined #openstack-ansible | 13:26 | |
| *** dxiri has quit IRC | 13:26 | |
| openstackgerrit | Markus Zoeller (markus_z) proposed openstack/openstack-ansible-lxc_container_create stable/ocata: Add s390x as supported architecture https://review.openstack.org/477533 | 13:26 |
| *** cathrich_ has joined #openstack-ansible | 13:27 | |
| *** cNilesh has quit IRC | 13:27 | |
| *** cathrichardson has quit IRC | 13:30 | |
| *** thorst has joined #openstack-ansible | 13:31 | |
| *** cNilesh has joined #openstack-ansible | 13:33 | |
| *** klamath has joined #openstack-ansible | 13:34 | |
| *** klamath has quit IRC | 13:34 | |
| *** klamath has joined #openstack-ansible | 13:35 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_keystone master: Use run_once instead of inventory scoping https://review.openstack.org/477103 | 13:35 |
| *** kylek3h has joined #openstack-ansible | 13:37 | |
| *** cNilesh has quit IRC | 13:38 | |
| *** jwitko has joined #openstack-ansible | 13:41 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible master: [WIP] Add nspawn container driver https://review.openstack.org/477017 | 13:44 |
| odyssey4me | looking for a review for this fix for os_keystone newton: https://review.openstack.org/477083 | 13:44 |
| cloudnull | mornings | 13:44 |
| odyssey4me | o/ | 13:44 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_gnocchi master: Updated from OpenStack Ansible Tests https://review.openstack.org/476570 | 13:44 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_monasca master: Updated from OpenStack Ansible Tests https://review.openstack.org/476577 | 13:45 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_searchlight master: Updated from OpenStack Ansible Tests https://review.openstack.org/476585 | 13:45 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-rabbitmq_server master: Updated from OpenStack Ansible Tests https://review.openstack.org/477538 | 13:45 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-repo_build master: Updated from OpenStack Ansible Tests https://review.openstack.org/476594 | 13:45 |
| Martin__ | odyssey4me I think I found the problem: lxcbr0 is not attached to any interface. It cannot because /etc/network/interfaces.d/lxc-net-bridge.cfg connects it to: bridge_ports none. However, overriding the default none to the proper interface | 13:48 |
| odyssey4me | Martin__ so it sounds like perhaps the routing is busted in LXC | 13:48 |
| Martin__ | still does not resolve the problem. But no adding an interface seems a bit strange to me... | 13:48 |
| odyssey4me | I think there's a script placed on the host to fix that | 13:48 |
| *** acormier has quit IRC | 13:49 | |
| openstackgerrit | Merged openstack/openstack-ansible-lxc_hosts master: Add s390x as supported architecture https://review.openstack.org/477526 | 13:49 |
| *** acormier has joined #openstack-ansible | 13:49 | |
| odyssey4me | Martin__ /usr/local/bin/lxc-system-manage | 13:49 |
| odyssey4me | that's a tool to fix up broken networking on the lxc host if stuff goes wonky | 13:50 |
| *** Martin_____ has joined #openstack-ansible | 13:50 | |
| openstackgerrit | Merged openstack/openstack-ansible-lxc_container_create master: Add s390x as supported architecture https://review.openstack.org/477527 | 13:52 |
| Martin_____ | odyssey4me damn, browser crashed :( | 13:53 |
| *** Martin__ has quit IRC | 13:53 | |
| odyssey4me | Martin_____ all IRC conversations in this channel are logged :) http://eavesdrop.openstack.org/irclogs/%23openstack-ansible/ | 13:54 |
| odyssey4me | it may take a bit of time for the logs to appear, but they will | 13:54 |
| *** smatzek has quit IRC | 13:54 | |
| Martin_____ | odyssey4me yap, I know, I regularly search the logs ;) | 13:55 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-lxc_container_create stable/ocata: Add s390x as supported architecture https://review.openstack.org/477533 | 13:55 |
| odyssey4me | markus_z ^ was that an intentional backport? | 13:56 |
| markus_z | odyssey4me: yes, would be cool to have that | 13:56 |
| markus_z | oh, I did the backport wrong? | 13:57 |
| odyssey4me | markus_z assuming the patches for support are not too onerous, it should be fine | 13:57 |
| odyssey4me | I would guess it'll be small patches like that for the most part. | 13:57 |
| odyssey4me | markus_z yeah, always wait for master to merge before proposing a backport | 13:58 |
| markus_z | odyssey4me: I assume that, yes, but I'm not fully through. | 13:58 |
| odyssey4me | https://docs.openstack.org/developer/openstack-ansible/developer-docs/contribute.html#backporting | 13:58 |
| markus_z | odyssey4me: got it, will be more careful in the future | 13:58 |
| odyssey4me | no worries :) | 13:58 |
| *** esberglu has quit IRC | 13:59 | |
| *** cpuga_ has quit IRC | 14:00 | |
| *** esberglu has joined #openstack-ansible | 14:00 | |
| markus_z | odyssey4me: I'm still not sure how to handle package dependencies like "libmariadbclient-dev" which seems to be in earliest in ubuntu zesty. Got a tip for that? | 14:02 |
| odyssey4me | markus_z we get that from the mariadb repositories | 14:02 |
| odyssey4me | not from ubuntu | 14:03 |
| markus_z | odyssey4me: Is that what "galera_repo_url" is used for? | 14:03 |
| coolkil | yea but the problem is the mariadb repo dont have support for s390x | 14:04 |
| *** esberglu has quit IRC | 14:04 | |
| odyssey4me | coolkil markus_z do any mariadb repositories have s390x packages? | 14:05 |
| markus_z | We're talking about this https://downloads.mariadb.org/mariadb/repositories/# ? | 14:05 |
| odyssey4me | yep | 14:05 |
| openstackgerrit | Merged openstack/openstack-ansible-os_nova stable/ocata: Add reload support to init file https://review.openstack.org/477072 | 14:05 |
| markus_z | I haven't yet found one TBH | 14:06 |
| odyssey4me | hmm, that's unfortunate | 14:06 |
| openstackgerrit | Merged openstack/openstack-ansible master: Change the variable name play_hosts to ansible_play_hosts https://review.openstack.org/473197 | 14:07 |
| openstackgerrit | Merged openstack/openstack-ansible stable/ocata: Implement rolling upgrades for neutron https://review.openstack.org/477100 | 14:07 |
| markus_z | I still have to ask around. Right now I know only the ubuntu zesty sources and this wiki: https://github.com/linux-on-ibm-z/docs/wiki/Building-MariaDB-10.1-on-RHEL-7,-RHEL6,-SLES-12-and-Ubuntu-16.04 | 14:07 |
| coolkil | what would maria db need to get the packages in their repo?? access to s390x vm?? | 14:09 |
| markus_z | the mariadb folks only link to mirros AFAIK, is that right? | 14:09 |
| *** cpuga has joined #openstack-ansible | 14:11 | |
| coolkil | actualy why bother. once ubuntu 18.04 comes out i asume openstack-ansible will move to that. ubuntu 18.04 will have the right mariadb in repo. till that time we need a simple way to add the ubuntu repo and set package pinning. or am I wrong? | 14:11 |
| odyssey4me | coolkil markus_z well, yeah - you can opt to use whatever is in the distro or UCA as an alternative, and push patches to allow disabling of anything that's not available | 14:13 |
| *** totos has quit IRC | 14:14 | |
| *** drifterza has quit IRC | 14:14 | |
| *** smatzek has joined #openstack-ansible | 14:14 | |
| markus_z | My folks here are usually on the conservative side when it comes to distro upgrades, IOW 16.04 will probably used for longer than until 18.04 | 14:15 |
| markus_z | comes out | 14:15 |
| markus_z | I have to think a little how to do that properly | 14:16 |
| coolkil | yeah that is also true.. will those folks also have issues with sneaking in packages from 17.04? | 14:16 |
| *** dxiri has joined #openstack-ansible | 14:16 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible master: [WIP] Add nspawn container driver https://review.openstack.org/477017 | 14:17 |
| markus_z | hm, maybe, I have to discuss it | 14:17 |
| markus_z | I think I'm going to ask Ubuntu folks for their opinion on backports to Xenial | 14:18 |
| *** cNilesh has joined #openstack-ansible | 14:18 | |
| coolkil | that would be the best solution | 14:18 |
| coolkil | i think | 14:18 |
| coolkil | either that or ask the mariadb guys to add s390x support for their repo | 14:19 |
| *** mrch has quit IRC | 14:20 | |
| markus_z | yes, that's the second road I can go | 14:21 |
| *** dxiri has quit IRC | 14:23 | |
| coolkil | for test purposes right now it is fine to use the work arround method of adding the zesty repo and apt-package pinning | 14:24 |
| markus_z | yeah, think so too | 14:24 |
| *** dxiri has joined #openstack-ansible | 14:27 | |
| *** jvidal has quit IRC | 14:30 | |
| *** tobberyd_ has joined #openstack-ansible | 14:31 | |
| *** esberglu has joined #openstack-ansible | 14:32 | |
| *** tobberydberg has quit IRC | 14:34 | |
| *** tobberyd_ has quit IRC | 14:35 | |
| lostRhino | is there anyway we can add a file to any container during setup-inventory.yml (or even a specific container?) | 14:38 |
| openstackgerrit | Dimitrios Markou proposed openstack/openstack-ansible-specs master: Blueprint for Openvswitch with NSH support in OSA https://review.openstack.org/476121 | 14:42 |
| *** chyka has joined #openstack-ansible | 14:43 | |
| *** jvidal has joined #openstack-ansible | 14:44 | |
| *** electrofelix has joined #openstack-ansible | 14:44 | |
| *** Guest58 has joined #openstack-ansible | 14:46 | |
| *** dxiri has quit IRC | 14:48 | |
| odyssey4me | lostRhino yes, hang on | 14:49 |
| odyssey4me | lostRhino you can add to the lxc cache | 14:49 |
| openstackgerrit | Merged openstack/openstack-ansible-os_keystone stable/newton: Correct behaviour for serialised execution https://review.openstack.org/477083 | 14:49 |
| odyssey4me | if the containers are already there, then that won't work for you though | 14:49 |
| *** nollide has joined #openstack-ansible | 14:49 | |
| lostRhino | if its for a fresh install of OSA could we use the LXC cache? | 14:50 |
| odyssey4me | lostRhino to get stuff into the cache, you can add commands to run into lxc_cache_prep_pre_commands or lxc_cache_prep_post_commands | 14:51 |
| odyssey4me | they're run before and after: https://github.com/openstack/openstack-ansible-lxc_hosts/blob/master/vars/ubuntu-16.04.yml#L58-L90 | 14:51 |
| lostRhino | odyssey4me: i think we found it - lxc_container_cache_files ? | 14:52 |
| *** cuongnv has joined #openstack-ansible | 14:52 | |
| odyssey4me | lostRhino alternatively you can use lxc_container_commands which will execute commands when the container is created | 14:52 |
| odyssey4me | lostRhino or yes, to add just files to the cache you can use lxc_container_cache_files | 14:53 |
| lostRhino | can lxc_container_commands be applied to individual containers for exampe “utility”> | 14:53 |
| lostRhino | ? | 14:53 |
| *** dxiri has joined #openstack-ansible | 14:55 | |
| *** woodard has joined #openstack-ansible | 15:03 | |
| openstackgerrit | Merged openstack/openstack-ansible master: Allow keystone playbook to work with limits https://review.openstack.org/477102 | 15:04 |
| openstackgerrit | Merged openstack/openstack-ansible stable/ocata: Add haproxy-endpoint-manage common task https://review.openstack.org/477470 | 15:04 |
| *** Martin_____ has quit IRC | 15:06 | |
| *** vnogin has quit IRC | 15:06 | |
| openstackgerrit | Merged openstack/openstack-ansible stable/ocata: Implement rolling upgrades for keystone https://review.openstack.org/477165 | 15:10 |
| *** mbuil has joined #openstack-ansible | 15:11 | |
| *** marst has joined #openstack-ansible | 15:17 | |
| *** jvidal has quit IRC | 15:19 | |
| *** dxiri has quit IRC | 15:20 | |
| *** jwitko has quit IRC | 15:21 | |
| mbuil | Hey, I have an aio deployment and I wanted to restart the installation. I ran lxc-containers-destroy.yml and then tried to execute setup-everything.yml but I keep getting the problem: "Failed to update apt cache.". Any idea what might be wrong? apt-get update does not work and throws apt-secure problems | 15:22 |
| *** Bico_Fino has joined #openstack-ansible | 15:23 | |
| Bico_Fino | Hello, its safe to use the branch ocata from openstack-ansible github? I’m getting these http://paste.openstack.org/show/613733/ | 15:24 |
| *** dxiri has joined #openstack-ansible | 15:25 | |
| mgariepy | mbuil, you'll need to clear out the apt-cacher-proxy config file - see https://docs.openstack.org/developer/openstack-ansible/developer-docs/quickstart-aio.html#rebuilding-an-aio | 15:29 |
| *** cNilesh has quit IRC | 15:32 | |
| *** coolkil has quit IRC | 15:32 | |
| mbuil | mgariepy: thanks! | 15:33 |
| mgariepy | mbuil, no problem I just copied what odyssey4me has said earlier :D | 15:34 |
| odyssey4me | lostRhino yes, if you use group_vars to apply it to the group alone | 15:34 |
| lostRhino | thank you sir | 15:35 |
| odyssey4me | Bico_Fino have you rebootstrapped ansible recently? | 15:36 |
| Bico_Fino | no odyssey4me | 15:36 |
| odyssey4me | if you update your git repo, you should also execute bootstrap-ansible to pick up the new roles and settings that come with it | 15:36 |
| Bico_Fino | Should I run scripts/bootstrap-ansible.sh again? | 15:36 |
| Bico_Fino | Thanks, going try that | 15:36 |
| odyssey4me | Bico_Fino also, some fixes for stable/ocata just merged - so update to the latest head of stable/ocata | 15:37 |
| *** Oku_OS is now known as Oku_OS-away | 15:37 | |
| *** dxiri has quit IRC | 15:39 | |
| *** dxiri has joined #openstack-ansible | 15:41 | |
| *** lostRhino has left #openstack-ansible | 15:42 | |
| Bico_Fino | odyssey4me: Just ran ./bootstrap-ansible.sh, it don’t create the system_crontab_coordination role inside common-playbooks | 15:42 |
| odyssey4me | Bico_Fino it won't, because that's not where it is | 15:42 |
| Bico_Fino | Should I start over the whole process? I’m on the part setup-openstack.yml | 15:42 |
| Bico_Fino | This is the error: ERROR! the role 'system_crontab_coordination' was not found in /opt/openstack-ansible/playbooks/common-playbooks/roles:/etc/ansible/roles:/root/roles:/opt/openstack-ansible/playbooks/common-playbooks | 15:42 |
| odyssey4me | try running the setup-openstack.yml playbook again | 15:43 |
| Bico_Fino | same error | 15:43 |
| *** nollide has left #openstack-ansible | 15:43 | |
| odyssey4me | hmm, I could have sworn that I had fixed that issue | 15:44 |
| Bico_Fino | The error appears to have been in '/opt/openstack-ansible/playbooks/common-playbooks/cinder.yml': line 116, column 7, but may | 15:44 |
| Bico_Fino | Started today after I did a git pull. :) | 15:46 |
| odyssey4me | Bico_Fino yep, this is why it's not released yet | 15:47 |
| odyssey4me | if you need something stable, please revert to the last published tag | 15:48 |
| Bico_Fino | on it odyssey4me | 15:48 |
| openstackgerrit | Nolan Brubaker proposed openstack/openstack-ansible-os_glance master: Only sync glance database on major versions https://review.openstack.org/475470 | 15:49 |
| *** markus_z has quit IRC | 15:50 | |
| *** dxiri has quit IRC | 15:54 | |
| *** dxiri has joined #openstack-ansible | 15:55 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: Ensure that role path is absolute https://review.openstack.org/477599 | 15:55 |
| odyssey4me | Bico_Fino ^ that should solve the issue - but will require a rebootstrap of ansible | 15:56 |
| Bico_Fino | going try ocata tag odyssey4me | 15:56 |
| Bico_Fino | I think all this time we were using the unstable. | 15:56 |
| odyssey4me | Bico_Fino I'm causing a bit of churn in the stable branches in order to get rolling upgrades implemented... apologies. :) | 15:58 |
| odyssey4me | I think the reward will be worth the pain. | 15:58 |
| Bico_Fino | I’m trying to implement 4 node compute/infra with ceph(4 nodes too) | 15:58 |
| odyssey4me | if any cores are available, assuming https://review.openstack.org/477599 passes we'll need to get it ported back asap. | 15:58 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: Ensure that role path is absolute https://review.openstack.org/477599 | 15:59 |
| *** drifterza has joined #openstack-ansible | 16:00 | |
| *** lkoranda has quit IRC | 16:01 | |
| openstackgerrit | Merged openstack/openstack-ansible-repo_build master: Updated from OpenStack Ansible Tests https://review.openstack.org/476594 | 16:03 |
| *** pcaruana has quit IRC | 16:03 | |
| *** lucasxu has quit IRC | 16:08 | |
| *** armaan has joined #openstack-ansible | 16:08 | |
| *** armaan_ has joined #openstack-ansible | 16:09 | |
| *** lkoranda has joined #openstack-ansible | 16:09 | |
| *** armaan_ has quit IRC | 16:09 | |
| *** klamath has quit IRC | 16:09 | |
| *** armaan has quit IRC | 16:11 | |
| *** dxiri has quit IRC | 16:11 | |
| *** dxiri has joined #openstack-ansible | 16:12 | |
| *** pcaruana has joined #openstack-ansible | 16:14 | |
| *** vnogin has joined #openstack-ansible | 16:15 | |
| *** vnogin has quit IRC | 16:16 | |
| *** armaan has joined #openstack-ansible | 16:22 | |
| *** vnogin has joined #openstack-ansible | 16:26 | |
| *** tobberydberg has joined #openstack-ansible | 16:28 | |
| *** cuongnv has quit IRC | 16:37 | |
| *** klamath has joined #openstack-ansible | 16:38 | |
| *** dxiri has quit IRC | 16:38 | |
| *** lostRhino has joined #openstack-ansible | 16:38 | |
| lostRhino | is there any OSA code that takes care of pydistutils.cfg for the nodes and/or containers? | 16:39 |
| odyssey4me | lostRhino not to my knowledge | 16:41 |
| lostRhino | well | 16:41 |
| lostRhino | there should be | 16:41 |
| lostRhino | :D | 16:41 |
| lostRhino | if pydistutils is not modified lets say under the container - it will continue to point to https://pypi.python.org/pypi not a different mirror specified - | 16:42 |
| lostRhino | when it does the wheel build | 16:42 |
| lostRhino | for example | 16:42 |
| *** nollide has joined #openstack-ansible | 16:42 | |
| odyssey4me | lostRhino but we have other vars to specify which mirrors should be used | 16:43 |
| odyssey4me | for the default and for extras | 16:43 |
| odyssey4me | https://github.com/openstack/openstack-ansible-repo_build/blob/master/defaults/main.yml#L73-L78 | 16:43 |
| lostRhino | we’ve been trying to use the default index (repo_build_pip_default_index) and it does not work for us… the build hangs | 16:44 |
| odyssey4me | oh really? | 16:45 |
| lostRhino | yes | 16:45 |
| lostRhino | which is why my comment above is noted… | 16:45 |
| odyssey4me | if the index correctly setup? | 16:45 |
| *** dxiri has joined #openstack-ansible | 16:45 | |
| lostRhino | yessir | 16:45 |
| odyssey4me | oh weird, I wonder if something broke in pip then | 16:46 |
| lostRhino | we’re using 9.0.1 | 16:46 |
| odyssey4me | I'm not really in a position to verify that right now, but if you can register a bug hopefully I can another time. | 16:47 |
| odyssey4me | or someone else might beat me to it | 16:47 |
| openstackgerrit | Merged openstack/openstack-ansible-os_keystone stable/newton: Split user create and ssh key generation https://review.openstack.org/476717 | 16:47 |
| lostRhino | I will try the extra indexes really quick… | 16:47 |
| *** mbuil has quit IRC | 16:47 | |
| odyssey4me | that definitely works - we use it for all gating | 16:48 |
| lostRhino | extra or index? or both | 16:48 |
| odyssey4me | the extra indexes | 16:48 |
| lostRhino | I’ll tell you in a couple if it works in our environment | 16:49 |
| *** armaan has quit IRC | 16:53 | |
| *** cpuga has quit IRC | 16:54 | |
| lostRhino | its not working sir | 17:01 |
| lostRhino | hangs on aodh | 17:01 |
| lostRhino | repo_build : Create OpenStack-Ansible requirement wheels | 17:01 |
| *** pcaruana has quit IRC | 17:03 | |
| *** dxiri has quit IRC | 17:03 | |
| *** dxiri has joined #openstack-ansible | 17:03 | |
| lostRhino | the error we get is : http://paste.openstack.org/show/613739/ | 17:04 |
| *** schwicht has quit IRC | 17:04 | |
| lostRhino | the repo_index and repo_extra_indexes is defined | 17:05 |
| spotz_ | lostRhino: You guys are doing offline install? | 17:06 |
| lostRhino | yes | 17:07 |
| lostRhino | we only point to internal resources | 17:07 |
| lostRhino | http://paste.openstack.org/show/613740/ | 17:07 |
| spotz_ | internal pypi configed too? | 17:08 |
| lostRhino | yes | 17:08 |
| lostRhino | however whenever we modify the pydistutils.cfg and point to our internal mirrors - everything works fine and nothing hangs | 17:09 |
| lostRhino | more specifically /root/.pydistutils.cfg under the repo container - it starts to do the whole shebang | 17:09 |
| *** lkoranda has quit IRC | 17:09 | |
| lostRhino | the problem is with the command ‘python setup.py egg_info’ | 17:10 |
| spotz_ | Yeah cause it looks like the efg_info isn't even trying your local pypi | 17:10 |
| spotz_ | from that error bit | 17:10 |
| lostRhino | ConnectionError: HTTPSConnectionPool(host='pypi.python.org', port=443): Max retries exceeded with url: /pypi/pip/json (Caused by NewConnectionError('<pip._vendor.requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x7f593d0196d0>: Failed to establish a new connection: [Errno 101] Network is unreachable',)) | 17:12 |
| *** schwicht has joined #openstack-ansible | 17:13 | |
| spotz_ | lostRhino: Ok assuming that you're allowing it to connect to external pypi. reachable from your network or no? Just trying to figure out the whole picture | 17:14 |
| *** pbandark has quit IRC | 17:16 | |
| lostRhino | we are NOT allowing it to route to an external pypi - | 17:16 |
| lostRhino | we have an internal pypi | 17:16 |
| spotz_ | Ok so the issue is that it's not even trying... got it | 17:17 |
| spotz_ | Running on little sleep, slower then normal | 17:17 |
| *** lkoranda has joined #openstack-ansible | 17:17 | |
| lostRhino | its not even getting to the other indexes what-so-ever , the overrides are not setting the correct things that need to be overriden for specific commands | 17:17 |
| spotz_ | lostRhino: Has it ever worked or first time trying? | 17:18 |
| lostRhino | nope - we’ve been troubleshooting for 3 days and finally said f-it .. - we had a similar issue with our deployment host | 17:19 |
| spotz_ | Hrm reading the scrolls looks like odyssey4me has run into something similar in getting. Can you remove the aodh part and see if it'll complete? Might end up being a bug in the one part | 17:22 |
| *** lucasxu has joined #openstack-ansible | 17:24 | |
| odyssey4me | nah, I think aodh is just the first one being done | 17:24 |
| lostRhino | it is | 17:24 |
| lostRhino | it just hangs | 17:24 |
| lostRhino | on aodh trying to get to pypi | 17:24 |
| *** dxiri_ has joined #openstack-ansible | 17:24 | |
| *** dxiri has quit IRC | 17:25 | |
| *** ricardoas has joined #openstack-ansible | 17:26 | |
| spotz_ | bah I was hoping for isolation! | 17:26 |
| lostRhino | please dont isolate yourself!!!! | 17:26 |
| spotz_ | So odyssey4me you broke it?:) | 17:26 |
| odyssey4me | lol | 17:27 |
| spotz_ | heheh I like to isolate parts when troubleshooting. Narrow things down but if it's stopping on the first thing:( | 17:27 |
| odyssey4me | it worked a few series ago | 17:27 |
| spotz_ | We could blame stevelle?:) I miss him:( | 17:28 |
| odyssey4me | I would suggest a bare system with just pip first to test whether using the default index on the CLI works at all for installing a package | 17:28 |
| odyssey4me | if that works, then try a git repo build | 17:28 |
| odyssey4me | if both work, then it must be something in what we're doing | 17:28 |
| odyssey4me | if you can register a bug and put all the info there then that'll be useful | 17:28 |
| lostRhino | Im also wondering if your gate servers are “”truly disconnected | 17:29 |
| odyssey4me | they're not | 17:29 |
| odyssey4me | our gating uses the pypi default, and the local servers as extras | 17:29 |
| lostRhino | the repo container will always build the wheels no matter what if it can reach pypi … | 17:30 |
| odyssey4me | lostRhino yep | 17:30 |
| odyssey4me | so for our environment we're doing the repo build in a CI job, then saving the resulting build into a data set which can be used offline | 17:31 |
| odyssey4me | the CI job is online, and it does the build | 17:31 |
| odyssey4me | the deployments are offline | 17:31 |
| odyssey4me | well, that's the aim - still busy working on the fully offline bit | 17:32 |
| lostRhino | have you heard of pypiserver (https://pypi.python.org/pypi/pypiserver) its a quick and simple pypi server that you can just copy wheels and tar.gz files to - and it will build a simple index | 17:32 |
| lostRhino | its very helpful…. for getting the wheels/pip files you need and serving them up offline | 17:32 |
| odyssey4me | well, our repo server does that already :p | 17:33 |
| *** cpuga has joined #openstack-ansible | 17:38 | |
| openstackgerrit | Miguel Alex Cantu (alextricity25) proposed openstack/openstack-ansible-plugins master: [WIP] Add ignore_none_type to config template https://review.openstack.org/477010 | 17:40 |
| *** cpuga_ has joined #openstack-ansible | 17:43 | |
| *** shardy is now known as shardy_afk | 17:44 | |
| *** cpuga has quit IRC | 17:46 | |
| *** electrofelix has quit IRC | 17:53 | |
| *** dxiri_ has quit IRC | 17:54 | |
| *** dxiri has joined #openstack-ansible | 17:56 | |
| *** tobberydberg has quit IRC | 17:57 | |
| *** tobberydberg has joined #openstack-ansible | 17:57 | |
| *** poopcat has joined #openstack-ansible | 18:00 | |
| *** tobberydberg has quit IRC | 18:02 | |
| *** stuartgr has quit IRC | 18:03 | |
| *** acormier_ has joined #openstack-ansible | 18:14 | |
| *** dxiri has quit IRC | 18:15 | |
| *** dxiri has joined #openstack-ansible | 18:15 | |
| *** craigs has joined #openstack-ansible | 18:15 | |
| *** acormier_ has quit IRC | 18:16 | |
| *** acormier has quit IRC | 18:18 | |
| *** acormier has joined #openstack-ansible | 18:18 | |
| *** dxiri has quit IRC | 18:18 | |
| *** dxiri has joined #openstack-ansible | 18:19 | |
| *** acormier has quit IRC | 18:24 | |
| *** acormier has joined #openstack-ansible | 18:24 | |
| *** lostRhino has left #openstack-ansible | 18:37 | |
| *** toddnni has quit IRC | 18:43 | |
| Bico_Fino | odyssey4me: finished the install | 18:46 |
| Bico_Fino | But when I restart an container it come out without networking | 18:47 |
| Bico_Fino | :( | 18:47 |
| odyssey4me | Bico_Fino check the lxc logs and dmesg/syslog in the container | 18:48 |
| Bico_Fino | networking.service: Failed with result 'timeout'. | 18:49 |
| Bico_Fino | If I attach to the container and run a service networking restart works fine. | 18:50 |
| Bico_Fino | :P | 18:50 |
| openstackgerrit | Miguel Alex Cantu (alextricity25) proposed openstack/openstack-ansible-plugins master: Add ignore_none_type to config template https://review.openstack.org/477010 | 18:53 |
| Bico_Fino | odyssey4me: lxc_conf - conf.c:lxc_delete_network:3072 - Interface "eth1" with index 283 already deleted or existing in different network namespace. | 18:54 |
| Bico_Fino | this is the only odd log | 18:54 |
| odyssey4me | Bico_Fino try using the container management script we put into /usr/local/bin on the lxc hosts? | 18:55 |
| Bico_Fino | inventory-manage.py? | 18:55 |
| Bico_Fino | ./lxc-system-manage | 18:56 |
| Bico_Fino | this one? ^ | 18:56 |
| odyssey4me | yeah, the last one | 18:58 |
| odyssey4me | we found that when containers restart they leave hanging veths | 18:58 |
| odyssey4me | that has a command to clean those up | 18:58 |
| odyssey4me | the interface restart is meant to activate it to fix them | 18:59 |
| Bico_Fino | looks like lxc-system-manage system-rebuild fixed them | 18:59 |
| *** shardy_afk is now known as shardy | 18:59 | |
| Bico_Fino | its safe to do a host reboot? | 18:59 |
| odyssey4me | Bico_Fino if not, we'd like to know! | 19:02 |
| openstackgerrit | Miguel Alex Cantu (alextricity25) proposed openstack/openstack-ansible-plugins master: Add ignore_none_type to config template https://review.openstack.org/477010 | 19:02 |
| Bico_Fino | this issue happened after a host reboot odyssey4me | 19:03 |
| odyssey4me | ok, a bug report will be important - with steps to reproduce | 19:03 |
| Bico_Fino | going try it again | 19:03 |
| *** hybridpollo has joined #openstack-ansible | 19:06 | |
| Bico_Fino | strange, nova hypervisor-list returns empty | 19:08 |
| openstackgerrit | Miguel Alex Cantu (alextricity25) proposed openstack/openstack-ansible-plugins master: Add ignore_none_type to config template https://review.openstack.org/477010 | 19:09 |
| *** smatzek has quit IRC | 19:11 | |
| openstackgerrit | Miguel Alex Cantu (alextricity25) proposed openstack/openstack-ansible-plugins master: Add ignore_none_type to config template https://review.openstack.org/477010 | 19:11 |
| *** hybridpollo has quit IRC | 19:11 | |
| *** hybridpollo has joined #openstack-ansible | 19:11 | |
| openstackgerrit | Miguel Alex Cantu (alextricity25) proposed openstack/openstack-ansible-plugins master: Add ignore_none_type to config template https://review.openstack.org/477010 | 19:12 |
| *** hybridpollo has quit IRC | 19:12 | |
| Bico_Fino | odyssey4me: that explains -> ERROR nova.compute.manager RuntimeError: rbd python libraries not found | 19:13 |
| odyssey4me | ah | 19:16 |
| odyssey4me | do you have all the ceph bits set to ensure that they install the packages? | 19:16 |
| Bico_Fino | inside openstack_user_config.yml and user_variables.yml? | 19:17 |
| Bico_Fino | I think I do. | 19:17 |
| *** acormier has quit IRC | 19:18 | |
| *** acormier has joined #openstack-ansible | 19:18 | |
| *** hybridpollo has joined #openstack-ansible | 19:19 | |
| Bico_Fino | odyssey4me: Looks like the same issue here -> https://bugs.launchpad.net/openstack-ansible/+bug/1655941 | 19:19 |
| openstack | Launchpad bug 1655941 in openstack-ansible "openstack-ansible rbd python libraries not found" [Undecided,Expired] | 19:19 |
| odyssey4me | Bico_Fino user_variables.yml | 19:19 |
| Bico_Fino | yeap | 19:20 |
| Bico_Fino | nova_libvirt_images_rbd_pool: ephemeral-vms | 19:20 |
| *** acormier_ has joined #openstack-ansible | 19:21 | |
| odyssey4me | these are the automated decisions made based on settings: https://github.com/openstack/openstack-ansible/blob/215bb56fb3666446ab34c33eb436d9fd8afeed28/group_vars/all/cinder.yml#L21-L33 | 19:21 |
| *** drifterza has quit IRC | 19:22 | |
| odyssey4me | so cinder will have rbd enabled if 'cinder.volume.drivers.rbd.RBDDriver' is one of the backend drivers | 19:22 |
| Bico_Fino | got that too | 19:22 |
| odyssey4me | glance_default_store will need rbd: https://github.com/openstack/openstack-ansible/blob/77ac88197c223968e845e57706b81af24d6aa443/playbooks/os-glance-install.yml#L76 | 19:22 |
| *** acormier has quit IRC | 19:22 | |
| odyssey4me | or it needs to be an additional store | 19:23 |
| odyssey4me | https://github.com/openstack/openstack-ansible/blob/77ac88197c223968e845e57706b81af24d6aa443/playbooks/os-nova-install.yml#L152 | 19:23 |
| odyssey4me | and nova_libvirt_images_rbd_pool needs to be set | 19:23 |
| odyssey4me | this is a bit complicated because our ceph implementation is a bit raw | 19:23 |
| odyssey4me | it needs work to make it more friendly | 19:23 |
| *** acormier_ has quit IRC | 19:24 | |
| Bico_Fino | When I was using the new versions(git pull) | 19:24 |
| Bico_Fino | didn’t had this issue | 19:24 |
| mgariepy | Bico_Fino, what is your userconfig like ? can you pastebin the ceph bits ? | 19:24 |
| *** acormier has joined #openstack-ansible | 19:24 | |
| Bico_Fino | mgariepy: sure, hold on. | 19:24 |
| mgariepy | and you are on ocata ? right ? | 19:24 |
| odyssey4me | I'm unfortunately not well versed on the ceph setup details. :/ | 19:24 |
| Bico_Fino | ok | 19:24 |
| Bico_Fino | fixed | 19:24 |
| Bico_Fino | ocata | 19:25 |
| Bico_Fino | let me put all on a pastebin | 19:25 |
| odyssey4me | if ocata has it fixed, then potentially there's a patch that can be ported back? | 19:25 |
| Bico_Fino | odyssey4me: http://paste.openstack.org/show/613750/ | 19:26 |
| Bico_Fino | mgariepy: http://paste.openstack.org/show/613750/ | 19:26 |
| Bico_Fino | Had to copy the lib to the virtualenv -> https://bugs.launchpad.net/openstack-ansible/+bug/1655941 | 19:27 |
| openstack | Launchpad bug 1655941 in openstack-ansible "openstack-ansible rbd python libraries not found" [Undecided,Expired] | 19:27 |
| mgariepy | this is the tasks that does it for you :: https://github.com/openstack/openstack-ansible-ceph_client/blob/master/tasks/ceph_install_python_libs.yml | 19:27 |
| odyssey4me | Bico_Fino you shouldn't need to - if you have the settings right then they're linked in | 19:29 |
| mgariepy | Bico_Fino, I do have : nova_ceph_client and nova_ceph_client_uuid | 19:29 |
| mgariepy | same for cinder and glance | 19:29 |
| mgariepy | in my user_variables. | 19:30 |
| *** dxiri has quit IRC | 19:30 | |
| *** ChanServ has quit IRC | 19:30 | |
| Bico_Fino | I see. | 19:30 |
| Bico_Fino | glance worked fine with ceph | 19:30 |
| Bico_Fino | with only that configs | 19:30 |
| Bico_Fino | was able to upload an image | 19:31 |
| mgariepy | no glance_ceph_client_uuid.. | 19:31 |
| mgariepy | but for cinder and nova I do. | 19:31 |
| Bico_Fino | So my user_variables.yml is missing these? | 19:32 |
| mgariepy | https://github.com/openstack/openstack-ansible-ceph_client/blob/master/tasks/ceph_auth.yml#L98-L121 | 19:33 |
| mgariepy | I think so. | 19:33 |
| mgariepy | Bico_Fino, you want to use ceph for ephemeral backend ? | 19:34 |
| *** smatzek has joined #openstack-ansible | 19:34 | |
| Bico_Fino | Yeap | 19:35 |
| *** ChanServ has joined #openstack-ansible | 19:35 | |
| *** card.freenode.net sets mode: +o ChanServ | 19:35 | |
| Bico_Fino | Trying to use ceph | 19:35 |
| *** smatzek has quit IRC | 19:35 | |
| *** smatzek has joined #openstack-ansible | 19:36 | |
| *** NeilHanlon has quit IRC | 19:36 | |
| *** NeilHanlon has joined #openstack-ansible | 19:36 | |
| mgariepy | hmm. maybe i'm wrong with the user and uuid, | 19:42 |
| mgariepy | logan-, would know tho. | 19:42 |
| Bico_Fino | I’m checking inside nova.conf on a host | 19:43 |
| Bico_Fino | the configuration for ceph is there, rbd_secret_uuid and rbd_user | 19:43 |
| mgariepy | ok and : virsh secret-list | 19:44 |
| Bico_Fino | ceph client.cinder secret | 19:45 |
| Bico_Fino | its there | 19:45 |
| mgariepy | ok: python -c 'import rados; print rados.__file__' | 19:46 |
| Bico_Fino | also good | 19:47 |
| Bico_Fino | root@openstack1:~# python -c 'import rados; print rados.__file__' | 19:47 |
| Bico_Fino | > /usr/lib/python2.7/dist-packages/rados.so | 19:47 |
| mgariepy | and is it linked on : /openstack/venv/nova-*/lib/python2.7/site-packages/ ? | 19:48 |
| Bico_Fino | yeap | 19:49 |
| Bico_Fino | But I copied the libs by hand | 19:49 |
| mgariepy | is it a link or a file ? | 19:49 |
| Bico_Fino | file | 19:49 |
| mgariepy | osa links it to the libs. | 19:49 |
| Bico_Fino | going restart the services and try to launch an instance | 19:50 |
| mgariepy | like so : /openstack/venvs/nova-14.2.4/lib/python2.7/site-packages/rados.so -> /usr/lib/python2.7/dist-packages/rados.so | 19:51 |
| mgariepy | https://github.com/openstack/openstack-ansible-ceph_client/blob/stable/ocata/tasks/ceph_install_python_libs.yml#L26-L33 | 19:51 |
| mgariepy | Bico_Fino, you are on Ubuntu ? | 19:51 |
| openstackgerrit | Charles Farquhar proposed openstack/openstack-ansible-repo_build stable/ocata: Honor proxy configuration when cloning git repos https://review.openstack.org/477654 | 19:52 |
| Bico_Fino | mgariepy: Yes, Ubuntu 16.04. | 19:52 |
| *** klamath has quit IRC | 19:53 | |
| openstackgerrit | Charles Farquhar proposed openstack/openstack-ansible-repo_build stable/newton: Honor proxy configuration when cloning git repos https://review.openstack.org/477656 | 19:53 |
| *** vnogin has quit IRC | 19:55 | |
| *** vnogin has joined #openstack-ansible | 19:56 | |
| mgariepy | ok | 19:56 |
| mgariepy | so is your test successful ? | 19:56 |
| Bico_Fino | not yet | 19:56 |
| Bico_Fino | restarting the containers | 19:56 |
| *** tobberydberg has joined #openstack-ansible | 19:58 | |
| mgariepy | I'll take a look at the scroll back tomorrow morning | 19:58 |
| Bico_Fino | sure | 19:59 |
| *** klamath has joined #openstack-ansible | 19:59 | |
| mgariepy | It's weird that you needed to copy the rados/rbd.so file manually and the ceph key was added to libvirt. | 19:59 |
| Bico_Fino | agree. :( | 19:59 |
| mgariepy | maybe you can take a look at your ansible logs to see if ceph_client tasks were executed when you installed your compute nodes. | 20:00 |
| *** vadim_ has joined #openstack-ansible | 20:00 | |
| mgariepy | Bico_Fino, should be in : /openstack/log/ansible-logging/ansible.log (I think) | 20:00 |
| Bico_Fino | good idea, checking. | 20:01 |
| mgariepy | or you can re-run the nova installation. | 20:01 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: [TEST] Revert "Work around bad DNS resolution" https://review.openstack.org/477661 | 20:01 |
| *** marst has quit IRC | 20:01 | |
| mgariepy | have a nice evening. let us know if it works or not. | 20:01 |
| *** marst has joined #openstack-ansible | 20:01 | |
| Bico_Fino | Thanks, I will mgariepy | 20:02 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-tests master: [TEST] Revert "Work around bad DNS resolution" https://review.openstack.org/477662 | 20:02 |
| *** vnogin has quit IRC | 20:02 | |
| Bico_Fino | ceph_install_python_libs.yml only ran for glance mgariepy ! | 20:03 |
| *** vnogin has joined #openstack-ansible | 20:04 | |
| *** shardy has quit IRC | 20:22 | |
| vadim_ | i'm haveing issues with nova-compute in a container. | 20:33 |
| vadim_ | 1st systemd-machined failed to start: Failed at step NAMESPACE spawning /usr/lib/systemd/systemd-machined: Invalid argument | 20:34 |
| *** schwicht has quit IRC | 20:35 | |
| *** schwicht has joined #openstack-ansible | 20:36 | |
| vadim_ | do you recommend to run nova-compute on metal or container? | 20:40 |
| *** schwicht has quit IRC | 20:45 | |
| odyssey4me | vadim_ we put it directly onto the host because some things it does don't work in a container | 20:47 |
| odyssey4me | for example, iscsi connectivity | 20:47 |
| vadim_ | i do not use iscsi, I use ceph. | 20:48 |
| *** dxiri has joined #openstack-ansible | 20:48 | |
| vadim_ | if i change PrivateDevice=no for mahined service, machined can start, so libvirtd will not fail on this. | 20:49 |
| vadim_ | however livirt will fail with libvirtError: cannot set CPU affinit y on process 941: Operation not permitted | 20:49 |
| odyssey4me | vadim_ sure, if you're doing ironic or lxc hypervisors then it definitely works | 20:49 |
| *** jheroux has quit IRC | 20:49 | |
| odyssey4me | for kvm as a hypervisor, though, it won't with the defaults we have | 20:50 |
| odyssey4me | this may help you along the way: https://www.google.co.uk/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&cad=rja&uact=8&ved=0ahUKEwiIsLrJsdzUAhWGJlAKHR9HD_IQFggqMAA&url=https%3A%2F%2Fcloudnull.io%2F2016%2F11%2Fmulti-hypervisor-openstack-cloud%2F&usg=AFQjCNElamU3jTxZ1RqsTYIb3pKlecMs4Q | 20:50 |
| odyssey4me | oops, https://cloudnull.io/2016/11/multi-hypervisor-openstack-cloud/ | 20:50 |
| cloudnull | ++ | 20:51 |
| odyssey4me | bbiab | 20:51 |
| *** vnogin has quit IRC | 20:57 | |
| *** vnogin has joined #openstack-ansible | 20:57 | |
| Bico_Fino | odyssey4me: looks like I found my issue | 20:59 |
| Bico_Fino | nova-placement-api is not running | 21:00 |
| *** DimGR has joined #openstack-ansible | 21:01 | |
| Bico_Fino | http://paste.openstack.org/show/613755/ | 21:02 |
| odyssey4me | Bico_Fino that's Ocata only though - I thought you were running Newton? | 21:02 |
| *** vnogin has quit IRC | 21:02 | |
| Bico_Fino | Should I stick with Ocata or go for Newton? | 21:02 |
| odyssey4me | Ocata should be fine, it deployed in the periodic this morning | 21:02 |
| odyssey4me | any of the tags | 21:03 |
| odyssey4me | well, the recent tags | 21:03 |
| odyssey4me | do you have your ansible logs for it? it's weird that the uwsgi file wasn't placed | 21:03 |
| Bico_Fino | I’m using 15.1.5 | 21:03 |
| odyssey4me | try re-running os-nova-install.yml --tags nova-config | 21:03 |
| Bico_Fino | ok | 21:04 |
| odyssey4me | what SHA is the role at? | 21:04 |
| odyssey4me | check ansible-role-requirements | 21:04 |
| Bico_Fino | running | 21:05 |
| Bico_Fino | interesting! | 21:07 |
| Bico_Fino | with os-nova-install.yml —tags nova-config it created the file | 21:07 |
| odyssey4me | hmm, that may mean that another task for that host failed in the earlier run | 21:07 |
| *** woodard_ has joined #openstack-ansible | 21:07 | |
| odyssey4me | I think we should probably enable any failure being a blanket failure | 21:08 |
| odyssey4me | especially on the control plane | 21:08 |
| Bico_Fino | Going try spin an instance now | 21:08 |
| odyssey4me | this is a test environment, right? | 21:08 |
| Bico_Fino | yes sir | 21:08 |
| odyssey4me | yeah, it's plausible that the tes/fail/retest is causing some trouble | 21:08 |
| odyssey4me | once you have a working config, it's good to refresh the hosts and get it all back to zero, then retry | 21:09 |
| odyssey4me | a good host kick system is your friend :) | 21:09 |
| odyssey4me | as is a good test & staging environment for better testing before shifting changes to prod | 21:09 |
| Bico_Fino | something else now. :P | 21:10 |
| Bico_Fino | No valid host was found. There are not enough hosts available. | 21:10 |
| *** woodard has quit IRC | 21:11 | |
| *** vnogin has joined #openstack-ansible | 21:11 | |
| Bico_Fino | finally some good logs | 21:12 |
| Bico_Fino | Failed to create resource provider record in placement API for UUID | 21:12 |
| *** woodard_ has quit IRC | 21:12 | |
| *** kevinbenton has quit IRC | 21:12 | |
| odyssey4me | that may be normal | 21:12 |
| odyssey4me | the placement api enrollment is a bit batshit | 21:12 |
| odyssey4me | you may need to restart nova-compute | 21:12 |
| Bico_Fino | let me try | 21:13 |
| odyssey4me | then these need to run: https://github.com/openstack/openstack-ansible-os_nova/blob/17dd2dd8c53897ce839f283fecb48b8c80410b07/tasks/nova_db_setup.yml | 21:13 |
| Bico_Fino | How do I run this specific task? | 21:14 |
| *** perniciouscaffei has joined #openstack-ansible | 21:14 | |
| *** kevinbenton has joined #openstack-ansible | 21:15 | |
| *** dxiri has quit IRC | 21:15 | |
| openstackgerrit | Merged openstack/openstack-ansible-plugins master: Add ignore_none_type to config template https://review.openstack.org/477010 | 21:15 |
| odyssey4me | they're all tagged 'nova-db-setup', so try using that tag | 21:15 |
| Bico_Fino | running | 21:16 |
| Bico_Fino | don’t need to restart nova containers?(conductor, api, etc) | 21:17 |
| *** dixiaoli has joined #openstack-ansible | 21:17 | |
| *** smatzek has quit IRC | 21:18 | |
| odyssey4me | Bico_Fino restarting containers is almost never necessary | 21:18 |
| odyssey4me | this is not windows ;) | 21:18 |
| Bico_Fino | :P | 21:18 |
| Bico_Fino | same error | 21:19 |
| Bico_Fino | ERROR nova.scheduler.client.report [req-35442bd6-bc3d-47e9-bc3b-5f892dd941bf - - - - -] Failed to create resource provider record in placement API for UUID a08b96b1-96ac-4c08-a925-128b6055df8e. Got 503: <html><body><h1>503 Service Unavailable</h1> | 21:19 |
| odyssey4me | hmm, and the api service - is it running? | 21:19 |
| *** ThiagoCMC has quit IRC | 21:19 | |
| Bico_Fino | placement? | 21:19 |
| odyssey4me | in the logs - do you see anything useful? | 21:19 |
| *** acormier has quit IRC | 21:19 | |
| odyssey4me | sorry - placement | 21:19 |
| odyssey4me | andymccr knows the placement service better than most of us, and logan- does a bit | 21:19 |
| Bico_Fino | checking placement container | 21:20 |
| *** acormier has joined #openstack-ansible | 21:20 | |
| odyssey4me | I doubt andymccr is in - he's at openstack days in germany right now. | 21:20 |
| Bico_Fino | hum | 21:21 |
| Bico_Fino | not a single hit on placement | 21:21 |
| Bico_Fino | checked the 3 containers | 21:21 |
| *** dixiaoli has quit IRC | 21:21 | |
| odyssey4me | the placement containers? | 21:21 |
| odyssey4me | are there placement containers? | 21:21 |
| Bico_Fino | yes | 21:22 |
| Bico_Fino | 3 | 21:22 |
| odyssey4me | ok, hmm | 21:22 |
| odyssey4me | is the wsgi file there? | 21:22 |
| odyssey4me | do the logs show anything? | 21:22 |
| Bico_Fino | only that it started | 21:22 |
| odyssey4me | do you have debug logging enabled? | 21:22 |
| openstackgerrit | Charles Farquhar proposed openstack/openstack-ansible-os_swift stable/ocata: Add support for swift3 middleware https://review.openstack.org/477676 | 21:22 |
| Bico_Fino | I think it has something with haproxy | 21:22 |
| odyssey4me | yep, good idea to check that haproxy thinks it's up | 21:23 |
| odyssey4me | ok, so IIRC the api service is a uwsgi service that's reverse proxied by nginx | 21:24 |
| odyssey4me | check whether nginx is running in the container? | 21:24 |
| *** acormier has quit IRC | 21:24 | |
| Bico_Fino | yes | 21:24 |
| Bico_Fino | it is | 21:24 |
| Bico_Fino | the endpoint that its using is broken | 21:25 |
| Bico_Fino | <html><body><h1>503 Service Unavailable</h1> | 21:25 |
| Bico_Fino | just did a openstack endpoint list | 21:25 |
| odyssey4me | ah ok | 21:25 |
| odyssey4me | does haproxy think it's up? | 21:25 |
| odyssey4me | and does nginx think that the uwsgi service is up? | 21:26 |
| odyssey4me | and does the URL for the endpoint look right in the service catalog | 21:26 |
| Bico_Fino | server infra1_nova_api_placement_container-374796a1 10.20.20.219:8780 check port 8780 inter 12000 rise 3 fall 3 | 21:26 |
| odyssey4me | does that look right? | 21:26 |
| Bico_Fino | no | 21:26 |
| odyssey4me | hmm, ok | 21:27 |
| Bico_Fino | inside the container | 21:27 |
| Bico_Fino | tcp 0 0 127.0.0.1:8781 0.0.0.0:* LISTEN 3076/uwsgi | 21:27 |
| Bico_Fino | see ^? | 21:27 |
| openstackgerrit | Merged openstack/openstack-ansible-repo_build stable/newton: Honor proxy configuration when cloning git repos https://review.openstack.org/477656 | 21:27 |
| odyssey4me | re-run the haproxy playbook? | 21:27 |
| openstackgerrit | Merged openstack/openstack-ansible-repo_build stable/ocata: Honor proxy configuration when cloning git repos https://review.openstack.org/477654 | 21:27 |
| Bico_Fino | let me check the nginx config inside the container | 21:27 |
| odyssey4me | it's unfortunate that openstack has so, so many moving parts | 21:28 |
| Bico_Fino | yeah, and the generic log errors are the best | 21:28 |
| odyssey4me | lol | 21:29 |
| odyssey4me | I have nightmares about the moving parts sometimes. | 21:29 |
| Bico_Fino | nginx is fine | 21:29 |
| Bico_Fino | just needed a restart!! | 21:29 |
| odyssey4me | ah yes | 21:29 |
| Bico_Fino | nginx is 8780 | 21:30 |
| Bico_Fino | placement is 8781 | 21:30 |
| odyssey4me | I've seen the web server sometimes mark a reverse proxied service as down for far too long. | 21:30 |
| odyssey4me | We should perhaps change the timouts/check times. | 21:30 |
| openstackgerrit | Charles Farquhar proposed openstack/openstack-ansible-os_swift stable/ocata: Add support for swift3 middleware https://review.openstack.org/477682 | 21:30 |
| Bico_Fino | looks good now | 21:30 |
| Bico_Fino | trying again | 21:31 |
| Bico_Fino | ;) | 21:31 |
| odyssey4me | we'll be revisiting the nginx/uwsgi design implementation for Queens, I think, seeing as more services are switching to uwsgi now. | 21:31 |
| odyssey4me | Also, apparently using a web server in front of uWSGI gives more than 10x a performance improvement. | 21:31 |
| Bico_Fino | oh | 21:31 |
| Bico_Fino | build! | 21:31 |
| odyssey4me | excellent :) | 21:31 |
| Bico_Fino | Block Device Mapping | 21:31 |
| *** esberglu has quit IRC | 21:32 | |
| Bico_Fino | Building! | 21:32 |
| Bico_Fino | ceph part is ok! | 21:32 |
| odyssey4me | brilliant! | 21:32 |
| *** esberglu has joined #openstack-ansible | 21:32 | |
| Bico_Fino | and... | 21:33 |
| Bico_Fino | No valid host was found. There are not enough hosts available | 21:33 |
| odyssey4me | hmm, ok | 21:33 |
| odyssey4me | check that the compute host is registered | 21:33 |
| odyssey4me | and check whether it's mapped to the cell | 21:33 |
| odyssey4me | maybe do that db registration bit again | 21:33 |
| Bico_Fino | ERROR nova.compute.manager [instance: 1f8a3184-acf2-4271-a888-70395f2bde1f] TypeError: Argument must be bytes or unicode, got 'NoneType' | 21:33 |
| Bico_Fino | I seem this before. | 21:33 |
| odyssey4me | unfortunately from ocata onwards, every host has to be in a cell | 21:34 |
| odyssey4me | and you can't rgister the host into the cell until it's up, and the placement service is up | 21:34 |
| odyssey4me | so now that the placement service is up, the host needs to get in there | 21:34 |
| odyssey4me | that error though - so descriptive | 21:34 |
| Bico_Fino | I think is cinder related | 21:35 |
| odyssey4me | ah, could be | 21:35 |
| odyssey4me | is cinder-volume up? | 21:35 |
| odyssey4me | and does it know about its back-ends - no log errors | 21:35 |
| Bico_Fino | you mean the service? | 21:36 |
| Bico_Fino | it shows on service list -> cinder | volume | | 21:36 |
| *** tobberydberg has quit IRC | 21:37 | |
| *** tobberydberg has joined #openstack-ansible | 21:37 | |
| *** esberglu has quit IRC | 21:37 | |
| *** tobberydberg has quit IRC | 21:38 | |
| odyssey4me | Bico_Fino no, I mean the service in the container | 21:38 |
| odyssey4me | not in the catalog | 21:39 |
| odyssey4me | the catalog entry of backends may have something to do with it | 21:39 |
| *** tobberydberg has joined #openstack-ansible | 21:39 | |
| Bico_Fino | looks up odyssey4me | 21:39 |
| odyssey4me | but first veryify that the service is up and that there are no errors in the log | 21:39 |
| Bico_Fino | ops | 21:40 |
| Bico_Fino | Jun 26 12:48:32 ceph1 cinder-volume[36611]: IOError: [Errno 2] No such file or directory: '/var/log/cinder/cinder-volume.log' | 21:40 |
| odyssey4me | hmm, odd | 21:40 |
| odyssey4me | is cinder-volume in a container or on a host | 21:40 |
| Bico_Fino | host | 21:40 |
| odyssey4me | this should be setting that up: https://github.com/openstack/openstack-ansible/blob/15.1.5/playbooks/os-cinder-install.yml#L65-L69 | 21:41 |
| *** lucasxu has quit IRC | 21:41 | |
| odyssey4me | try running the os-cinder-install.yml playbook again? | 21:42 |
| Bico_Fino | trying | 21:42 |
| odyssey4me | if it creates that folder then you'll likely need to restart the service | 21:42 |
| Bico_Fino | running | 21:43 |
| *** tobberydberg has quit IRC | 21:43 | |
| *** tobberydberg has joined #openstack-ansible | 21:43 | |
| *** thorst has quit IRC | 21:44 | |
| *** thorst has joined #openstack-ansible | 21:46 | |
| Bico_Fino | done | 21:47 |
| Bico_Fino | trying again | 21:48 |
| *** tobberydberg has quit IRC | 21:48 | |
| odyssey4me | for future reference, you can put cinder-volume into a container for ceph: https://docs.openstack.org/developer/openstack-ansible-os_cinder/configure-cinder.html#configuring-cinder-to-use-ceph | 21:48 |
| odyssey4me | however, that will not work for *any* iSCSI back-ends | 21:48 |
| odyssey4me | and I've also heard about trouble using cinder-backup, but that's not fully confirmed | 21:49 |
| Bico_Fino | Scheduling | 21:49 |
| Bico_Fino | new status. :P | 21:49 |
| Bico_Fino | and... | 21:50 |
| Bico_Fino | Error: No valid host was found. There are not enough hosts available | 21:50 |
| odyssey4me | argh, scheduler logs are the worst | 21:50 |
| odyssey4me | but with a debug log it should show the hosts its considering - hopefully | 21:50 |
| *** thorst has quit IRC | 21:50 | |
| odyssey4me | if you look at the logs you should see the check-ins and what they're showing they have. | 21:51 |
| Bico_Fino | nova-scheduler now | 21:51 |
| Bico_Fino | right | 21:51 |
| odyssey4me | it could be something like a pool with no space, or a back-end not showing up as ready to use. | 21:51 |
| Bico_Fino | no errors on compute node | 21:51 |
| odyssey4me | well, it's either cinder, neutron or nova | 21:52 |
| odyssey4me | if the nova-compute agent is registered and assigned to a cell, then that should be fine | 21:52 |
| odyssey4me | if the neutron lb agent is registered, itll be fine as long as the subnet is running and allocated to a running compute node | 21:52 |
| odyssey4me | I have, in the past, edited the code to get more debug info - but I'm not sure if that's necessary these days | 21:53 |
| Bico_Fino | trying to find the log | 21:53 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible master: [WIP] Add nspawn container driver https://review.openstack.org/477017 | 21:53 |
| odyssey4me | I used to run an Essex/Folsom/Grizzly set of clusters. | 21:53 |
| odyssey4me | are there any back-ends registered for cinder now? | 21:54 |
| odyssey4me | volume types | 21:54 |
| openstackgerrit | Merged openstack/openstack-ansible-os_glance master: Only sync glance database on major versions https://review.openstack.org/475470 | 21:54 |
| Bico_Fino | how do I check that? | 21:54 |
| odyssey4me | https://github.com/openstack/openstack-ansible-os_cinder/blob/stable/ocata/tasks/cinder_backends.yml | 21:55 |
| odyssey4me | so it looks like doing 'cinder type-list' | 21:55 |
| Bico_Fino | | 5627c318-db6c-4a7a-9d5a-cc98927c86e8 | rbd | - | True | | 21:56 |
| odyssey4me | then maybe 'type show' on the item | 21:56 |
| odyssey4me | to verify that it's assigned right | 21:56 |
| odyssey4me | to the volume_backend_name | 21:56 |
| Bico_Fino | | extra_specs | {'volume_backend_name': 'rbd'} | | 21:56 |
| *** nollide has quit IRC | 21:57 | |
| odyssey4me | can you create volumes outside of nova? | 21:57 |
| Bico_Fino | | name | rbd | | 21:57 |
| *** esberglu has joined #openstack-ansible | 21:57 | |
| odyssey4me | just cinder on its own | 21:57 |
| Bico_Fino | I think we passed this part | 21:57 |
| Bico_Fino | because I don’t got the None error again | 21:57 |
| Bico_Fino | The instance has an ID -> 74ef1eaa-587c-40ad-9104-6812661c09d2 | 21:58 |
| Bico_Fino | it’s safe to grep logs with this ID? | 21:58 |
| *** esberglu has quit IRC | 21:58 | |
| *** esberglu has joined #openstack-ansible | 21:58 | |
| Bico_Fino | neutron logs looks good: [{"status": "completed", "code": 200, "name": "network-changed", "server_uuid": "74ef1eaa-587c-40ad-9104-6812661c09d2"}]} | 21:59 |
| odyssey4me | you can try | 22:00 |
| odyssey4me | bear in mind that you have an rsyslog container which should contain all logs | 22:00 |
| Bico_Fino | ah | 22:00 |
| Bico_Fino | found the error | 22:00 |
| Bico_Fino | u"RescheduledException: Build of instance 74ef1eaa-587c-40ad-9104-6812661c09d2 was re-scheduled: Argument must be bytes or unicode, got 'NoneType'\n" | 22:00 |
| odyssey4me | also, in /openstack/logs on every container host you'll find all the container logs | 22:00 |
| *** nollide has joined #openstack-ansible | 22:00 | |
| odyssey4me | hmm, have you verified that the host is assigned to a cell? | 22:01 |
| Bico_Fino | how I do that? | 22:01 |
| *** kylek3h has quit IRC | 22:01 | |
| odyssey4me | honestly, I have no idea | 22:02 |
| *** kylek3h has joined #openstack-ansible | 22:02 | |
| *** acormier has joined #openstack-ansible | 22:02 | |
| odyssey4me | the clues are here: https://github.com/openstack/openstack-ansible-os_nova/blob/17dd2dd8c53897ce839f283fecb48b8c80410b07/tasks/nova_db_setup.yml | 22:03 |
| odyssey4me | maybe there's a way to verify that the cell is there | 22:03 |
| Bico_Fino | going check it out | 22:03 |
| Bico_Fino | thanks for all the help odyssey4me | 22:03 |
| *** nollide has left #openstack-ansible | 22:04 | |
| *** acormier has quit IRC | 22:04 | |
| *** thorst has joined #openstack-ansible | 22:05 | |
| *** acormier has joined #openstack-ansible | 22:05 | |
| *** kylek3h has quit IRC | 22:06 | |
| *** thorst has quit IRC | 22:09 | |
| odyssey4me | Bico_Fino It's been a pleasure. I've learned, or refreshed a few things in my mind. :) | 22:13 |
| *** cpuga_ has quit IRC | 22:14 | |
| *** mrhillsman is now known as codebauss | 22:22 | |
| *** codebauss is now known as mrhillsman | 22:22 | |
| *** marst has quit IRC | 22:32 | |
| *** marst has joined #openstack-ansible | 22:35 | |
| *** chyka_ has joined #openstack-ansible | 22:44 | |
| *** chyka has quit IRC | 22:46 | |
| *** acormier has quit IRC | 22:55 | |
| *** acormier has joined #openstack-ansible | 22:55 | |
| *** acormier has quit IRC | 23:00 | |
| *** marst has quit IRC | 23:13 | |
| *** acormier has joined #openstack-ansible | 23:15 | |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/openstack-ansible-tests master: gen-projects-list.sh: Add ability to include extra OSA projects https://review.openstack.org/477706 | 23:17 |
| *** drifterza has joined #openstack-ansible | 23:19 | |
| *** drifterza has quit IRC | 23:23 | |
| *** klamath has quit IRC | 23:30 | |
| *** vnogin has quit IRC | 23:32 | |
| *** marst has joined #openstack-ansible | 23:44 | |
| *** charcol has joined #openstack-ansible | 23:46 | |
| *** perniciouscaffei has quit IRC | 23:50 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible master: [WIP] Add nspawn container driver https://review.openstack.org/477017 | 23:52 |
| *** acormier has quit IRC | 23:53 | |
| *** acormier has joined #openstack-ansible | 23:53 | |
| *** acormier has quit IRC | 23:58 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!