| *** tosky has quit IRC | 00:16 | |
| *** cshen has joined #openstack-ansible | 00:31 | |
| *** cshen has quit IRC | 00:36 | |
| openstackgerrit | zhongshengping proposed openstack/openstack-ansible-tests master: Dissuade .gitignore references to personal tools https://review.openstack.org/609229 | 00:43 |
|---|---|---|
| *** jonher_ has joined #openstack-ansible | 00:50 | |
| *** jonher has quit IRC | 00:53 | |
| *** jonher_ is now known as jonher | 00:53 | |
| openstackgerrit | Guilherme Steinmuller Pimentel proposed openstack/openstack-ansible-os_placement master: [WIP] Create base files to install placement https://review.openstack.org/618820 | 01:07 |
| *** pabelanger has joined #openstack-ansible | 01:23 | |
| pabelanger | o/ | 01:24 |
| pabelanger | is /opt/openstack-ansible a hard requirement (location of git repo) or just standard way for documentation? | 01:24 |
| *** hw_wutianwei_ has joined #openstack-ansible | 01:36 | |
| *** aedc has quit IRC | 01:45 | |
| *** nyloc has quit IRC | 03:17 | |
| *** nyloc has joined #openstack-ansible | 03:18 | |
| *** maddtux has joined #openstack-ansible | 03:28 | |
| *** udesale has joined #openstack-ansible | 03:54 | |
| openstackgerrit | Merged openstack/ansible-hardening master: fix tox python3 overrides https://review.openstack.org/579778 | 04:19 |
| *** udesale has quit IRC | 04:31 | |
| *** ivve has joined #openstack-ansible | 04:31 | |
| *** chandan_kumar has joined #openstack-ansible | 05:26 | |
| *** chandan_kumar is now known as chkumar|ruck | 05:28 | |
| *** chhagarw has joined #openstack-ansible | 06:10 | |
| *** e-vad has quit IRC | 06:40 | |
| *** faizy98 has joined #openstack-ansible | 06:51 | |
| *** amab has joined #openstack-ansible | 06:57 | |
| *** ahosam has joined #openstack-ansible | 07:02 | |
| amab | Hi, During the AIO installation follwoing error occured: | 07:03 |
| amab | TASK [os_nova : Register libvirt-python files] ********************************* ******************************************************************************** ***************************************************************************** Sunday 25 November 2018 16:29:21 +0330 (0:00:00.356) 0:07:11.339 ******* fatal: [aio1]: FAILED! => {"changed": false, "cmd": | 07:04 |
| amab | "dpkg -L python-libvirt | grep '^/usr/local/lib/python2.7/dist-packages/'", | 07:05 |
| amab | during "openstack-ansible setup-openstack.yml " command | 07:06 |
| amab | "python-libvirt" does not include /usr/local/lib/python2.7/dist-packages/ directory and it is normal, it should check /usr/lib/python2.7/dist-packages probably!! | 07:08 |
| *** jawad_axd has joined #openstack-ansible | 07:16 | |
| *** ahosam has quit IRC | 07:16 | |
| *** pcaruana has joined #openstack-ansible | 07:23 | |
| *** mma has joined #openstack-ansible | 07:43 | |
| *** DanyC_ has quit IRC | 08:00 | |
| *** DanyC has joined #openstack-ansible | 08:01 | |
| *** DanyC has quit IRC | 08:05 | |
| *** gkadam has joined #openstack-ansible | 08:08 | |
| *** chhagarw has quit IRC | 08:18 | |
| *** priteau has joined #openstack-ansible | 08:22 | |
| *** cshen has joined #openstack-ansible | 08:24 | |
| *** shardy has joined #openstack-ansible | 08:25 | |
| cshen | my patch needs more review, https://review.openstack.org/#/c/581248/ thanks. | 08:25 |
| *** tosky has joined #openstack-ansible | 08:38 | |
| jrosser | amab: which release of OSA are you using? | 08:45 |
| amab | jrosser stable/queens | 08:45 |
| jrosser | amab take a look at the code here https://github.com/openstack/openstack-ansible-os_nova/blob/stable/queens/tasks/drivers/kvm/nova_compute_kvm_install.yml#L79-L95 | 08:54 |
| jrosser | the first task looks for where the libvirt-python module is on your system, and from your error it seems that it was found in /usr/local/lib/python2.7/....... | 08:55 |
| jrosser | so i think that is the problem, that somehow you have those python modules in /usr/local rather than /usr/lib | 08:56 |
| *** hamzaachi has joined #openstack-ansible | 08:59 | |
| *** gillesMo has joined #openstack-ansible | 09:06 | |
| *** udesale has joined #openstack-ansible | 09:09 | |
| *** ahosam has joined #openstack-ansible | 09:15 | |
| *** jackivanov has joined #openstack-ansible | 09:16 | |
| amab | jrosser: Right, when I install python-libvirt manually (by apt) and run dpkg -L python-libvirt, the real directory would be /usr/lib/python2.7/.... | 09:16 |
| jrosser | amab: yes, thats what i see on my hosts, do you know how those libvirt files got into /usr/local? | 09:17 |
| *** DanyC has joined #openstack-ansible | 09:20 | |
| amab | jrosser: No! | 09:20 |
| jrosser | amab: here is what happens on a queens build http://logs.openstack.org/18/619718/2/check/openstack-ansible-deploy-aio_lxc-ubuntu-xenial/5d8fa94/logs/ara-report/result/002b9ce0-1033-4424-9b8e-186a1b43d93c/ | 09:20 |
| jrosser | did you start with a completely fresh 16.04 host? | 09:20 |
| amab | jrosser: Not completely | 09:21 |
| amab | I am going to remvoe all packages related to kvm, libvirt, qemu... | 09:21 |
| amab | and try again | 09:21 |
| jrosser | amab: i would recommend that you build AIO like this on a host you can really easily reinstall, or do it in a VM, then you can start again without worry | 09:23 |
| *** rgogunskiy has joined #openstack-ansible | 09:24 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible master: Add retries for utility container distro package installs https://review.openstack.org/618711 | 09:28 |
| jrosser | amab: also note that those tasks you have an error with are removed from the rocky release - is there a reason you're trying queens? | 09:30 |
| odyssey4me | pabelanger /opt/openstack-ansible is the only thing we test - ymmv but if it's a hard requirement for anything, we'd like it not to be | 09:33 |
| odyssey4me | pabelanger there is also https://review.openstack.org/559105 to make it better :) | 09:33 |
| arxcruz | evrardjp: odyssey4me https://review.openstack.org/#/c/619986/ :) | 09:37 |
| *** hamzaachi has quit IRC | 09:38 | |
| Miouge- | Good morning everyone :) | 09:40 |
| jrosser | looks like we need some centos mirrors fu http://mirror.ord.rax.openstack.org:8080/rdo/centos7-master/current-passed-ci/python-openstackclient-lang-3.16.1-0.20181026105338.3e5a2d2.el7.noarch.rpm: [Errno 14] HTTP Error 404 | 09:40 |
| *** faizy_ has joined #openstack-ansible | 09:43 | |
| *** faizy98 has quit IRC | 09:46 | |
| *** aedc has joined #openstack-ansible | 09:51 | |
| openstackgerrit | Jonathan Rosser proposed openstack/ansible-role-systemd_service master: Allow user/group to be overriden in unit section https://review.openstack.org/620116 | 09:53 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible stable/rocky: Remove unnecessary octavia scenario AIO bootstrap https://review.openstack.org/619792 | 10:05 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible stable/rocky: Ensure AIO container_tech/install_method vars are namespaced https://review.openstack.org/620278 | 10:05 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible stable/rocky: Implement documentation changes for translations https://review.openstack.org/620279 | 10:05 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible stable/rocky: [docs] Clean up the AIO user story https://review.openstack.org/620280 | 10:05 |
| jrosser | i think rocky is blocked on ceph/suse failures - we should decide what to do about that | 10:12 |
| openstackgerrit | Merged openstack/openstack-ansible-ops master: reduce container count on cluster test https://review.openstack.org/619436 | 10:18 |
| amab | jrosser: Sure, Thanks. As I have already worked on queens, I prefered to check it on OSA. I want to work with OSA to use that is production environment. How you compare OSA and kolla? is kolla stable enough? | 10:18 |
| admin0 | \o | 10:20 |
| *** aedc has quit IRC | 10:21 | |
| admin0 | amab, i had to evaluate between kolla and OSA recently .. the issue is not technical ..the issue is in the support you get back .. i also am in the kolla channel to see the quality of questions/answers .. i stick to OSA because here there more people who do OPS and are able to guide you .. there if you are faced with a difficult question, there is no answer on it for days and sometimes no direction at all | 10:24 |
| admin0 | does it work .. yes, can you guarantee that there will be more knowledgeable people to help you when you face issues in setup,upgrade,issues - no | 10:25 |
| *** aedc has joined #openstack-ansible | 10:29 | |
| *** nowster has joined #openstack-ansible | 10:31 | |
| amab | admin0: Thnak you. Is there any prefer on ubuntu or centos? I usually prefer centos but as I checked most of the discussions on OSA are based on Ubuntu | 10:32 |
| admin0 | amab, ubuntu :) | 10:33 |
| admin0 | because of dependencies and upgrade of packages | 10:33 |
| *** rpittau has joined #openstack-ansible | 10:33 | |
| *** udesale has quit IRC | 10:33 | |
| *** udesale has joined #openstack-ansible | 10:34 | |
| jrosser | amab: there are folks here who run quite big clouds on centos with OSA - so its a supported OS and the tooling does work.... on the other hand there are probably many more people using OSA/ubuntu so perhaps the issues/bugs tend to get discovered and ironed out quicker there | 10:37 |
| jrosser | having said all that mnaser has done a great job recently on the centos support so it's really up to you and how important the choice of underlying OS is in your environment | 10:37 |
| *** nowster is now known as nowster- | 10:38 | |
| *** nowster has joined #openstack-ansible | 10:38 | |
| *** nowster- has left #openstack-ansible | 10:38 | |
| admin0 | amab, my viewpoints are based on stuff outside of openstack specific .. like backup, monitoring, other packages that you want to install and you either find that the package is not even there in centos but in ubuntu, or the package is old and you cannot upgrade to do the rpm dependency stuff | 10:39 |
| *** udesale has quit IRC | 10:39 | |
| Miouge- | It also comes down to what you are familiar with. If your org tooling around this or that OS it can make your life easier | 10:41 |
| amab | admin0: Good :) I will try ubuntu | 10:44 |
| *** mma has quit IRC | 10:47 | |
| cshen | Hi all, need some reviews(+2) for my patch https://review.openstack.org/#/c/581248/ thanks. | 10:47 |
| amab | admin0: right | 10:51 |
| *** mma has joined #openstack-ansible | 10:52 | |
| *** mma has quit IRC | 11:02 | |
| *** udesale has joined #openstack-ansible | 11:06 | |
| *** hamzaachi has joined #openstack-ansible | 11:14 | |
| *** electrofelix has joined #openstack-ansible | 11:17 | |
| *** hamzaachi_ has joined #openstack-ansible | 11:21 | |
| *** mma has joined #openstack-ansible | 11:24 | |
| *** hamzaachi has quit IRC | 11:24 | |
| *** hamzaachi__ has joined #openstack-ansible | 11:24 | |
| *** hamzaachi_ has quit IRC | 11:27 | |
| *** mma has quit IRC | 11:28 | |
| *** ahosam has quit IRC | 11:31 | |
| *** chkumar|ruck is now known as chkumar|away | 11:32 | |
| *** udesale has quit IRC | 11:38 | |
| jhonni99 | hi all, i have question regarding haproxy , how can i overwrite the "haproxy_whitelist_networks" param bedore deploying , the default ones doesnt work for our ENV. | 11:41 |
| *** ahosam has joined #openstack-ansible | 11:41 | |
| jhonni99 | its not documented | 11:42 |
| *** cshen has quit IRC | 11:44 | |
| *** udesale has joined #openstack-ansible | 11:45 | |
| *** udesale has quit IRC | 11:45 | |
| *** udesale has joined #openstack-ansible | 11:46 | |
| Miouge- | jhonni99: I think you can set ‘haproxy_whitelist_networks’ for a global setting, or haproxy_glance_registry_whitelist_networks, haproxy_nova_metadata_whitelist_networks etc.. for a service specific override, see https://github.com/openstack/openstack-ansible/blob/master/inventory/group_vars/haproxy/haproxy.yml | 11:47 |
| *** mma has joined #openstack-ansible | 11:51 | |
| *** udesale has quit IRC | 11:52 | |
| *** udesale has joined #openstack-ansible | 11:53 | |
| jhonni99 | @Miouge- thanks alot :) | 11:53 |
| *** chhagarw has joined #openstack-ansible | 11:54 | |
| *** faizy_ has quit IRC | 11:54 | |
| *** cshen has joined #openstack-ansible | 11:54 | |
| *** mma has quit IRC | 11:56 | |
| *** mma has joined #openstack-ansible | 12:00 | |
| openstackgerrit | Merged openstack/openstack-ansible master: Use discuss ML instead of dev/operators https://review.openstack.org/620015 | 12:01 |
| *** ahosam has quit IRC | 12:05 | |
| *** ahosam has joined #openstack-ansible | 12:05 | |
| openstackgerrit | Christian Zunker proposed openstack/openstack-ansible-os_neutron master: Fix OVS scenario with changes introduced with Queens https://review.openstack.org/617155 | 12:05 |
| *** mma has quit IRC | 12:07 | |
| openstackgerrit | Christian Zunker proposed openstack/openstack-ansible-os_neutron master: Fix OVS scenario with changes introduced with Queens https://review.openstack.org/617155 | 12:07 |
| *** aedc has quit IRC | 12:11 | |
| jamesdenton | good morning all | 12:16 |
| *** rpittau_ has joined #openstack-ansible | 12:16 | |
| *** rpittau has quit IRC | 12:19 | |
| *** udesale has quit IRC | 12:20 | |
| *** ahosam has quit IRC | 12:20 | |
| *** ahosam has joined #openstack-ansible | 12:20 | |
| *** vnogin has joined #openstack-ansible | 12:21 | |
| *** vnogin has quit IRC | 12:22 | |
| *** rpittau has joined #openstack-ansible | 12:24 | |
| *** rpittau_ has quit IRC | 12:28 | |
| *** ansmith has quit IRC | 12:29 | |
| *** udesale has joined #openstack-ansible | 12:32 | |
| *** udesale has quit IRC | 12:32 | |
| *** mma has joined #openstack-ansible | 12:33 | |
| *** udesale has joined #openstack-ansible | 12:34 | |
| jamesdenton | jrosser Did that centos build issue get resolved from yesterday? Related to python2-os-client-config i think? | 12:35 |
| jrosser | jamesdenton: i think i saw it again today, let me just check | 12:36 |
| jrosser | jamesdenton: here is a job which failed 7hrs ago for the same thing https://review.openstack.org/#/c/620034/ | 12:37 |
| jamesdenton | looks like the mirror now has an updated version: python2-os-client-config-1.31.2-0.20181105100255.a024da3.el7.noarch.rpm | 12:38 |
| jamesdenton | wah wah | 12:38 |
| jrosser | recheck recheck recheck | 12:38 |
| jamesdenton | thank you | 12:38 |
| pabelanger | odyssey4me: ack, that is what I figured | 12:38 |
| jrosser | jamesdenton: did you get any insight into the lxc cache prep timeout? | 12:40 |
| jamesdenton | none whatsoever | 12:40 |
| noonedeadpunk | Folks, what do you think about this patch? https://review.openstack.org/#/c/620076/ | 12:40 |
| *** mma has quit IRC | 12:41 | |
| jrosser | jamesdenton: http://mirror.dfw.rax.openstack.org:8080/rdo/centos7-master/current-passed-ci/python-openstackclient-lang-3.16.1-0.20181026105338.3e5a2d2.el7.noarch.rpm: [Errno 14] HTTP Error 404 | 12:42 |
| jamesdenton | there's 10 packages getting 404s just from the review you showed me | 12:43 |
| *** mma has joined #openstack-ansible | 12:44 | |
| * jrosser wonders why these rdo repos are not using a hash for the path instead of current-passed-ci....... | 12:48 | |
| *** mma has quit IRC | 12:48 | |
| vakuznet | Review please https://review.openstack.org/#/c/617776/ | 12:50 |
| *** vnogin has joined #openstack-ansible | 12:51 | |
| *** udesale has quit IRC | 12:54 | |
| *** cshen_ has joined #openstack-ansible | 12:56 | |
| *** cshen has quit IRC | 12:59 | |
| *** arxcruz is now known as arxcruz|brb | 13:00 | |
| jrosser | mnaser: you around? | 13:04 |
| *** rpittau_ has joined #openstack-ansible | 13:08 | |
| *** maddtux has quit IRC | 13:10 | |
| *** rpittau has quit IRC | 13:11 | |
| *** aedc has joined #openstack-ansible | 13:15 | |
| mnaser | Hi jrosser | 13:16 |
| mnaser | Just rolling in to the office | 13:16 |
| *** ahosam has quit IRC | 13:17 | |
| jrosser | mnaser: you added this to the tests repo which fixed all sorts of repo 404 errors for centos https://github.com/openstack/openstack-ansible-tests/blob/master/common-tasks/test-set-nodepool-vars.yml#L44-L55 | 13:17 |
| jrosser | i cant see the same thing in the main repo though? | 13:18 |
| mnaser | Yep, I’m gonna guess it’s not in the integrated | 13:18 |
| mnaser | Yeah probably it was missed :( | 13:18 |
| jrosser | snap :) no worries easy to add. there are a bunch of integrated tests suffering the same way | 13:18 |
| *** rpittau_ is now known as rpittau | 13:20 | |
| mnaser | Hopefully this wil all be eliminated soon | 13:20 |
| mnaser | Thanks to the awesome work by odyssey4me | 13:20 |
| mnaser | Can you push a patch jrosser ? | 13:20 |
| jrosser | sure | 13:20 |
| mnaser | Thank you | 13:21 |
| odyssey4me | still looking for a second review for https://review.openstack.org/551791 which enables integrated AIO usage for role tests - it doesn't cover everything, but gets the basics done for a defaults-only functional check with whatever services we want | 13:22 |
| odyssey4me | once that merges I can work on adding more things - like increased affinity for certain containers to allow cluster tests (like we do for keystone, rabbit, galera)... but I'd like to get the basics done and ported back so that we can start doing upgrade tests asap | 13:23 |
| *** mma has joined #openstack-ansible | 13:25 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible master: Ensure that a consistent mirror is used for RDO https://review.openstack.org/620313 | 13:28 |
| mnaser | and upgrade tests are pretty critical | 13:32 |
| mnaser | esp with os_placement stuff | 13:32 |
| guilhermesp | mnaser: when is the ideal deadline to have os_placement done? At least the first test scenarios | 13:33 |
| mnaser | guilhermesp: the earlier the better? :D haha, right now we might be the first project to have a working version | 13:34 |
| mnaser | but you're probably really reall close ot it anyways | 13:34 |
| guilhermesp | yeah, just need some help with this http://logs.openstack.org/20/618820/28/check/openstack-ansible-functional-centos-7/3156ada/ara-report/result/e6cd2526-f565-4cc9-87e2-b636fe9a5ff5/ | 13:34 |
| guilhermesp | :P | 13:34 |
| guilhermesp | have no ideal at the moment which dependencies to add in vars files | 13:34 |
| openstackgerrit | Merged openstack/openstack-ansible stable/ocata: Fix ansible-sshd version https://review.openstack.org/619800 | 13:34 |
| mnaser | ooo i think i know | 13:34 |
| *** strattao has joined #openstack-ansible | 13:34 | |
| mnaser | guilhermesp: https://review.openstack.org/#/c/618820/28/vars/redhat-7.yml | 13:35 |
| mnaser | you have it set to placement_distro_packages: [] | 13:35 |
| mnaser | i mean you have it set to "placement_distro_packages: " which is like setting it to None | 13:35 |
| mnaser | if you do "placement_distro_packages: []" it will be an empty list | 13:35 |
| mnaser | and the len, etc stuff should work | 13:35 |
| mnaser | so the stuff that should be a list should just be [] to be an empty list | 13:35 |
| guilhermesp | the conditional is failing with lenght > 0, would be the same? | 13:36 |
| mnaser | guilhermesp: yes, its because its trying to compare "None" to a list | 13:38 |
| mgariepy | morning everyone | 13:38 |
| guilhermesp | k, going to fix that mnaser thanks | 13:38 |
| mnaser | no problem guilhermesp | 13:39 |
| mnaser | hi mgariepy | 13:39 |
| mgariepy | failure on this one is weird. https://review.openstack.org/#/c/620140/ | 13:39 |
| mgariepy | haha | 13:39 |
| openstackgerrit | Guilherme Steinmuller Pimentel proposed openstack/openstack-ansible-os_placement master: [WIP] Create base files to install placement https://review.openstack.org/618820 | 13:40 |
| mgariepy | but both patch that depends on it succeeded. | 13:40 |
| *** vnogin has quit IRC | 13:41 | |
| *** jhesketh_ has joined #openstack-ansible | 13:44 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible master: Ensure that a consistent mirror is used for RDO https://review.openstack.org/620313 | 13:48 |
| *** jhesketh has quit IRC | 13:50 | |
| *** hughsaunders has quit IRC | 13:50 | |
| *** gillesMo has quit IRC | 13:51 | |
| mnaser | mgariepy: centos related review if you're around -- https://review.openstack.org/#/c/620152/ | 13:54 |
| *** ansmith has joined #openstack-ansible | 13:54 | |
| *** udesale has joined #openstack-ansible | 13:56 | |
| mgariepy | mnaser, done | 14:02 |
| mnaser | thank you | 14:03 |
| mgariepy | https://review.openstack.org/#/c/619796/ and https://review.openstack.org/#/c/620140/ should fix ceph on suse in Rocky | 14:05 |
| *** udesale has quit IRC | 14:05 | |
| mgariepy | and this one would need review : https://review.openstack.org/#/c/620075/ | 14:05 |
| *** udesale has joined #openstack-ansible | 14:05 | |
| *** strattao has quit IRC | 14:15 | |
| openstackgerrit | Andy Smith proposed openstack/openstack-ansible master: Add qdrouterd role for oslo.messaging backend deployment https://review.openstack.org/570261 | 14:24 |
| openstackgerrit | kourosh vivan proposed openstack/openstack-ansible-os_keystone master: Deployment host can provide shibboleth SP key-pair https://review.openstack.org/620324 | 14:25 |
| *** chhagarw has quit IRC | 14:27 | |
| *** stuartgr has quit IRC | 14:30 | |
| openstackgerrit | Maxime Guyot proposed openstack/openstack-ansible master: Update doc to use chrony instead of ntp https://review.openstack.org/620329 | 14:30 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/ansible-role-python_venv_build master: Apply constraints when building the venv https://review.openstack.org/620331 | 14:32 |
| *** vnogin has joined #openstack-ansible | 14:32 | |
| *** gisak has joined #openstack-ansible | 14:36 | |
| gisak | hello everyone | 14:36 |
| gisak | can some1 tell me: lxc containers need network to be accessible by ansible with python, or they need python in order to install network ? | 14:38 |
| Miouge- | gisak: I think it uses DHCP (lxc-dnsmaq) on eth0 to bootstrap things | 14:40 |
| *** Miouge- has left #openstack-ansible | 14:43 | |
| *** Miouge- has joined #openstack-ansible | 14:43 | |
| gisak | very strange things happen on my controller node, when I run playbooks for deploying hosts, it fails to connect to containers, only the neutron from network node is created and deployed perfectly | 14:43 |
| gisak | http://paste.openstack.org/show/736030/ | 14:43 |
| gisak | @noonedeadpunk and @admin0 suggested me to install python manually on each container, but I cant do that as containers have no network .. | 14:48 |
| Miouge- | gisak: AFAIK eth0 is wired the lxcbr0 which gives access to outside world (apt & all) | 14:51 |
| *** chhagarw has joined #openstack-ansible | 14:51 | |
| jamesdenton | gisak Are you following the documentation precisely? Which OS are you using and how did you kick them? The issues you're describing are not really common | 14:52 |
| Miouge- | gisak: Have a look at “ip route show” and “ip addr show eth0” in your LXC containers. | 14:52 |
| *** gkadam has quit IRC | 14:52 | |
| gisak | agree, I understand that, but have no idea how to manually link lxc's eth0 to lxcbr0 | 14:53 |
| mnaser | odyssey4me: i just realized we missed something in the removal of apt-cacher-ng | 14:53 |
| gisak | using ubuntu 18 | 14:53 |
| mnaser | in an upgrade, the /etc/haproxy/conf.d/repo_cache file is not removed | 14:53 |
| odyssey4me | mnaser what's that? | 14:54 |
| mnaser | so the config file still exists | 14:54 |
| mnaser | and so haproxy config gets rebuilt with that and forever complains that the service is down | 14:54 |
| odyssey4me | hmm, yeah - apologies for that | 14:54 |
| mnaser | no worries! | 14:54 |
| noonedeadpunk | gisak: check, lxc containers network has nat on the lxc host | 14:54 |
| mnaser | i can push a patch.. but im not sure if there's a way to setup something in the haproxy role | 14:54 |
| mnaser | where we kinda empty out anything that we dont define in /etc/haproxy/conf.d | 14:54 |
| mnaser | puppet had stuff like that but dunno if we can do the same in ansible | 14:55 |
| odyssey4me | mnaser https://github.com/openstack/openstack-ansible-haproxy_server/commit/972ebbe5db3b5e340176640129b4dc7923ead8aa would be handy for that | 14:55 |
| mnaser | yeah that's one way of going about it | 14:55 |
| odyssey4me | mnaser otherwise the simplest is probably to wipe that folder in the run-upgrade script so that the role rebuilds it | 14:55 |
| mnaser | odyssey4me: yeah but in my case i dont run-upgrade so that might hit others, ok, let me push up a patch to add it with state=absent | 14:56 |
| mnaser | as a rocky-only patch | 14:56 |
| Miouge- | jamesdenton: I ran into a similar issue than what gisak describes a while back. I don’t recall all the details but a lxc-stop && lxc-start fixed the situation | 14:56 |
| odyssey4me | yeah, I prefer that because it allows handling of minor/major upgrades | 14:56 |
| mnaser | okay great, ill do it | 14:56 |
| jamesdenton | Good to know, Miouge-. Thanks. | 14:57 |
| noonedeadpunk | gisak it shoulod have smth like this http://paste.openstack.org/show/736112/ | 14:58 |
| gisak | lxc-stop/start does not work .. in containers config file network is not described as it should be | 14:59 |
| gisak | yeah you are right the iptables -L -t nat is ok | 14:59 |
| gisak | the only problem is in containers network | 14:59 |
| noonedeadpunk | gisak: probably the best way here is really to drop all containers, clean lxc cache and create them from scratch | 15:01 |
| gisak | well.. that is what I am doing all day today :) | 15:02 |
| gisak | still cant understand why on network node, which is also the deployer, containers are created without problems while on controller node not | 15:04 |
| openstackgerrit | Merged openstack/openstack-ansible-lxc_container_create master: redhat: stop updating system unnecessarily https://review.openstack.org/620152 | 15:05 |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible stable/rocky: haproxy: remove repo_cache service https://review.openstack.org/620336 | 15:05 |
| Miouge- | gisak: is it possible to dump the output of “ip route show” and “ip addr show eth0” from one of the failed container (with a lxc-attach for instance)? | 15:06 |
| *** ianychoi_ is now known as ianychoi | 15:06 | |
| *** chhagarw has quit IRC | 15:07 | |
| gisak | root@controller01:~# lxc-attach controller01_aodh_container-4b94f543 | 15:07 |
| gisak | root@controller01_aodh_container-4b94f543:~# ip addr show eth0 | 15:07 |
| gisak | Device "eth0" does not exist. | 15:07 |
| gisak | http://paste.openstack.org/show/736114/ | 15:08 |
| gisak | http://paste.openstack.org/show/736115/ | 15:09 |
| gisak | compare these 2 configs, first from controller node and second from network node | 15:10 |
| gisak | http://paste.openstack.org/show/736116/ | 15:11 |
| gisak | http://paste.openstack.org/show/736117/ | 15:11 |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible-haproxy_server stable/rocky: Add 'absent' service state https://review.openstack.org/620337 | 15:11 |
| gisak | and what is stored in containers dir folder | 15:12 |
| *** noonedeadpunk[h] has joined #openstack-ansible | 15:12 | |
| *** noonedeadpunk has quit IRC | 15:13 | |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible stable/rocky: haproxy: remove repo_cache service https://review.openstack.org/620336 | 15:14 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/ansible-role-python_venv_build master: Use the repo if available https://review.openstack.org/620339 | 15:18 |
| *** rpittau has quit IRC | 15:21 | |
| *** strattao has joined #openstack-ansible | 15:24 | |
| *** vollman has joined #openstack-ansible | 15:26 | |
| *** shardy has quit IRC | 15:26 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_glance master: Remove glance_developer_mode and glance_venv_download https://review.openstack.org/620340 | 15:28 |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible-lxc_container_create stable/rocky: redhat: stop updating system unnecessarily https://review.openstack.org/620342 | 15:30 |
| *** cshen_ has quit IRC | 15:33 | |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible-lxc_container_create master: cleanup: remove rocky-only upgrade code https://review.openstack.org/620343 | 15:34 |
| *** strattao has quit IRC | 15:36 | |
| mnaser | odyssey4me: https://github.com/openstack/openstack-ansible/commit/43a5c874ef7ff1c43c51086022655bb44492794e#diff-80113ee2b90d9100a81da4dbd71abd17 | 15:37 |
| mnaser | how do you feel about dropping this in S instead of T.. because i dont think we released with apt-cacher-ng enabled? | 15:37 |
| *** weezS has joined #openstack-ansible | 15:38 | |
| odyssey4me | yes, but that's there for anyone who deployed prior to release | 15:38 |
| mnaser | odyssey4me: fair enough, trying to minimize our tasks because that one runs a looot :p | 15:39 |
| mnaser | maybe we'd need a more explicit 'support' policy | 15:39 |
| *** vnogin has quit IRC | 15:40 | |
| odyssey4me | well, if you want to drop it - make sure there's a release note informing people to clean up themselves if they're falling in that group | 15:41 |
| *** vnogin has joined #openstack-ansible | 15:41 | |
| mnaser | not super tied to it | 15:41 |
| *** shardy has joined #openstack-ansible | 15:44 | |
| *** noonedeadpunk[h] is now known as noonedeadpunk | 15:49 | |
| *** jawad_axd has quit IRC | 15:49 | |
| mnaser | odyssey4me: lmao we're not getting rid of htat job | 15:50 |
| mnaser | # grep proxy /etc/yum.conf => proxy=http://<foo>.vexxhost.local:3142 | 15:50 |
| mnaser | the jobs assume quotes | 15:50 |
| *** jawad_axd has joined #openstack-ansible | 15:50 | |
| odyssey4me | oops | 15:50 |
| mnaser | s/jobs/tasks/ | 15:50 |
| mnaser | heh, all good | 15:50 |
| mnaser | im wondering why we dont use ini_file | 15:50 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-ops master: MNAIO: Add ability to deploy Ceph https://review.openstack.org/607338 | 15:51 |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible master: redhat: clean-up proxy settings properly https://review.openstack.org/620346 | 15:54 |
| evrardjp | o/ | 15:54 |
| *** jawad_axd has quit IRC | 15:55 | |
| evrardjp | Oh I forgot to say somewhere that I saw someone passing *_proxy in *_PROXY in env vars for tox, and I thought that was quite cleaner than listing all the vars like we do | 15:55 |
| evrardjp | change the location of the word 'somewhere' after 'tox'. | 15:55 |
| odyssey4me | perhaps, but hardly worth changing all our tox configs again | 15:56 |
| evrardjp | odyssey4me: agreed. | 15:56 |
| odyssey4me | nice to know wildcards are possible though | 15:56 |
| evrardjp | odyssey4me: just saying in case | 15:56 |
| evrardjp | exactly :) | 15:56 |
| odyssey4me | we'll be ditching tox for most of our tests anyway in the near future | 15:56 |
| *** udesale has quit IRC | 15:56 | |
| *** udesale has joined #openstack-ansible | 15:57 | |
| evrardjp | I suppose we'll keep them for docs/reno | 15:57 |
| odyssey4me | once I manage to get another core review for https://review.openstack.org/551791 :) | 15:57 |
| odyssey4me | yep, docs/reno will stay using it | 15:57 |
| odyssey4me | that's part of the PTI | 15:57 |
| evrardjp | indeed. | 15:58 |
| *** udesale has quit IRC | 15:58 | |
| mnaser | cloudnull, DimGR, d34dh0r53, hughsaunders, b3rnard0, palendae, odyssey4me, serverascode, rromans, erikmwilson, mancdaz, _shaps_, BjoernT, claco, echiu, dstanek, jwagner, ayoung, prometheanfire, evrardjp, arbrandes, scarlisle, luckyinva, ntt, javeriak, spotz, vdo, jmccrory, alextricity25, jasondotstar, admin0, michaelgugino, ametts, bgmccollum, darrenc, JRobinson__, colinmcnamara, thorst, adreznec, eil397, | 16:00 |
| mnaser | qwang,nishpatwa_, cathrichardson, drifterza, hwoarang, cshen, ullbeking, mnaser, nicolasbock, jrosser, cjloader, antonym, dcdamien | 16:00 |
| mnaser | #startmeeting openstack_ansible_meeting | 16:00 |
| mnaser | #topic rollcall | 16:00 |
| openstack | Meeting started Tue Nov 27 16:00:21 2018 UTC and is due to finish in 60 minutes. The chair is mnaser. Information about MeetBot at http://wiki.debian.org/MeetBot. | 16:00 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 16:00 |
| *** openstack changes topic to " (Meeting topic: openstack_ansible_meeting)" | 16:00 | |
| openstack | The meeting name has been set to 'openstack_ansible_meeting' | 16:00 |
| *** openstack changes topic to "rollcall (Meeting topic: openstack_ansible_meeting)" | 16:00 | |
| mnaser | o/ | 16:00 |
| spotz | o/ | 16:00 |
| evrardjp | o/ | 16:00 |
| cloudnull | o/ | 16:01 |
| odyssey4me | o/ although I'm sharing with another meeting :/ | 16:01 |
| mnaser | i'm sorry, i haven't been updating things properly | 16:01 |
| mnaser | last week highlights imho has been: odyssey4me getting us closer to running integrated for all roles | 16:02 |
| mnaser | so please eyes on https://review.openstack.org/#/c/551791/ | 16:02 |
| odyssey4me | need one more core review for https://review.openstack.org/551791 - then I'll be porting those back so that we can do upgrade tests along with deploy tests | 16:02 |
| mnaser | i thought i reviewd that | 16:02 |
| mnaser | ok ill review this shortly after the meeting to get us the second +2 | 16:03 |
| odyssey4me | ta | 16:03 |
| mnaser | other things is os_placement still in progress | 16:03 |
| mnaser | we're having a lot of gate failures i think | 16:03 |
| evrardjp | os_manila is also in progress | 16:03 |
| evrardjp | (while of lesser importance than placement, I agree) | 16:04 |
| mnaser | i can look at the centos jobs to help make them stable but if we're going to have always-red nonvoting jobs, i would want to remove them | 16:04 |
| mnaser | it takes 1.5h of wasted upstream CI time on every job and if no one is actively looking at them.. makes it hard | 16:04 |
| odyssey4me | bear in mind we agreed that distro jobs on master stay non-voting, regardless of consistent failures | 16:04 |
| odyssey4me | this allows people to work on fixing things as we go, rather than having to focus on fixing it at the end of the cycle | 16:05 |
| evrardjp | well the idea was to report back -- for example suse current breakages in horizon is on my todo list, even if I didn't get the chance to tackle it | 16:05 |
| mnaser | odyssey4me: i agree, but it seems like there aren't a lot of eyes on them presently | 16:05 |
| evrardjp | we need more ppl to help there, anyone, feel free to help ! :) | 16:06 |
| odyssey4me | sure, but it'll be easier to troubleshoot and solve them once we're using integrated build for all tests - because everyone is more familiar with it | 16:06 |
| mnaser | odyssey4me: yep, i agree | 16:06 |
| mnaser | so just something to keep in mind | 16:06 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_tempest master: Add support for aarch64 images https://review.openstack.org/620032 | 16:07 |
| mnaser | we've identified a few responsible people per distro so just keep up with those and feel free to ping them when/if we need things | 16:07 |
| mnaser | over the next few days i'm upgrading a few environments so https://review.openstack.org/#/q/topic:queens-to-rocky | 16:07 |
| evrardjp | mnaser: great news! | 16:07 |
| mnaser | that's most of what i've identified, anyone else (surely i missed things!) | 16:07 |
| evrardjp | shouldn't that be a topic of the open discussion though? It's less a report, more a heads-up :p | 16:08 |
| vakuznet | Review please https://review.openstack.org/#/c/618046/ | 16:08 |
| mnaser | sure | 16:09 |
| mnaser | mostly just a "what i've been working on" thing, anyways | 16:09 |
| mnaser | #topic bug triage | 16:10 |
| *** openstack changes topic to "bug triage (Meeting topic: openstack_ansible_meeting)" | 16:10 | |
| mnaser | #link https://bugs.launchpad.net/openstack-ansible/+bug/1805239 | 16:10 |
| openstack | Launchpad bug 1805239 in openstack-ansible "missing logrotate of the syslog, daemon.log " [Undecided,New] | 16:10 |
| mnaser | hm | 16:11 |
| mnaser | we dont use rsyslog_client anymore right | 16:11 |
| odyssey4me | we do, for suse | 16:11 |
| evrardjp | maybe asking which branch does this apply? | 16:12 |
| evrardjp | and if that is for hosts or containers? | 16:12 |
| evrardjp | vakuznet: are you the owner of this bug? ^ | 16:12 |
| mnaser | this says centos | 16:12 |
| vakuznet | yes | 16:12 |
| vakuznet | Queens, host | 16:13 |
| evrardjp | vakuznet: we supposed master :) | 16:13 |
| evrardjp | presumed* | 16:13 |
| evrardjp | so yeah that could apply. | 16:13 |
| mnaser | so in rocky we no longer use rsyslog stuff and moved to journals for everything afaik | 16:13 |
| mnaser | it is a queens bug | 16:13 |
| evrardjp | indeed. | 16:13 |
| mnaser | vakuznet: are you able to push a patch regarding this? i can help with how to get it properly landed if you have questions | 16:14 |
| vakuznet | sure | 16:14 |
| odyssey4me | I'm not feeling very well, so I'm going offline. I'll catch y'all tomorrow. | 16:14 |
| mnaser | take care odyssey4me | 16:14 |
| mnaser | vakuznet: cool, ill mark as confirmed/medium and assign to you is that ok? | 16:15 |
| *** vnogin has quit IRC | 16:15 | |
| vakuznet | https://github.com/openstack/openstack-ansible/blob/master/playbooks/common-tasks/rsyslog-client.yml#L18 | 16:15 |
| evrardjp | see you later odyssey4me | 16:15 |
| mnaser | vakuznet: we can talk about the potential fix after our meeting right now, is that okay? we're just triaging things right now | 16:16 |
| mnaser | just to see bugs or not bugs :) | 16:16 |
| vakuznet | ok | 16:16 |
| evrardjp | and important or not :) | 16:16 |
| evrardjp | or less* | 16:16 |
| mnaser | #link https://bugs.launchpad.net/openstack-ansible/+bug/1804837 | 16:17 |
| openstack | Launchpad bug 1804837 in openstack-ansible "Failed to checkout version 0.5.1 of ansible-sshd for stable/pike" [Undecided,New] | 16:17 |
| mnaser | wasnt this fixed? | 16:17 |
| * mnaser checks | 16:17 | |
| mnaser | evrardjp: do you know if the shabump includes it? | 16:18 |
| mnaser | oh no shabump required | 16:18 |
| mnaser | ok so this will be included in 16.0.23 | 16:19 |
| mnaser | #link https://bugs.launchpad.net/openstack-ansible/+bug/1804827 | 16:20 |
| openstack | Launchpad bug 1804827 in openstack-ansible "Keystone ldap only configures domain on one node" [Undecided,New] | 16:20 |
| mnaser | aw darn | 16:20 |
| *** aedc has quit IRC | 16:20 | |
| evrardjp | sorry I was to something else | 16:20 |
| *** aedc has joined #openstack-ansible | 16:20 | |
| mnaser | yep thats valid | 16:21 |
| mnaser | regression by https://github.com/openstack/openstack-ansible-os_keystone/commit/096ed19665dad45fe9b1c1790ba599d90ccf233c | 16:21 |
| *** cshen has joined #openstack-ansible | 16:21 | |
| evrardjp | I am still reading... | 16:22 |
| mnaser | confirmed/high assigned to jesse .. problem is that we made a change to run ldap setup on one container only to avoid race conditions | 16:22 |
| mnaser | but that also meant the steps to config the local config files were only happening on one container | 16:23 |
| mnaser | so that patch caused it, it should simply do then when: on the os_keystone_domain task only, not the entire script | 16:23 |
| evrardjp | yeah | 16:23 |
| evrardjp | I saw your message | 16:23 |
| evrardjp | great content in the bug | 16:23 |
| evrardjp | I agree | 16:23 |
| mnaser | #link https://bugs.launchpad.net/openstack-ansible/+bug/1804770 | 16:24 |
| openstack | Launchpad bug 1804770 in openstack-ansible "Pike -> Queens Upgrade: Documentation for OVS setup and neutron agent rebalancing missing" [Undecided,New] | 16:24 |
| mnaser | thats an amazing bug | 16:24 |
| mnaser | great docs included | 16:24 |
| mnaser | posted asking for input | 16:25 |
| cloudnull | oh wow, thats a bummer ... | 16:25 |
| mnaser | if they want/can contribute it | 16:25 |
| evrardjp | yeah | 16:26 |
| cloudnull | jamesdenton had something on that the other day ? | 16:26 |
| jamesdenton | yes - i can take it and work on docs updates | 16:26 |
| mnaser | cloudnull: i think he might have been helping the bug repoter here | 16:26 |
| cloudnull | ah . | 16:26 |
| mnaser | jamesdenton: oh awesome, a lot of info was put there too, so ill assign to you, is that okay? | 16:26 |
| cloudnull | i think its something we can generally add to the upgrade scripts / playbooks | 16:26 |
| jamesdenton | sure thing. | 16:27 |
| mnaser | thank you | 16:27 |
| mnaser | oh thats an easy one | 16:27 |
| mnaser | #link https://bugs.launchpad.net/openstack-ansible/+bug/1804695 | 16:28 |
| openstack | Launchpad bug 1804695 in openstack-ansible "Credentials needed to "Verifying OpenStack operation in openstack-ansible" step" [Undecided,New] | 16:28 |
| openstackgerrit | Merged openstack/ansible-role-systemd_service master: Allow user/group to be overriden in unit section https://review.openstack.org/620116 | 16:28 |
| mnaser | added as low-hanging-fruit, great if anyone wants to push up a small fix :) | 16:28 |
| mnaser | #link https://bugs.launchpad.net/openstack-ansible/+bug/1804283 | 16:29 |
| openstack | Launchpad bug 1804283 in openstack-ansible "Rabbitmq Error: unable to connect to node 'rabbit@ostack-controller-01': nodedown" [Undecided,New] | 16:29 |
| mnaser | so i run into this all the time | 16:29 |
| mnaser | i have to do | 16:29 |
| mnaser | rabbitmqctl -n rabbit@`hostname` blah | 16:29 |
| mnaser | it's a bug, not sure how's the best way to resolve it, but maybe eliminated with nspawn | 16:30 |
| mnaser | confirmed/medium -- i'll add a comment about it.. maybe someone has insight if there's an easy fix? | 16:30 |
| evrardjp | mmm | 16:30 |
| evrardjp | that rings me a bell | 16:30 |
| evrardjp | mnaser: two questions for you | 16:31 |
| evrardjp | what's in /etc/hosts , and what do you have HOST or HOSTNAME in your bashrc/environment ? | 16:31 |
| evrardjp | oh third question, which branch? | 16:31 |
| mnaser | evrardjp: those are probably good questions to add for the bug | 16:32 |
| mnaser | :D | 16:32 |
| evrardjp | well we don't know if what you had is the same | 16:32 |
| evrardjp | but I can ask the questions. | 16:32 |
| evrardjp | let me comment on the bug | 16:32 |
| mnaser | it is the same issue :) | 16:32 |
| mnaser | ill move onto the next one while you comment on that | 16:33 |
| mnaser | #link https://bugs.launchpad.net/openstack-ansible/+bug/1803444 | 16:33 |
| openstack | Launchpad bug 1803444 in openstack-ansible "gnocchi playbook failing on 18.0.0" [Undecided,New] | 16:33 |
| *** chhagarw has joined #openstack-ansible | 16:34 | |
| evrardjp | I think that might be a duplicate of another one we talked in the past | 16:34 |
| mnaser | evrardjp: the gnocchi one? | 16:34 |
| evrardjp | yeah, doesn't that ring you a bell? | 16:35 |
| mnaser | yes, i didnt mark it as anything so it showed up next week :) | 16:35 |
| evrardjp | It seems we talked about that in the past. | 16:35 |
| evrardjp | hahah | 16:35 |
| evrardjp | that's why | 16:35 |
| mnaser | so i think the solution is to add `import_playbook` .. but i dunno the implications of doing that | 16:35 |
| evrardjp | yeah we even discussed on what to do , but nobody got cycles, IIRC | 16:36 |
| mnaser | okay well | 16:36 |
| mnaser | confirmed medium and we can ask for that patch to be pushed up | 16:36 |
| mnaser | #link https://bugs.launchpad.net/openstack-ansible/+bug/1802132 | 16:37 |
| openstack | Launchpad bug 1802132 in openstack-ansible "OpenStack Ansible dashboard not able to log in with admin credentials" [Undecided,New] | 16:37 |
| evrardjp | mnaser: should we point to http://eavesdrop.openstack.org/meetings/openstack_ansible_meeting/2018/openstack_ansible_meeting.2018-11-20-16.00.log.html#l-73 ? | 16:37 |
| evrardjp | ok let me add that link into the bug | 16:37 |
| mnaser | yes that would be useful | 16:37 |
| evrardjp | done | 16:38 |
| evrardjp | I haven't tried horizon for ages | 16:39 |
| mnaser | i dunno | 16:40 |
| mnaser | i've deployed countless times and havent seen aynthing | 16:40 |
| evrardjp | I recall horizon default endpoint policy to change to move to external by default, but I think we are explicit in endpoints usage. So it's probably not that, but I haven't touched horizon for a while, as I said earlier. | 16:41 |
| mnaser | posted a comment requesting user_vars and etc | 16:41 |
| evrardjp | yeah it seem to deploy just fine for me, but I haven't tested it. | 16:41 |
| mnaser | but it doesnt fel likea bug | 16:41 |
| evrardjp | yeah | 16:41 |
| evrardjp | I have the impression something wrong got inserted in the horizon config | 16:41 |
| evrardjp | making django scream | 16:41 |
| mnaser | yup | 16:41 |
| mnaser | #topic open discussion | 16:42 |
| *** openstack changes topic to "open discussion (Meeting topic: openstack_ansible_meeting)" | 16:42 | |
| evrardjp | next? | 16:42 |
| mnaser | thats all :) | 16:42 |
| evrardjp | so upgrades? :) | 16:42 |
| mnaser | going okay so far, i'll have a bunch of patches up | 16:42 |
| mnaser | i will write an email to the mailing list about meeting engagement. i will maybe suggest we actually come up with another time, another method of communication or dropping it | 16:42 |
| evrardjp | I think there were a few ppl asking for feedback | 16:42 |
| evrardjp | we can say in the future you did just fine? | 16:42 |
| evrardjp | :) | 16:42 |
| mnaser | once my patches land | 16:43 |
| mnaser | :P | 16:43 |
| evrardjp | haha | 16:43 |
| evrardjp | well | 16:43 |
| evrardjp | tbh it's been always fluctuating | 16:43 |
| evrardjp | sometimes you find 9 ppl, sometimes I felt pretty much alone | 16:43 |
| evrardjp | I asked if the time was right, and at the end it was the best time. | 16:43 |
| evrardjp | But I agree we shouldn't be only two in meetings. | 16:44 |
| evrardjp | this is an opportunity for everyone to raise awareness of their work/issue, yet the media is not used... | 16:44 |
| mnaser | yeah | 16:44 |
| * evrardjp shrugs | 16:44 | |
| mnaser | ill raise to ml | 16:45 |
| evrardjp | mnaser: it was worse when we had two meetings a week | 16:45 |
| mnaser | i can imagine | 16:45 |
| evrardjp | I think having a meeting per week is the minimum, as one every two week will make our bug list and goal focus too thin | 16:45 |
| evrardjp | bug list too big* | 16:45 |
| mnaser | i dunno | 16:45 |
| openstackgerrit | Merged openstack/openstack-ansible stable/queens: Add MariaDB infrastructure mirrors and disable nested virt https://review.openstack.org/619718 | 16:45 |
| mnaser | i feel like we do a good job communicating things together at the same time | 16:46 |
| mnaser | its an ML discussion which ill send write away | 16:46 |
| evrardjp | yup, I think it's good to put it in the open at least. | 16:46 |
| jrosser | perhaps we suffer today from not having the 1hr warning? | 16:46 |
| evrardjp | I'm alwys for LESS meetings :D | 16:46 |
| mnaser | jrosser: i dunno, the 1 hour warning usualyl doesn't help much | 16:47 |
| mnaser | i'm more about the engagement | 16:47 |
| evrardjp | oh yeah I am kinda not doing it anymore because I have other meetings | 16:47 |
| evrardjp | mnaser: it has been working for me in the past though : ) | 16:47 |
| evrardjp | (at least for the beginning of the meeting) | 16:47 |
| mnaser | i try to do it closer to the meeting | 16:47 |
| mnaser | so people kinda still remember it | 16:47 |
| evrardjp | haha | 16:48 |
| mnaser | if you tell me about something that will happen in an hour surely ill forget :) | 16:48 |
| evrardjp | that's because you're all over the place, not everyone is like you :p | 16:48 |
| mnaser | ha | 16:48 |
| mnaser | :P | 16:48 |
| mnaser | anyone has any other topics? | 16:49 |
| evrardjp | no other topic | 16:49 |
| mnaser | cool, thanks everyone | 16:50 |
| mnaser | #endmeeting | 16:50 |
| *** openstack changes topic to "Launchpad: https://launchpad.net/openstack-ansible || Weekly Meetings: https://wiki.openstack.org/wiki/Meetings/openstack-ansible || Review Dashboard: http://bit.ly/2xA1eZC" | 16:50 | |
| openstack | Meeting ended Tue Nov 27 16:50:17 2018 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 16:50 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/openstack_ansible_meeting/2018/openstack_ansible_meeting.2018-11-27-16.00.html | 16:50 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/openstack_ansible_meeting/2018/openstack_ansible_meeting.2018-11-27-16.00.txt | 16:50 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/openstack_ansible_meeting/2018/openstack_ansible_meeting.2018-11-27-16.00.log.html | 16:50 |
| evrardjp | nicolasbock: if you have extra cycles, could you check on why this failure on the installation of OSA with suse packages happen? http://logs.openstack.org/05/620005/1/check/openstack-ansible-deploy-aio_distro_lxc-opensuse-150/6381125/logs/ara-report/result/9c6cdb53-3fb9-4e82-aba4-9a3cce4eeccf/ . | 16:52 |
| evrardjp | nicolasbock: ImportError: No module named gbpclient.v2_0 probably means some package we need to add (or remove conditionally) on suse | 16:53 |
| *** arxcruz|brb is now known as arxcruz | 16:54 | |
| *** hamzaachi__ has quit IRC | 16:55 | |
| * tosky things that voice meetings are a way to *lower* the number of attendees, not raise them | 16:56 | |
| tosky | thinks* | 16:56 |
| *** gyee has joined #openstack-ansible | 16:59 | |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible master: redhat: clean-up proxy settings properly https://review.openstack.org/620346 | 17:06 |
| *** cshen has quit IRC | 17:09 | |
| *** mma has quit IRC | 17:10 | |
| *** jawad_axd has joined #openstack-ansible | 17:11 | |
| *** DanyC has quit IRC | 17:14 | |
| *** cshen has joined #openstack-ansible | 17:15 | |
| *** jawad_axd has quit IRC | 17:15 | |
| *** cshen has quit IRC | 17:20 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible master: Ensure that a consistent mirror is used for RDO https://review.openstack.org/620313 | 17:34 |
| pabelanger | mnaser: odyssey4me: evrardjp: cloudnull: logan- boom! Post job in ansible-network running OSA stable/rocky, from zuul: https://object-storage-ca-ymq-1.vexxhost.net/swift/v1/a0b4156a37f9453eb4ec7db5422272df/logs/ce/cebd2edf7b65986f46973502cf63094eb22f88eb/post/packet-ci-cloud-deploy/0f1c93e/job-output.html | 17:35 |
| pabelanger | only doing setup-hosts right now, but expect other playbooks to work fine | 17:35 |
| *** cshen has joined #openstack-ansible | 17:35 | |
| mnaser | pabelanger: w00t | 17:35 |
| pabelanger | How is ansible.cfg managed in OSA? Is there anything special I need to do if I want to modify it? | 17:36 |
| evrardjp | pabelanger: that's great | 17:36 |
| evrardjp | pabelanger: no setup.cfg | 17:36 |
| evrardjp | pabelanger: we rely on env vars should you want to override things | 17:36 |
| evrardjp | we load an rc file though, which sets a few env variables by default | 17:37 |
| pabelanger | k, do recommended way is envars | 17:37 |
| pabelanger | so* | 17:37 |
| pabelanger | I don't think I've setup ARA properly, can seem to find sqlite db | 17:38 |
| *** mma has joined #openstack-ansible | 17:39 | |
| *** aedc has quit IRC | 17:41 | |
| evrardjp | pabelanger: by default ARA isn't wired for everyone (just in gates), but I seem to recall a recent patch that landed in master to ensure ARA could be usable by everyone | 17:41 |
| evrardjp | pabelanger: https://review.openstack.org/#/c/612775/ | 17:42 |
| evrardjp | (there was a bug fix later, but that gives you an idea) | 17:42 |
| pabelanger | evrardjp: yah, I'm using that, but scripts/openstack-ansible.rc doesn't take the CALLBACK path into account | 17:43 |
| pabelanger | let me manually set it and see if it works | 17:43 |
| evrardjp | did you get this patch too? https://review.openstack.org/#/c/617389/ ? | 17:44 |
| pabelanger | evrardjp: no https://review.openstack.org/#/c/617785/ | 17:46 |
| pabelanger | I think the issue I will have, is script/bootstrap-ansible.sh is called in its own ansible task, so if we setup any environmental variables they will not be carried over to the next task | 17:47 |
| pabelanger | which is fine, I can set it up manually myself | 17:47 |
| evrardjp | nothing prevents you from using a ansible.cfg though | 17:48 |
| evrardjp | if that can help you | 17:48 |
| evrardjp | I am not sure about the current state of the precedence though. | 17:48 |
| pabelanger | sure, I'll manually set it now then figure out the best way | 17:49 |
| *** DanyC has joined #openstack-ansible | 17:49 | |
| pabelanger | today logan- wraps his deploy in once large bash script, where I am doing per tasks | 17:49 |
| gisak | hey guys, from where is better to clone the repo? git.openstack.org or github.com ? | 17:52 |
| evrardjp | gisak: github is a mirror of git.openstack.org | 17:52 |
| evrardjp | I am lazy and I have hub installed so I do openstack/openstack-ansible which fetches from gh | 17:53 |
| gisak | :) got it, thanks | 17:53 |
| *** DanyC has quit IRC | 17:54 | |
| *** mma has quit IRC | 17:57 | |
| *** jawad_axd has joined #openstack-ansible | 17:57 | |
| evrardjp | I have to go. Have a good night/day everyone! | 17:57 |
| pabelanger | thanks! | 17:57 |
| *** ansmith has quit IRC | 17:59 | |
| *** ansmith has joined #openstack-ansible | 18:01 | |
| *** ahosam has joined #openstack-ansible | 18:03 | |
| *** hamzaachi has joined #openstack-ansible | 18:07 | |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible-rabbitmq_server master: upgrade: start service before applying policies https://review.openstack.org/620378 | 18:11 |
| *** gary_perkins has quit IRC | 18:17 | |
| *** gary_perkins has joined #openstack-ansible | 18:18 | |
| gisak | in a 3 node env: network node, controller node and compute node, where glance is going to be on controller node, do I need to have br-vlan on controller node? | 18:33 |
| gisak | on controller I have: br-mgmt, br-vxlan and br-storage is it good ? | 18:33 |
| *** strattao has joined #openstack-ansible | 18:47 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/ansible-role-systemd_service stable/rocky: Allow user/group to be overriden in unit section https://review.openstack.org/620386 | 18:48 |
| *** nicolasbock has quit IRC | 18:54 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Update elk_6x for 6.5.x https://review.openstack.org/619448 | 18:56 |
| *** vollman has quit IRC | 18:56 | |
| jamesdenton | gisak you should not need br-vlan on a controller node if it is not running network services | 18:57 |
| jamesdenton | or br-vxlan, for that matter | 18:57 |
| *** jawad_axd has quit IRC | 19:05 | |
| jrosser | jamesdenton: if you ever want Octavia or dbaas then that might be more complicated? | 19:08 |
| jamesdenton | yes, you definitely have a point | 19:08 |
| jamesdenton | it really doesn't hurt to have it | 19:08 |
| *** ahosam has quit IRC | 19:08 | |
| openstackgerrit | Guilherme Steinmuller Pimentel proposed openstack/openstack-ansible-os_placement master: [WIP] Create base files to install placement https://review.openstack.org/618820 | 19:10 |
| openstackgerrit | Merged openstack/openstack-ansible-lxc_hosts master: ensure the mount unit is started after reboot https://review.openstack.org/617776 | 19:10 |
| openstackgerrit | Vadim Kuznetsov proposed openstack/openstack-ansible-lxc_hosts stable/queens: ensure the mount unit is started after reboot https://review.openstack.org/620389 | 19:12 |
| *** strattao has quit IRC | 19:14 | |
| *** jawad_axd has joined #openstack-ansible | 19:16 | |
| *** hamzaachi has quit IRC | 19:18 | |
| *** hamzaachi has joined #openstack-ansible | 19:18 | |
| *** electrofelix has quit IRC | 19:25 | |
| *** chhagarw has quit IRC | 19:28 | |
| openstackgerrit | Vadim Kuznetsov proposed openstack/openstack-ansible-lxc_hosts stable/rocky: ensure the mount unit is started after reboot https://review.openstack.org/620390 | 19:33 |
| *** hughsaunders has joined #openstack-ansible | 19:35 | |
| *** shardy has quit IRC | 19:35 | |
| logan- | pabelanger: nice! | 19:53 |
| *** mma has joined #openstack-ansible | 19:58 | |
| pabelanger | logan-: running all 3 playbooks now | 19:59 |
| *** mma has quit IRC | 20:02 | |
| logan- | was just reading thru your job setup, pretty nice how you've laid it out | 20:05 |
| *** goldenfri has joined #openstack-ansible | 20:09 | |
| gisak | damn... i dont get this guys ... http://paste.openstack.org/show/736181/ | 20:12 |
| gisak | wiped everything and started over, facing the same issue, attached full paste of playbooks | 20:13 |
| gisak | would be really nice if somebody can tell me where is my mistake | 20:13 |
| jamesdenton | looks like that paste ends in the middle of a task | 20:13 |
| gisak | guess there is a limit in pasting .. | 20:15 |
| gisak | https://jpst.it/1ussv | 20:18 |
| gisak | here is the paste of bootstrap-ansible.sh as well: https://jpst.it/1ussV | 20:21 |
| pabelanger | logan-: yah, used some of your logic for ssh auth sock | 20:21 |
| gisak | in first paste, at line 1866, maybe here is the problem?>>> fatal: [controller01]: FAILED! => {"ansible_job_id": "161256119402.28173", "attempts": 120, "changed": false, "finished": 0, "started": 1} | 20:24 |
| gisak | maybe after this error, containers are not created correctly and here is my problem | 20:26 |
| jamesdenton | quick! Can i get a review before zuul changes its mind? https://review.openstack.org/#/c/584069 | 20:30 |
| jamesdenton | gisak looking | 20:30 |
| gisak | sec | 20:30 |
| jrosser | gisak: TASK [lxc_hosts : Ensure that the LXC cache has been prepared] <= that fails on the controller, but works on the network node | 20:33 |
| jamesdenton | try bumping the timeout with lxc_cache_prep_timeout: 1200 | 20:33 |
| jamesdenton | you can add that to user_variables.yml | 20:33 |
| openstackgerrit | Vadim Kuznetsov proposed openstack/openstack-ansible-openstack_hosts master: use ansible_hostname to construct canonical_hostname in /etc/hosts https://review.openstack.org/620399 | 20:33 |
| jamesdenton | actually, it may already default to 1200. i guess it depends on the version. You could go 1800 which would be 30 min | 20:35 |
| jrosser | it really shouldnt take that long, unless you have very small upstream bandwidth to download the image | 20:35 |
| gisak | ok, now just try again, or need to wipe everything and start over ? | 20:35 |
| jamesdenton | you can set and rerun the playbook | 20:35 |
| jamesdenton | but jrosser is right, it's unusual but it does happen. | 20:36 |
| gisak | yeah, set it up to 1800 just to be sure | 20:36 |
| jrosser | if you do have a small internet uplink then you should be able to download the image somewhere local and override the url | 20:36 |
| gisak | internet is ok, but indeed I would like to download the image manually, how can I get the right link ? | 20:37 |
| gisak | I'm using ubuntu 18.04.1 LTS | 20:38 |
| jrosser | oh hold on - this is the cache prep, ignore me | 20:38 |
| jrosser | the imageis already downloaded | 20:38 |
| gisak | yep, and on neutron runs perfectly | 20:38 |
| jrosser | were you still using queens? | 20:39 |
| gisak | no, its Rocky | 20:40 |
| jrosser | you should be able to find /var/log/lxc-cache-prep-commands.log on each host | 20:40 |
| jrosser | there might be a clue there about whats happening | 20:41 |
| *** xgerman has joined #openstack-ansible | 20:42 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_keystone master: Update playbook to newer syntax. https://review.openstack.org/619932 | 20:43 |
| jrosser | jamesdenton: i think that some of the default timeouts for apt are really quite long - this might be similar to what you were seeing in the gate? | 20:44 |
| jamesdenton | it's possible - i don't have a failure to look at anymore. I rechecked twice today and finally got a +1 | 20:45 |
| gisak | here is the /var/log/lxc-cache-prep-commands.log outout:>>> http://paste.openstack.org/show/736184/ | 20:46 |
| jamesdenton | but yesterday I did have the issue gisak is seeing in the gate. but it was fleeting | 20:47 |
| gisak | now running the same task, hope it will pass successfully | 20:48 |
| jamesdenton | you bump the timeout? | 20:48 |
| gisak | yes i put it 1800 in /etc/openstack_deploy/user_variables.yml | 20:49 |
| gisak | http://paste.openstack.org/show/736185/ | 20:50 |
| gisak | reached 130 left already | 20:50 |
| jamesdenton | K | 20:51 |
| *** jawad_axd has quit IRC | 20:58 | |
| *** hamzaachi has quit IRC | 21:03 | |
| *** ahosam has joined #openstack-ansible | 21:04 | |
| *** DanyC has joined #openstack-ansible | 21:07 | |
| gisak | well ... the cache was successfully configured, but still fails to setup correctly containers .. | 21:09 |
| gisak | http://paste.openstack.org/show/736186/ | 21:10 |
| gisak | this errors are my nightmares for few days already ) | 21:11 |
| *** DanyC has quit IRC | 21:11 | |
| jamesdenton | what was the retry counter at when it finally finished that task? | 21:17 |
| jamesdenton | just curious | 21:17 |
| gisak | FAILED - RETRYING: Ensure that the LXC cache has been prepared (38 retries left). | 21:19 |
| gisak | changed: [controller01] | 21:19 |
| gisak | now I destroyed all containers and rerun the playbooks, lets see what will happen this time | 21:20 |
| *** ivve has quit IRC | 21:21 | |
| *** cshen has quit IRC | 21:21 | |
| gisak | cant believe to my eyes, I think its working :) | 21:22 |
| gisak | give a few min to check it :D | 21:23 |
| gisak | yeah, its still running, but at least now I see containers running as they should | 21:26 |
| gisak | yes! it finally worked! :D | 21:33 |
| jamesdenton | so it's moved beyond this task? TASK [lxc_container_create : Gather container facts] | 21:34 |
| gisak | yeah this TASK failed, and eventually all others were stopping | 21:35 |
| gisak | now i run openstack-ansible setup-infrastructure.yml playbook | 21:36 |
| jamesdenton | Ok cool. Smooth sailing from here :D | 21:36 |
| gisak | yeah! thank you guys so much :) | 21:39 |
| jamesdenton | glad to help. | 21:39 |
| gisak | if knew you personally, beer would be by my side for sure :D | 21:39 |
| jamesdenton | it's good to know that no matter where I go, a beer awaits me | 21:40 |
| gisak | :D | 21:40 |
| gisak | thats a good thing, in case you'll trip in Moldova, don't hesitate to contact ;) | 21:41 |
| *** cshen has joined #openstack-ansible | 21:41 | |
| *** ansmith has quit IRC | 21:41 | |
| jamesdenton | jrosser mnaser cloudnull sorry to bug, but if you have time can you bless https://review.openstack.org/#/c/584069 | 21:42 |
| jamesdenton | thanks gisak :) | 21:42 |
| jonher | https://review.openstack.org/581248/ is awaiting +workflow :) | 21:44 |
| *** pcaruana has quit IRC | 21:46 | |
| *** priteau has quit IRC | 21:47 | |
| openstackgerrit | Merged openstack/openstack-ansible master: Update playbook to newer syntax. https://review.openstack.org/619933 | 22:03 |
| *** ahosam has quit IRC | 22:24 | |
| *** ansmith has joined #openstack-ansible | 22:26 | |
| jrosser | mnaser: could you check that this has done the right thing? http://logs.openstack.org/13/620313/3/check/openstack-ansible-deploy-aio_lxc-centos-7/f01d063/job-output.txt.gz#_2018-11-27_20_26_09_187053 | 22:27 |
| *** dcdamien has joined #openstack-ansible | 22:32 | |
| *** gisak has quit IRC | 22:46 | |
| *** dcdamien has quit IRC | 22:52 | |
| *** dcdamien has joined #openstack-ansible | 22:54 | |
| admin0 | "fatal: [c2_ceph-mon_container-cf1019d3]: FAILED! => {"failed": true, "msg": "'dict object' has no attribute u'ansible_all_ipv4_addresses'"}" -- comes on a rerun of the playbook .. what coul dbe the issue | 22:54 |
| *** strattao has joined #openstack-ansible | 22:54 | |
| *** priteau has joined #openstack-ansible | 22:54 | |
| *** strattao has quit IRC | 22:57 | |
| *** radeks has quit IRC | 22:57 | |
| *** priteau has quit IRC | 22:58 | |
| cloudnull | jamesdenton done | 23:10 |
| cloudnull | also the change for systemd-service has merged, just waiting for the cherry-pick to get back into rocky . | 23:11 |
| *** aedc has joined #openstack-ansible | 23:14 | |
| admin0 | hmm.. rerunning the playbooks without limit fixed | 23:18 |
| *** DanyC has joined #openstack-ansible | 23:21 | |
| *** DanyC has quit IRC | 23:26 | |
| *** dcdamien has quit IRC | 23:30 | |
| *** DanyC has joined #openstack-ansible | 23:42 | |
| *** DanyC has quit IRC | 23:46 | |
| *** jhesketh_ is now known as jhesketh | 23:51 | |
| openstackgerrit | Guilherme Steinmuller Pimentel proposed openstack/openstack-ansible-os_placement master: [WIP] Create base files to install placement https://review.openstack.org/618820 | 23:59 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!