| *** dcdamien has quit IRC | 00:20 | |
| *** hamerins has quit IRC | 00:24 | |
| *** hamerins has joined #openstack-ansible | 00:27 | |
| *** hamerins has quit IRC | 00:30 | |
| *** weezS has quit IRC | 00:33 | |
| *** vnogin has joined #openstack-ansible | 00:33 | |
| *** xjra has quit IRC | 00:37 | |
| *** vnogin has quit IRC | 00:37 | |
| *** ansmith has joined #openstack-ansible | 00:41 | |
| *** DanyC has joined #openstack-ansible | 00:48 | |
| *** tosky has quit IRC | 00:50 | |
| *** DanyC has quit IRC | 00:52 | |
| *** markvoelker has joined #openstack-ansible | 01:23 | |
| *** markvoelker has quit IRC | 01:28 | |
| *** cshen has joined #openstack-ansible | 01:46 | |
| *** trident has quit IRC | 01:50 | |
| *** cshen has quit IRC | 01:51 | |
| *** gyee has quit IRC | 01:54 | |
| *** trident has joined #openstack-ansible | 01:55 | |
| *** hamerins has joined #openstack-ansible | 02:00 | |
| *** markvoelker has joined #openstack-ansible | 02:30 | |
| *** hamerins has quit IRC | 02:33 | |
| *** radeks has joined #openstack-ansible | 02:55 | |
| openstackgerrit | Merged openstack/openstack-ansible stable/pike: Update all SHAs for 16.0.23 https://review.openstack.org/618840 | 03:08 |
|---|---|---|
| *** maddtux has joined #openstack-ansible | 03:15 | |
| *** hamerins has joined #openstack-ansible | 03:20 | |
| *** hamerins has quit IRC | 03:34 | |
| *** cshen has joined #openstack-ansible | 03:47 | |
| *** cshen has quit IRC | 03:51 | |
| *** chhagarw has joined #openstack-ansible | 04:15 | |
| *** vnogin has joined #openstack-ansible | 04:34 | |
| *** dave-mccowan has quit IRC | 04:34 | |
| *** dave-mccowan has joined #openstack-ansible | 04:35 | |
| *** vnogin has quit IRC | 04:38 | |
| *** dave-mccowan has quit IRC | 04:49 | |
| *** udesale has joined #openstack-ansible | 05:06 | |
| *** cshen has joined #openstack-ansible | 05:49 | |
| *** jackivanov has joined #openstack-ansible | 05:49 | |
| *** cshen has quit IRC | 05:54 | |
| *** klamath has quit IRC | 06:15 | |
| *** klamath has joined #openstack-ansible | 06:15 | |
| *** radeks has quit IRC | 06:20 | |
| *** radeks has joined #openstack-ansible | 06:26 | |
| *** DanyC has joined #openstack-ansible | 06:59 | |
| *** fghaas has joined #openstack-ansible | 07:01 | |
| *** ahosam has joined #openstack-ansible | 07:02 | |
| *** DanyC has quit IRC | 07:04 | |
| *** fghaas has quit IRC | 07:05 | |
| *** chkumar|away has quit IRC | 07:13 | |
| *** chandan_kumar has joined #openstack-ansible | 07:15 | |
| *** cshen has joined #openstack-ansible | 07:24 | |
| *** ahosam has quit IRC | 07:26 | |
| *** ahosam has joined #openstack-ansible | 07:26 | |
| *** hamzaachi has joined #openstack-ansible | 07:50 | |
| openstackgerrit | Chandan Kumar proposed openstack/openstack-ansible-os_tempest master: Enable support to remove unnecessary tempest options from conf https://review.openstack.org/620800 | 08:01 |
| chandan_kumar | odyssey4me: Hello | 08:04 |
| chandan_kumar | odyssey4me: in openstack-ansible where we generate clouds.yaml file? | 08:04 |
| *** gkadam has joined #openstack-ansible | 08:07 | |
| *** fghaas has joined #openstack-ansible | 08:16 | |
| *** fghaas has quit IRC | 08:25 | |
| *** hamzaachi_ has joined #openstack-ansible | 08:26 | |
| *** hamzaachi has quit IRC | 08:27 | |
| *** olivierbourdon38 has joined #openstack-ansible | 08:27 | |
| *** hamzaachi__ has joined #openstack-ansible | 08:28 | |
| *** hamzaachi_ has quit IRC | 08:31 | |
| *** hamzaachi_ has joined #openstack-ansible | 08:36 | |
| *** hamzaachi__ has quit IRC | 08:39 | |
| *** hamzaachi__ has joined #openstack-ansible | 08:41 | |
| *** fghaas has joined #openstack-ansible | 08:42 | |
| *** hamzaachi_ has quit IRC | 08:45 | |
| *** pcaruana has joined #openstack-ansible | 08:48 | |
| *** markvoelker has quit IRC | 08:50 | |
| *** tosky has joined #openstack-ansible | 09:03 | |
| *** shardy has joined #openstack-ansible | 09:04 | |
| noonedeadpunk | jamesdenton oh, great! Thanks for the notice:) | 09:09 |
| *** dcdamien has joined #openstack-ansible | 09:10 | |
| *** ahosam has quit IRC | 09:10 | |
| *** ahosam has joined #openstack-ansible | 09:10 | |
| *** fghaas has quit IRC | 09:10 | |
| *** shardy has quit IRC | 09:13 | |
| noonedeadpunk | It seems, that I may abandon then https://review.openstack.org/#/c/620076/2 | 09:13 |
| noonedeadpunk | Or at least place it for rocky/queens, if https://review.openstack.org/#/c/333829/7 won't be backported | 09:14 |
| *** DanyC has joined #openstack-ansible | 09:16 | |
| *** hamzaachi_ has joined #openstack-ansible | 09:20 | |
| *** hamzaachi__ has quit IRC | 09:22 | |
| *** kaiokmo has quit IRC | 09:38 | |
| *** DimGR has joined #openstack-ansible | 09:40 | |
| *** shardy has joined #openstack-ansible | 09:51 | |
| *** markvoelker has joined #openstack-ansible | 09:51 | |
| *** flaviosr has quit IRC | 09:54 | |
| *** flaviosr has joined #openstack-ansible | 09:57 | |
| *** hamzaachi__ has joined #openstack-ansible | 10:06 | |
| *** ahosam has quit IRC | 10:06 | |
| *** hamzaachi_ has quit IRC | 10:08 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: Set the utility container service setup interpreter automatically https://review.openstack.org/620651 | 10:11 |
| odyssey4me | cloudnull given it's only really for stable, I'd say let's just avoid overwriting any system implementations | 10:15 |
| *** cshen has quit IRC | 10:16 | |
| odyssey4me | ktims by default both ceph-ansible and we lay the repo down IIRC, I don't know why you'd be stuck on lxc_hosts though.... what exactly are you stuck on? | 10:17 |
| odyssey4me | chandan_kumar we use the openstack_openrc role at the moment, although it's not particularly universal - it might be nice to evolve that to something more generally useful, or to switch to another role that already is | 10:18 |
| odyssey4me | evrardjp are you around to look at https://review.openstack.org/#/c/620607/ ? it's a quick voting0->non-voting patch as agreed yesterday | 10:20 |
| *** cshen has joined #openstack-ansible | 10:22 | |
| *** fresta has joined #openstack-ansible | 10:22 | |
| *** markvoelker has quit IRC | 10:24 | |
| *** udesale has quit IRC | 10:27 | |
| *** ahosam has joined #openstack-ansible | 10:31 | |
| *** sum12 has quit IRC | 10:32 | |
| *** flaviosr has quit IRC | 10:33 | |
| *** mbuil has quit IRC | 10:33 | |
| admin0 | anyone faced issue of a controller reboot with the ceph mon, and when you bring it back up, its out of quorum .. and running ansible playbook gives TASK [ceph-mon : collect admin and bootstrap keys] failed": true, "msg": "non-zero return code", "rc": 1, "start": "2018-11-29 10:33:46.040584", "stderr": "INFO:ceph-create-keys:ceph-mon is not in quorum: u'probing'\nINFO:ceph-create-keys:ceph-mon is not in quorum: | 10:39 |
| *** shardy has quit IRC | 10:41 | |
| *** dcdamien has quit IRC | 10:42 | |
| *** shardy has joined #openstack-ansible | 10:43 | |
| *** dcdamien has joined #openstack-ansible | 10:44 | |
| *** hamzaachi__ has quit IRC | 10:46 | |
| evrardjp | odyssey4me: | 10:46 |
| evrardjp | hey I am here | 10:47 |
| evrardjp | will look | 10:47 |
| evrardjp | oh already voted this morning | 10:47 |
| evrardjp | ping if there is somehting else | 10:47 |
| odyssey4me | thanks evrardjp | 10:49 |
| evrardjp | np | 10:49 |
| odyssey4me | evrardjp this one will also helpo to grease the wheels: https://review.openstack.org/#/c/620629/2 | 10:49 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible stable/rocky: Update all SHAs for 18.1.1 https://review.openstack.org/618822 | 10:50 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible stable/rocky: Add extra volume types to AIO https://review.openstack.org/617053 | 10:51 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible stable/rocky: Update playbook to newer syntax. https://review.openstack.org/620569 | 10:51 |
| evrardjp | okay, I am not super familiar with RDO but it makes sense and is a backport | 10:51 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible stable/rocky: Remove unnecessary octavia scenario AIO bootstrap https://review.openstack.org/619792 | 10:52 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible stable/rocky: Ensure AIO container_tech/install_method vars are namespaced https://review.openstack.org/620278 | 10:52 |
| evrardjp | thanksfor the rebase | 10:52 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible stable/rocky: Implement documentation changes for translations https://review.openstack.org/620279 | 10:52 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible stable/rocky: [docs] Clean up the AIO user story https://review.openstack.org/620280 | 10:52 |
| odyssey4me | evrardjp ok, that should unblock rocky and there's a whole chain of patches after those now rebased to get them passing | 10:53 |
| odyssey4me | evrardjp what do you think of https://review.openstack.org/620651 ? | 10:54 |
| odyssey4me | I think it's nice from a functionality standpoint, but it's messy. I'm thinking that I might rather want to just document the two settings and add a commented example into user_variables, rather than carry that jinja script variable | 10:55 |
| *** aedc has joined #openstack-ansible | 10:57 | |
| *** electrofelix has joined #openstack-ansible | 11:04 | |
| *** sum12 has joined #openstack-ansible | 11:19 | |
| *** markvoelker has joined #openstack-ansible | 11:21 | |
| odyssey4me | hmm, that's nice - jrosser have you seen http://logs.openstack.org/91/551791/42/check/openstack-ansible-deploy-aio_ceph-ubuntu-bionic/822ab39/job-output.txt.gz#_2018-11-29_02_47_35_014673 - that's now what's failing master's builds | 11:26 |
| odyssey4me | Nov 29 02:47:55 localhost ceph-osd-prestart.sh[30605]: OSD data directory /var/lib/ceph/osd/ceph-loop6 does not exist; bailing out. | 11:27 |
| chandan_kumar | odyssey4me: evrardjp https://review.openstack.org/#/c/620800/ please have a look | 11:30 |
| *** ahosam has quit IRC | 11:45 | |
| *** markvoelker has quit IRC | 11:55 | |
| *** cshen has quit IRC | 12:05 | |
| *** cshen has joined #openstack-ansible | 12:14 | |
| *** dave-mccowan has joined #openstack-ansible | 12:16 | |
| *** flaviosr has joined #openstack-ansible | 12:22 | |
| *** gkadam_ has joined #openstack-ansible | 12:23 | |
| *** gisak has joined #openstack-ansible | 12:23 | |
| gisak | hello guys | 12:24 |
| chandan_kumar | odyssey4me: https://review.openstack.org/#/c/620800/ and https://review.openstack.org/#/c/619986/4 both does different tasks | 12:25 |
| gisak | facing some problems with haproxy service during playbooks, could somebody please advice me with this? | 12:25 |
| *** gkadam has quit IRC | 12:25 | |
| chandan_kumar | odyssey4me: I am not getting how it is similar need help here | 12:25 |
| odyssey4me | chandan_kumar ok, I see - quote honestly, I don't see why there needs to be two options - why not just have one generalised option, something like tempestconf_cli_parameters - then use that for anything extra you want to add into the CLI | 12:26 |
| odyssey4me | gisak you'd have to provide specifics for anyone to help - please share logs and failure messages via a paste service or osmething similar | 12:27 |
| *** gkadam_ has quit IRC | 12:28 | |
| *** gkadam has joined #openstack-ansible | 12:28 | |
| gisak | sure, sec please | 12:29 |
| gisak | http://paste.openstack.org/show/736399/ | 12:30 |
| *** ansmith has quit IRC | 12:31 | |
| gisak | after rerunning the setup-infrastructure.yml playbook, now I get the following error: | 12:44 |
| gisak | http://paste.openstack.org/show/736401/ | 12:45 |
| jamesdenton | Try configuring 172.29.236.11 as a secondary IP on br-mgmt | 12:45 |
| jamesdenton | that is, if you don't see it on the server currently | 12:46 |
| gisak | 172.29.236.11 is br-mgmt IP for controller host | 12:46 |
| openstackgerrit | Merged openstack/openstack-ansible stable/rocky: Set SUSE ceph-distro job to non-voting https://review.openstack.org/620607 | 12:49 |
| jamesdenton | you should consider using a different address for the VIP. What is the output of "netstat -plnt | grep 172.29.236.11"? | 12:51 |
| *** markvoelker has joined #openstack-ansible | 12:52 | |
| pabelanger | morning | 12:52 |
| pabelanger | odyssey4me: did you see my comments yesterday about also making changes to keystone_service_setup_host_python_interpreter for all other services too? Is that something you are going to do, or should I start to push up patches | 12:53 |
| *** vakuznet has quit IRC | 12:59 | |
| *** shardy has quit IRC | 13:03 | |
| *** maddtux has quit IRC | 13:03 | |
| chandan_kumar | odyssey4me: evrardjp you mean something like this http://paste.openstack.org/show/736405/ ? | 13:07 |
| *** shardy has joined #openstack-ansible | 13:10 | |
| evrardjp | odyssey4me mnaser I think you should join #ansible-community | 13:13 |
| *** ansmith has joined #openstack-ansible | 13:23 | |
| *** markvoelker has quit IRC | 13:24 | |
| *** gkadam_ has joined #openstack-ansible | 13:25 | |
| *** gkadam has quit IRC | 13:28 | |
| *** kaiokmo has joined #openstack-ansible | 13:31 | |
| *** vakuznet has joined #openstack-ansible | 13:31 | |
| vakuznet | second review please https://review.openstack.org/#/c/620399 | 13:34 |
| gisak | http://paste.openstack.org/show/736407/ | 13:36 |
| gisak | I am never gonna install correctly openstack .. | 13:37 |
| jamesdenton | not true! | 13:38 |
| jamesdenton | 172.29.236.11 is on your controller node, but you're running haproxy on the network node? | 13:39 |
| gisak | yes | 13:39 |
| jamesdenton | are you intending on running haproxy on both the controller and the network node? | 13:39 |
| gisak | no, just network | 13:40 |
| gisak | network node has 172.29.236.10 on br-mgmt | 13:40 |
| jamesdenton | ok - that's likely your issue right now. All of the API services live on the controller, and the VIP is configured on the controller. haproxy can't run on the network node and bind to the IP you've chosen for the VIP. | 13:40 |
| jamesdenton | for now, you might consider just configuring a secondary IP for the VIP on the network node (i.e. 172.29.236.8) and updating openstack_user_config.yml to use that IP instead | 13:41 |
| jamesdenton | same for the external VIP address | 13:41 |
| gisak | i need to have haproxy on network, cause probably controller node might be down sometimes during production | 13:42 |
| jamesdenton | well this is a 3-node cluster right now, right? 1 controller, 1 network, 1 compute? | 13:42 |
| gisak | so instances could run without problems with compute and network nodes up | 13:43 |
| *** vollman has joined #openstack-ansible | 13:43 | |
| gisak | yes | 13:43 |
| openstackgerrit | Merged openstack/openstack-ansible stable/rocky: Add MariaDB infrastructure mirrors https://review.openstack.org/619714 | 13:43 |
| openstackgerrit | Merged openstack/openstack-ansible stable/rocky: Ensure that a consistent mirror is used for RDO https://review.openstack.org/620629 | 13:43 |
| jamesdenton | Not really a production setup at the moment, but i get what you're saying. In prod, you'd have haproxy running on multiple nodes w/ keepalived keeping the VIP straight | 13:44 |
| jamesdenton | in this scenario, given it's your first time, i would try to keep things as simple as possible | 13:44 |
| *** vollman has quit IRC | 13:44 | |
| gisak | well I am looking for a setup, that could be completed in the close future | 13:45 |
| gisak | + 2 compute, + storages etc.. | 13:45 |
| gisak | but for the start I need these 3 nodes | 13:46 |
| *** vollman has joined #openstack-ansible | 13:46 | |
| jamesdenton | Well, you'll need an IP that can float around (configured by keepalived on the active node), so it can't be the primary br-mgmt ip for any host. you'll need to have a secondary address (from the same subnet) | 13:47 |
| gisak | not sure that fully understand | 13:49 |
| jamesdenton | the point of haproxy is the load balance services. you can have multiple instances of haproxy scattered across infra hosts that can provide redundancy. keepalived is used to determine which of those load balancers is the ACTIVE load balancer. the others remain in a STANDBY state. If the active haproxy instance failed (i.e. controller node hosting haproxy goes down), then one of the standby nodes becomes active. | 13:52 |
| jamesdenton | For this to work, the VIP address must be able to float between nodes based on which keepalived becomes master | 13:52 |
| jamesdenton | If you only have 1 instance of haproxy, that fine for a lab scenario. The VIP, however, must be an address local to that host. Since you are using the network node, the VIP would need to be 172.29.236.10 and not 172.29.236.11, which is configured on the controller. haproxy on the network node cannot bind to an IP address that is not configured.. on that node. | 13:53 |
| jamesdenton | make sense? | 13:53 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible stable/rocky: Move ARA install to end of bootstrap https://review.openstack.org/617785 | 13:54 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible stable/rocky: Add openstackclient bash completion https://review.openstack.org/619354 | 13:55 |
| gisak | yes, now understand, then one more question: if i will bind internal_lb_vip_address to 172.29.236.10, will it work even if all containers except neutron are on controller node ? | 13:56 |
| jamesdenton | it should be fine, yes. haproxy.cfg will use 172.29.236.10 for the front end address and all of the respective backend (container) addresses will be configured for the respective pools | 13:57 |
| jamesdenton | so as long as you can ping those containers from the network node you should be OK | 13:58 |
| gisak | what happens if controller node fails while instances are running on compute, will they still be up ? | 13:59 |
| gisak | of course the network is up to | 13:59 |
| gisak | of course the network is up too | 13:59 |
| jamesdenton | i guess the answer is, it depends. | 14:00 |
| jamesdenton | If you're using VLAN networks without virtual routers, then you shouldn't see an impact to VM traffic. If you are using routers, those would be hosted on the network node and you shouldn't be impacted there, either, in most cases. You wouldn't be able to spin up new instances (obviously) and API would be impacted | 14:01 |
| jamesdenton | you design redundancy with multiple controller/infra nodes and a load balancer to limit or eliminate impact | 14:02 |
| gisak | yes I understand that no dashboard, API will not work, i just want my instances to be up while controller is down | 14:02 |
| jamesdenton | with the proper architecture that should not be a concern | 14:03 |
| odyssey4me | pabelanger morning - sorry, only just saw your msg... I could do with help pushing up for the rest of the roles | 14:05 |
| gisak | thanks for advice <jamesdenton> I appreciate that :) | 14:05 |
| odyssey4me | pabelanger did you see I updated the integrated repo patch - although quite honestly I'm not sure doing it is a good idea | 14:05 |
| jamesdenton | sure thing, gisak. I recommend getting a few installs under your belt using tried-and-true methods before deviating too much, that way it's easier to see where things went wrong. | 14:06 |
| gisak | its pretty complex all this infra, not sure I could reach even this step without this channel :) | 14:08 |
| odyssey4me | logan- jrosser there's a new failure in master for the ceph/bionic build ... any chance you guys could dig into it? | 14:09 |
| odyssey4me | logan- jrosser mnaser http://eavesdrop.openstack.org/irclogs/%23openstack-ansible/latest.log.html#t2018-11-29T11:26:09 FYI | 14:11 |
| odyssey4me | no master builds will pass until that's sorted out, and I'm tied up elsewhere right now | 14:11 |
| mnaser | odyssey4me: boo, ok | 14:20 |
| mnaser | i will dig in | 14:20 |
| odyssey4me | thanks mnaser | 14:23 |
| mnaser | found the erorr messasge | 14:30 |
| mnaser | "OSD data directory /var/lib/ceph/osd/ceph-loop6 does not exist; bailing out." | 14:30 |
| odyssey4me | mnaser yep, that's in my eavesdrop link above :p | 14:31 |
| mnaser | ugh | 14:31 |
| mnaser | maybe i should read more | 14:31 |
| mnaser | llol | 14:31 |
| mnaser | ok the weird thing is | 14:31 |
| mnaser | you're not supposed to do | 14:31 |
| mnaser | systemctl start ceph-osd@<dev> | 14:31 |
| mnaser | its supposed to be systemctl start ceph-osd@<osd-id> | 14:31 |
| *** KeithMnemonic has joined #openstack-ansible | 14:32 | |
| gisak | jamesdenton: this error is the last one I reached yesterday and so I do today .. : http://paste.openstack.org/show/736413/ | 14:33 |
| jamesdenton | gisak i recommend connecting to controller01_glance_container-5a510c22 and run 'systemctl status glance-api' and see why it failed | 14:34 |
| jamesdenton | or systemctl status var-lib-glance-images.mount | 14:35 |
| gisak | http://paste.openstack.org/show/736414/ | 14:36 |
| gisak | well it is running | 14:36 |
| pabelanger | odyssey4me: I did see the update, thanks. I don't mind we we skip landing that, I can see it being too much magic. As long as I can set it up myself I am fine | 14:37 |
| gisak | but this is the log of syslog on controller: http://paste.openstack.org/show/736415/ | 14:37 |
| jamesdenton | sorry, i meant systemctl status var-lib-glance-images.mount | 14:38 |
| *** mma has joined #openstack-ansible | 14:40 | |
| gisak | http://paste.openstack.org/show/736417/ | 14:40 |
| *** mma_ has joined #openstack-ansible | 14:41 | |
| jamesdenton | Do you have this block? https://github.com/openstack/openstack-ansible/blob/master/etc/openstack_deploy/openstack_user_config.yml.prod.example#L148-L155 | 14:43 |
| *** mma has quit IRC | 14:45 | |
| gisak | http://paste.openstack.org/show/736418/ | 14:45 |
| jamesdenton | ok, yeah. that's a configuration that let's you mount an nfs volume for storing glance images. Just remove the container_vars line [112] thru line 119 | 14:46 |
| odyssey4me | mnaser I dunno if this shed any light on the matter, but https://github.com/ceph/ceph-ansible/commit/2cea33f7fc4bf59eaa249ca26ba326105e392402 merged 12 hours ago into stable-3.2, which seems to co-incide with the new failures | 14:46 |
| gisak | that is the openstack_user_config.yml I run | 14:46 |
| jamesdenton | understood. it's trying to reach out to an nfs target that i assume doesn't exist at 172.29.244.11 | 14:47 |
| gisak | well, but in this case I will not have any images container ... | 14:49 |
| gisak | maybe I should just put instead of 172.29.244.11, the glance container's IP ? | 14:50 |
| gisak | or network's ? | 14:50 |
| Miouge- | jamesdenton: doesn’t that block assume that an NFS server is available at 172.29.244.15 ? | 14:51 |
| jamesdenton | no, it will simply store the images locally inside the glance container | 14:51 |
| jamesdenton | Miouge- Yes, in that example it would be 172.29.244.15. | 14:52 |
| jamesdenton | But in gisak's existing config, it's 172.29.244.11, which is likely the ip configured on br-storage if i had to guess | 14:52 |
| jamesdenton | The IP would need to be an NFS server in the storage network | 14:53 |
| Miouge- | gisak: is there an NFS server there? I think the prod example relies on an external NAS for NFS | 14:53 |
| *** SimAloo has joined #openstack-ansible | 14:54 | |
| jamesdenton | if you eliminate that 'glance_nfs_client' override, then glance images are stored locally, which is fine for a single glance deploy | 14:54 |
| gisak | yes it is on br-storage | 14:54 |
| gisak | so I should either have an NFS server on controller node, or store images locally on glance container | 14:55 |
| jamesdenton | yeah, so forget that particular configuration for now by removing it, and let the images live locally. Once you have NFS configured (on something other than the controller node) and have multiple glance services needing a shared backend, you can put it back | 14:56 |
| gisak | done it, but still the same issue | 14:58 |
| *** udesale has joined #openstack-ansible | 14:58 | |
| jamesdenton | you reran the playbook and it failed in the same place? | 14:59 |
| gisak | yes | 14:59 |
| admin0 | hello \o | 15:02 |
| admin0 | quick gnocchi question .. is the gnocchi stats also replicated in all the controllers ? | 15:02 |
| odyssey4me | admin0 I know very little about gnocchi, but I would guess that depends on what the storage back-end is for it. | 15:05 |
| openstackgerrit | Merged openstack/openstack-ansible-os_tempest master: Added task to list tempest tests https://review.openstack.org/619024 | 15:05 |
| admin0 | hmm.. i used the default ansible install :) | 15:05 |
| admin0 | just specified the infra hosts for it | 15:06 |
| Miouge- | gisak: can you dump the output in paste? If you removed the 'glance_nfs_client’ it shouldn’t try to mount that NFS | 15:06 |
| admin0 | issue what i see is not all UUIDs are in all servers .. so they are either in 2 or skipped in 1 .. or only in 1 .. | 15:06 |
| admin0 | biggest issue is df -i reaching 99% and this folder being the culprit | 15:06 |
| admin0 | when df-i reaches 100%, the keepalive/vip ip suddenly disappears | 15:07 |
| odyssey4me | well yes, it's likely that the default back-end storage is file, and that's not good unless you have NFS file storage or something | 15:08 |
| odyssey4me | the telemetry systems don't get much attention in OSA because of the smaller user-base... it'd be nice to see it grow and to have more sensible defaults | 15:08 |
| odyssey4me | it'd even be nice to see a panko playbook integrated | 15:08 |
| admin0 | odyssey4me, file it is . and i do not see option for nfs in playbook .. means the common share needs to be done manually in the container before service can start | 15:12 |
| odyssey4me | admin0 well, I think gnocchi has database/swift back-end options - you'll have to explore and understand the service better - I know nothing :p | 15:13 |
| gisak | well strange, I have just installed nfs-kernel-server on controller node and mounted /images folder on 172.29.244.11, and still getting this error | 15:17 |
| gisak | kinda like that looks my /etc/exports:>> "/images 172.29.244.11/24(rw,no_root_squash)" | 15:20 |
| *** hamerins has joined #openstack-ansible | 15:23 | |
| Miouge- | gisak: and a test mount works? | 15:24 |
| *** udesale has quit IRC | 15:29 | |
| *** udesale has joined #openstack-ansible | 15:29 | |
| *** hamerins has quit IRC | 15:30 | |
| *** hamerins has joined #openstack-ansible | 15:31 | |
| gisak | but for that I need a nfs client on glance container, right ? | 15:35 |
| Miouge- | gisak: first I would try to mount it from somewhere else to check that the NFS server is ready to go | 15:39 |
| *** ahosam has joined #openstack-ansible | 15:39 | |
| *** cshen has quit IRC | 15:41 | |
| *** neith has joined #openstack-ansible | 15:42 | |
| mnaser | odyssey4me: fyi our broken gate is due to https://github.com/ceph/ceph-ansible/issues/3388 | 15:46 |
| mnaser | https://github.com/ceph/ceph-ansible/pull/3391 is the fix | 15:46 |
| jrosser | how is that happening? i thought we were on 3.1? | 15:48 |
| odyssey4me | 3.2 on master | 15:48 |
| jrosser | doh :( | 15:48 |
| odyssey4me | 3.1 on stable/rocky | 15:48 |
| odyssey4me | we could take the option of just using stable until a new stable | 15:49 |
| mnaser | do we have a way of testing a pr in github | 15:49 |
| mnaser | just as a wip to validate that patch | 15:49 |
| odyssey4me | mnaser depends-on: <pr url> | 15:49 |
| mnaser | ah but that wont check things out from ceph-ansible probably.. im thinking a change in ansible-role-requirements | 15:50 |
| odyssey4me | I dunno if that will actually work though, but it's worth a try | 15:50 |
| odyssey4me | a-r-r can do refspecs and such too | 15:50 |
| mnaser | https://github.com/ceph/ceph-ansible/commits/start-ceph-disk-non-container | 15:50 |
| mnaser | ok that branch has the fix | 15:50 |
| mnaser | ill push up a patch to test it | 15:50 |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible master: DNM: test start-ceph-disk-non-container https://review.openstack.org/620944 | 15:51 |
| * logan- pushes up a patch to test the PR with depends-on | 15:53 | |
| openstackgerrit | Logan V proposed openstack/openstack-ansible master: [DNM] Test ceph fix https://review.openstack.org/620946 | 15:53 |
| logan- | i think it should work | 15:53 |
| openstackgerrit | Logan V proposed openstack/openstack-ansible master: [DNM] Test ceph fix https://review.openstack.org/620946 | 15:58 |
| mnaser | i dont think it will logan- because we don't have anything that includes ceph-ansible in our required-projects | 15:59 |
| mnaser | so zuul wont check it out | 15:59 |
| gisak | yes, i was able to mount from 2 different servers, including glance container, now i rerun the playbook | 15:59 |
| gisak | lets see what it gets ) | 15:59 |
| logan- | a project is always considered required when there is a depends-on mnaser | 15:59 |
| mnaser | TIL | 15:59 |
| logan- | the only question is whether our role cloner is smart enough to link it from the zuul src path | 16:00 |
| mnaser | yep thats the #2 | 16:00 |
| logan- | i believe it is though | 16:00 |
| odyssey4me | I think so | 16:03 |
| *** weezS has joined #openstack-ansible | 16:04 | |
| gisak | Miouge-: thanks, it worked, the mistake was in file /etc/exports I should use 177.29.244.11/22 instead of 177.29.244.11/24 :) | 16:04 |
| gisak | 172 * | 16:05 |
| *** mma_ has quit IRC | 16:09 | |
| openstackgerrit | Merged openstack/openstack-ansible stable/rocky: haproxy: remove repo_cache service https://review.openstack.org/620336 | 16:17 |
| *** strattao has joined #openstack-ansible | 16:19 | |
| *** udesale has quit IRC | 16:20 | |
| *** gkadam_ has quit IRC | 16:20 | |
| openstackgerrit | Merged openstack/openstack-ansible-ops master: Allow the option to specify OSA git repo URL https://review.openstack.org/620662 | 16:25 |
| *** hamerins has quit IRC | 16:34 | |
| *** hamerins has joined #openstack-ansible | 16:36 | |
| *** ahosam has quit IRC | 16:46 | |
| *** hamzy has quit IRC | 16:46 | |
| *** cshen has joined #openstack-ansible | 16:56 | |
| *** strattao has quit IRC | 17:03 | |
| *** shardy has quit IRC | 17:09 | |
| *** gyee has joined #openstack-ansible | 17:09 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_tempest master: Enable support to pass override option to tempestconf https://review.openstack.org/619986 | 17:12 |
| *** mma has joined #openstack-ansible | 17:18 | |
| gisak | I saw recently there were some problems regarding this: http://paste.openstack.org/show/736429/ | 17:18 |
| gisak | and it should be fixed in branch, but unfortunately running playbook failed .. | 17:19 |
| *** mma has quit IRC | 17:22 | |
| *** DanyC has quit IRC | 17:23 | |
| *** DanyC has joined #openstack-ansible | 17:24 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: add short name for distros https://review.openstack.org/620967 | 17:28 |
| *** DanyC has quit IRC | 17:29 | |
| *** hamerins has quit IRC | 17:29 | |
| *** hamerins has joined #openstack-ansible | 17:31 | |
| gisak | if add "default('rabbitmq_all')" to # Nova notifications, will it fix this ? | 17:31 |
| gisak | in /etc/ansible/roles/os_ceilometer/templates/ceilometer.conf.j2 file | 17:31 |
| *** DanyC has joined #openstack-ansible | 17:42 | |
| *** DanyC has quit IRC | 17:46 | |
| *** dcdamien has quit IRC | 17:58 | |
| openstackgerrit | Merged openstack/openstack-ansible-lxc_hosts stable/rocky: ensure the mount unit is started after reboot https://review.openstack.org/620390 | 18:06 |
| *** DanyC has joined #openstack-ansible | 18:12 | |
| *** hamzy has joined #openstack-ansible | 18:12 | |
| *** vnogin has joined #openstack-ansible | 18:15 | |
| *** DanyC has quit IRC | 18:16 | |
| *** vakuznet has quit IRC | 18:19 | |
| *** vnogin has quit IRC | 18:20 | |
| *** hamzaachi has joined #openstack-ansible | 18:29 | |
| *** hamzy has quit IRC | 18:31 | |
| *** brad[] has joined #openstack-ansible | 18:38 | |
| *** hamzy has joined #openstack-ansible | 18:52 | |
| *** chhagarw has quit IRC | 18:53 | |
| *** strattao has joined #openstack-ansible | 18:54 | |
| *** hamzy_ has joined #openstack-ansible | 18:57 | |
| *** hamzy has quit IRC | 18:58 | |
| *** hamzaachi has quit IRC | 18:59 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Add env options https://review.openstack.org/620991 | 19:01 |
| jrosser | cloudnull: superb patch there :) | 19:05 |
| jrosser | Understand where that is coming from..... | 19:06 |
| *** strattao has quit IRC | 19:06 | |
| *** chhagarw has joined #openstack-ansible | 19:06 | |
| *** electrofelix has quit IRC | 19:12 | |
| cloudnull | will have more in a min | 19:13 |
| cloudnull | working with my folks trying to make that go | 19:13 |
| *** francois has quit IRC | 19:19 | |
| *** francois has joined #openstack-ansible | 19:20 | |
| openstackgerrit | James Denton proposed openstack/openstack-ansible-os_neutron master: Install haproxy on ovn-controller nodes https://review.openstack.org/620994 | 19:23 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Updated the connection plugins and add env options https://review.openstack.org/620991 | 19:30 |
| cloudnull | ^ better commit message :) | 19:30 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Updated the connection plugins and add env options https://review.openstack.org/620991 | 19:31 |
| *** gisak has quit IRC | 19:35 | |
| *** markvoelker has joined #openstack-ansible | 19:36 | |
| *** markvoelker has quit IRC | 19:40 | |
| admin0 | i see ovn :) | 19:41 |
| admin0 | nice | 19:41 |
| admin0 | jamesdenton, good book .. thanks for writing it | 19:42 |
| jamesdenton | OVN is coming along. Thanks for the feedback! | 19:42 |
| *** chhagarw has quit IRC | 19:48 | |
| *** derks has joined #openstack-ansible | 19:55 | |
| derks | I am having trouble with openstack-aio deployment. for specific purpose we need to disable ssl on the public ports. for some reason setting 'openstack_external_ssl: false' and 'openstack_service_publicuri_proto: http' in /etc/openstack_deploy/user_variables.yml is not taking | 19:57 |
| derks | haproxy still sets up ssl on the public ports | 19:57 |
| *** DanyC has joined #openstack-ansible | 19:57 | |
| *** DanyC has quit IRC | 19:58 | |
| *** DanyC has joined #openstack-ansible | 19:58 | |
| *** markvoelker has joined #openstack-ansible | 20:00 | |
| *** cshen has quit IRC | 20:00 | |
| *** cshen has joined #openstack-ansible | 20:08 | |
| openstackgerrit | Guilherme Steinmuller Pimentel proposed openstack/openstack-ansible-os_placement master: [WIP] Create base files to install placement https://review.openstack.org/618820 | 20:25 |
| *** DanyC_ has joined #openstack-ansible | 20:34 | |
| *** radeks has quit IRC | 20:34 | |
| *** radeks has joined #openstack-ansible | 20:34 | |
| *** DanyC has quit IRC | 20:34 | |
| *** DanyC_ has quit IRC | 20:34 | |
| *** mmercer has joined #openstack-ansible | 20:37 | |
| *** DanyC has joined #openstack-ansible | 20:42 | |
| *** DanyC has quit IRC | 20:46 | |
| *** ansmith has quit IRC | 20:51 | |
| *** olivierbourdon38 has quit IRC | 20:57 | |
| *** ivve has quit IRC | 21:01 | |
| *** hamerins has quit IRC | 21:05 | |
| nsmeds | does openstack-ansible use the `is_admin_project` configuration option anywhere? run into an interesting error | 21:18 |
| nsmeds | https://gist.github.com/nikosmeds/6bc29b00c05222f94fa62a3adfffff5a | 21:19 |
| *** hamerins has joined #openstack-ansible | 21:34 | |
| jrosser | cloudnull: a fair few of these recently http://logs.openstack.org/53/617053/4/check/openstack-ansible-deploy-aio_lxc-ubuntu-xenial/aa73ff3/job-output.txt.gz#_2018-11-29_17_53_35_880865 | 21:37 |
| *** markvoelker has quit IRC | 21:38 | |
| jamesdenton | it's been a tough week for the gate | 21:38 |
| jrosser | looks like the apt package install just stops dead in it's tracks part way through, with a couple of errors in the cache prep log to do with /dev/pts | 21:38 |
| *** markvoelker has joined #openstack-ansible | 21:38 | |
| jrosser | the timestamp on the cache prep log file is several minutes earlier than when the retries run out, so it looks to have just stalled completely somehow | 21:39 |
| jonher | nsmeds: same error on re-run of the playbook? Looking at https://github.com/openstack/openstack-ansible-os_keystone/blob/stable/queens/tasks/keystone_service_setup.yml#L51 and https://github.com/openstack/openstack-ansible-plugins/blob/stable/queens/library/keystone I don't see "is_admin_project" being used at least | 21:40 |
| *** markvoelker has quit IRC | 21:43 | |
| nsmeds | jonher: I'll go attempt two runs in a row and confirm, be back in a bit | 21:45 |
| *** hamzy_ has quit IRC | 21:47 | |
| *** radeks has quit IRC | 21:52 | |
| nsmeds | jonher: yep, fails on consecutive runs. Reverting the one line change and playbook succeeds. | 21:52 |
| nsmeds | o.O | 21:52 |
| *** hamerins has quit IRC | 21:57 | |
| *** hamerins has joined #openstack-ansible | 21:57 | |
| jonher | OK, I couldn't find any reference to "is_admin_project" around that playbook, but maybe someone else will | 21:59 |
| nsmeds | yeah me neither, but I'm just going to review playbook for now and hopefully find something | 22:00 |
| nsmeds | appreciate it <3 | 22:00 |
| nsmeds | it makes... no sense | 22:00 |
| *** markvoelker has joined #openstack-ansible | 22:01 | |
| *** hamerins has quit IRC | 22:02 | |
| openstackgerrit | Merged openstack/openstack-ansible stable/rocky: Move ARA install to end of bootstrap https://review.openstack.org/617785 | 22:04 |
| *** strattao has joined #openstack-ansible | 22:11 | |
| *** strattao has quit IRC | 22:12 | |
| *** rjgibson has joined #openstack-ansible | 22:22 | |
| *** klamath has quit IRC | 22:29 | |
| *** klamath has joined #openstack-ansible | 22:29 | |
| *** rjgibson has left #openstack-ansible | 22:30 | |
| *** cshen has quit IRC | 22:45 | |
| *** cshen has joined #openstack-ansible | 22:47 | |
| *** cshen has quit IRC | 22:52 | |
| *** weezS_ has joined #openstack-ansible | 22:52 | |
| *** weezS has quit IRC | 22:54 | |
| *** weezS_ is now known as weezS | 22:54 | |
| nsmeds | jonher: found the underlying role and disabled `no_log`, got a more detailed error https://gist.github.com/nikosmeds/6bc29b00c05222f94fa62a3adfffff5a | 22:58 |
| *** SimAloo has quit IRC | 22:59 | |
| *** cshen has joined #openstack-ansible | 23:04 | |
| openstackgerrit | Merged openstack/openstack-ansible-ops master: Update elk_6x for 6.5.x https://review.openstack.org/619448 | 23:06 |
| *** jamesdenton has quit IRC | 23:23 | |
| *** dhellmann_ has joined #openstack-ansible | 23:26 | |
| *** dhellmann has quit IRC | 23:26 | |
| *** aedc has quit IRC | 23:28 | |
| *** dhellmann_ is now known as dhellmann | 23:30 | |
| *** jamesdenton has joined #openstack-ansible | 23:32 | |
| *** aedc has joined #openstack-ansible | 23:36 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!