| Pavo | ok how do I do this Instruct every docker daemon to trust that certificate. | 00:01 |
|---|---|---|
| Pavo | This is done by copying the domain.crt file to /etc/docker/certs.d/myregistrydomain.com:5000/ca.crt | 00:01 |
| Pavo | is there is no /etc/docker/certs.d directory | 00:01 |
| Pavo | do I make it? | 00:01 |
| rstarmer | yes, you have to make the domain:4000/ directory as well, so you can just do make -p /etc/docker/certs.d/mydomain.com:4000/ | 00:02 |
| rstarmer | BTW, I've not verified that, but that's how I read the insructions | 00:02 |
| Pavo | ok | 00:02 |
| *** zhurong has quit IRC | 00:03 | |
| Pavo | ok got a new message when I try to curl | 00:07 |
| Pavo | https://paste.pound-python.org/show/V2BSibEUBrRnMazunzIy/ | 00:07 |
| Pavo | oh nvm I forgot to update-ca-trust | 00:08 |
| Pavo | now I get | 00:08 |
| Pavo | curl https://deploy.local:4000/v2/_catalog | 00:08 |
| Pavo | {"repositories":[]} | 00:08 |
| Pavo | take that its working | 00:08 |
| *** jtriley has joined #openstack-kolla | 00:21 | |
| *** tfukushima has joined #openstack-kolla | 00:24 | |
| rstarmer | yes! | 00:26 |
| kbaegis | well done | 00:27 |
| kbaegis | :) | 00:27 |
| Pavo | so... took down notes on what I did this time lol | 00:28 |
| *** tfukushima has quit IRC | 00:29 | |
| Pavo | created /etc/docker/certs.d/deploy.local:4000 and copied /certs/domain.crt there | 00:29 |
| Pavo | then did a cp certs/domain.crt /etc/pki/ca-trust/source/anchors/deploy.local.crt && update-ca-trust | 00:30 |
| Pavo | and then did docker run -d -p 192.168.1.50:4000:5000 --restart=always --name registry -v /certs:/certs -e REGISTRY_HTTP_TLS_CERTIFICATE=/certs/domain.crt -e REGISTRY_HTTP_TLS_KEY=/certs/domain.key registry:2 | 00:30 |
| Pavo | and its working perfectly it seems | 00:30 |
| Pavo | doing the kolla-build now, so far so good | 00:31 |
| rstarmer | =] | 00:31 |
| Pavo | hope kolla deploy works as good *crosses fingers* | 00:31 |
| kbaegis | Good luck! :) | 00:32 |
| Pavo | so if I add all these to my local private registry and then if I decide to save that registry to use on a completely offline system I could right? | 00:32 |
| kbaegis | Yes | 00:33 |
| kbaegis | You can rely on the tag you did kolla-build with | 00:33 |
| Pavo | cool | 00:33 |
| Pavo | thats my goal | 00:33 |
| kbaegis | Or you could simply isolate your lab, but not the registry | 00:33 |
| kbaegis | (preferable) | 00:33 |
| Pavo | yeah need to be able to use this on a complete offline system | 00:34 |
| rstarmer | where does the registry store it's state? seem slike there's a path to the acutal image store, but the mapping? | 00:34 |
| rstarmer | I guess you'd make a state dump of your currently running registry with that info? | 00:34 |
| Pavo | well after I get testing down I would add a -v flag to have the registry saved to a filesystem directory and then just docker save > registry.tar | 00:35 |
| rstarmer | so on my end, neutron_linuxbridge_agent container is failing. I think this is because of an assumption made somewhere as to the type of underlying networks available, but I can't look at the config in the container, because it won't start. any idea how to debug this? | 00:36 |
| rstarmer | cool on the registry, seems like it'd be a nice way to "package" a known good OpenStack :D | 00:36 |
| Pavo | never used linuxbridges with neutron before, only used openvswitch | 00:36 |
| Pavo | my main goal is to use on a completely offline system that allows for ingesting traffic to be monitored in openstack instances | 00:38 |
| rstarmer | I have an issue with OVS having been defined as the "default" virtual switch, so I _always_ use linuxbridge. | 00:38 |
| rstarmer | yeah, OpenStack as appliance model. makes total sense. | 00:38 |
| Pavo | well I would love to rather use a Cisco N1000V switch instead of openvswitch lol | 00:39 |
| Pavo | but thats a nightmare in itself | 00:40 |
| rstarmer | Yeah, with the overall complexity of this enviornment, the last thing I need is a difficult to debug and understand component in the middle. and path based switching, while novel, doesn't buy me anything in my use cases. | 00:41 |
| *** daneyon_ has joined #openstack-kolla | 00:41 | |
| Pavo | exactly, feel the same | 00:42 |
| Pavo | its enough of a headache just debugging openstack itself | 00:42 |
| *** duonghq has joined #openstack-kolla | 00:44 | |
| *** daneyon_ has quit IRC | 00:46 | |
| rstarmer | =] | 00:48 |
| rstarmer | ok, so if you do need to get "into" the deployment config. 'docker inspect {container_name_or_id}' will give yo info about where it's actually getting it's config from (e.g. where the mappings are) | 00:49 |
| Pavo | ah nice | 00:50 |
| *** kbaegis has quit IRC | 00:56 | |
| *** kbaegis1 has joined #openstack-kolla | 00:56 | |
| *** kbaegis1 has quit IRC | 01:17 | |
| *** jtriley has quit IRC | 01:19 | |
| *** jtriley has joined #openstack-kolla | 01:19 | |
| *** kbaegis has joined #openstack-kolla | 01:23 | |
| Pavo | ok now having issues with the target nodes | 01:25 |
| Pavo | with the cert | 01:25 |
| *** tfukushima has joined #openstack-kolla | 01:25 | |
| Pavo | do I need to copy the certs over to them also? | 01:26 |
| kbaegis | Yes | 01:26 |
| kbaegis | If they need to push to the repo | 01:26 |
| Pavo | in the same places? | 01:26 |
| Pavo | they need to pull from the repo I don't know about push | 01:27 |
| Pavo | https://paste.pound-python.org/show/htLrTWYT9q0li9ZDEnYC/ | 01:27 |
| Pavo | thats the error I am getting for kolloa deploy | 01:28 |
| Pavo | kolla | 01:28 |
| *** jtriley has quit IRC | 01:28 | |
| *** jtriley has joined #openstack-kolla | 01:29 | |
| *** tfukushima has quit IRC | 01:30 | |
| Pavo | hmmm I don't get it, the certs are there and update-ca-trust and kolla deploy still erroring out with the same error | 01:35 |
| Pavo | and I can curl from both nodes | 01:35 |
| kbaegis | expired or not yet valid? | 01:35 |
| Pavo | https://paste.pound-python.org/show/w0c8Gbjqc0nqmguNtbOp/ | 01:36 |
| kbaegis | Could either be ntp or the cert-update function | 01:36 |
| Pavo | yeah I don't get that | 01:36 |
| kbaegis | CA's are funny. Because of the attack methods, certs are only good for a certain period | 01:36 |
| kbaegis | So if you certified it tomorrow, it won't be usable until then | 01:36 |
| Pavo | hmm | 01:37 |
| Pavo | maybe ntp issue? | 01:37 |
| kbaegis | very possibly | 01:37 |
| Pavo | thinking so, deployer Sun Aug 14 21:37:37 EDT 2016 | 01:37 |
| Pavo | target nodes Sun Aug 14 18:51:16 EDT 2016 | 01:38 |
| *** tfukushima has joined #openstack-kolla | 01:39 | |
| Pavo | ok updated ntp trying again | 01:39 |
| Pavo | jesus lol now getting x509: certificate signed by unknown authority | 01:40 |
| Pavo | hmm weird errored on node1 but worked fine on node2 | 01:41 |
| *** jtriley has quit IRC | 01:41 | |
| *** jtriley has joined #openstack-kolla | 01:42 | |
| *** huikang has joined #openstack-kolla | 01:43 | |
| Pavo | woot finally got it working on both nodes now | 01:44 |
| Pavo | had to cp the /etc/docker/certs.d/deploy.local:4000/ca.crt to both nodes lol | 01:45 |
| openstackgerrit | Tin Lam proposed openstack/kolla: Add support for building vmtp container https://review.openstack.org/354362 | 01:46 |
| Pavo | uh oh getting TASK [mariadb : Waiting for MariaDB service to be ready] *********************** | 01:47 |
| Pavo | FAILED - RETRYING: TASK: mariadb : Waiting for MariaDB service to be ready (10 retries left). | 01:47 |
| Pavo | FAILED - RETRYING: TASK: mariadb : Waiting for MariaDB service to be ready (10 retries left). | 01:47 |
| Pavo | FAILED - RETRYING: TASK: mariadb : Waiting for MariaDB service to be ready (9 retries left). | 01:47 |
| Pavo | FAILED - RETRYING: TASK: mariadb : Waiting for MariaDB service to be ready (9 retries left). | 01:47 |
| Pavo | but when I do a docker ps on node1 I see | 01:48 |
| Pavo | 4d81194ace0c deploy.local:4000/kolla/centos-binary-mariadb:3.0.0 "kolla_start" 2 minutes ago Up 16 seconds mariadb | 01:48 |
| *** jtriley has quit IRC | 01:55 | |
| *** jtriley has joined #openstack-kolla | 01:56 | |
| kbaegis | docker logs | 01:57 |
| kbaegis | It will probably tell you that it can't bind to the address/port combination specified | 01:57 |
| kbaegis | You'll need to use a different vip or manage the port | 01:58 |
| kbaegis | "Error: Could not find default role "_member_" in Keystone" | 01:58 |
| kbaegis | Anyone seen this gem? | 01:58 |
| kbaegis | "Error: Could not find default role "_member_" in Keystone" after a deployment? | 01:58 |
| Pavo | ok I'll take a look | 01:58 |
| Pavo | I'm not seeing anything | 02:01 |
| Pavo | https://paste.pound-python.org/show/NeQKMfk5fFzIlWvVkmB2/ | 02:01 |
| *** jtriley has quit IRC | 02:04 | |
| *** jtriley has joined #openstack-kolla | 02:05 | |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: Fix the Ubuntu Xenial build gate https://review.openstack.org/355274 | 02:07 |
| *** jtriley has quit IRC | 02:14 | |
| *** jtriley has joined #openstack-kolla | 02:14 | |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: Pin to the Ubuntu 16.04 https://review.openstack.org/329454 | 02:18 |
| openstackgerrit | Lu lei proposed openstack/kolla: Fix Kolla home-page URL https://review.openstack.org/355275 | 02:22 |
| *** britthouser has quit IRC | 02:23 | |
| rstarmer | the fact that _member_ isn't there implies that something didn't make it through the configuration. I actually recall that being a database migration default, so perhaps the db scripts didn't run properly? | 02:25 |
| *** daneyon_ has joined #openstack-kolla | 02:30 | |
| *** jtriley has quit IRC | 02:32 | |
| *** jtriley has joined #openstack-kolla | 02:33 | |
| *** klint has joined #openstack-kolla | 02:33 | |
| *** daneyon_ has quit IRC | 02:34 | |
| kbaegis | shouldn't take 3 minutes to spawn an instance under kolla should it? | 02:36 |
| *** jtriley has quit IRC | 02:50 | |
| *** jtriley has joined #openstack-kolla | 02:51 | |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: Pin to the Ubuntu 16.04 https://review.openstack.org/329454 | 02:56 |
| *** jtriley has quit IRC | 02:59 | |
| *** zhurong has joined #openstack-kolla | 02:59 | |
| *** jtriley has joined #openstack-kolla | 03:00 | |
| duonghq | kbaegis: it's depend on your hardware, but 3 min is quite long | 03:02 |
| *** huikang has quit IRC | 03:02 | |
| duonghq | have you tried install OpenStack directly | 03:03 |
| *** jtriley has quit IRC | 03:09 | |
| *** jtriley has joined #openstack-kolla | 03:09 | |
| *** huikang has joined #openstack-kolla | 03:15 | |
| *** jtriley has quit IRC | 03:18 | |
| *** jtriley has joined #openstack-kolla | 03:19 | |
| *** jtriley has quit IRC | 03:32 | |
| openstackgerrit | zhubingbing proposed openstack/kolla: Add gnocchi ansible role https://review.openstack.org/349351 | 03:37 |
| *** zhubingbing has joined #openstack-kolla | 03:41 | |
| *** tfukushima has quit IRC | 03:53 | |
| *** jtriley has joined #openstack-kolla | 04:28 | |
| *** jtriley has quit IRC | 04:37 | |
| *** zhubingbing has quit IRC | 04:39 | |
| *** huikang has quit IRC | 04:40 | |
| *** kbaegis1 has joined #openstack-kolla | 04:41 | |
| *** kbaegis has quit IRC | 04:41 | |
| *** zhurong has quit IRC | 04:43 | |
| *** zhurong has joined #openstack-kolla | 04:44 | |
| *** dmueller-evoila has joined #openstack-kolla | 04:50 | |
| *** dmueller-evoila has quit IRC | 04:55 | |
| *** bjolo has joined #openstack-kolla | 05:09 | |
| bjolo | morning :) | 05:10 |
| *** daneyon_ has joined #openstack-kolla | 05:12 | |
| *** senk_ has joined #openstack-kolla | 05:16 | |
| *** daneyon_ has quit IRC | 05:16 | |
| *** rbergeron has quit IRC | 05:24 | |
| *** rbergeron has joined #openstack-kolla | 05:24 | |
| *** dmsimard has quit IRC | 05:25 | |
| *** dmsimard has joined #openstack-kolla | 05:26 | |
| *** kbaegis has joined #openstack-kolla | 05:31 | |
| *** kbaegis1 has quit IRC | 05:31 | |
| *** kbaegis1 has joined #openstack-kolla | 05:34 | |
| *** kbaegis has quit IRC | 05:34 | |
| *** zhubingbing has joined #openstack-kolla | 05:35 | |
| *** kbaegis1 has quit IRC | 05:36 | |
| *** kbaegis has joined #openstack-kolla | 05:36 | |
| *** jtriley has joined #openstack-kolla | 05:38 | |
| *** kbaegis1 has joined #openstack-kolla | 05:39 | |
| *** kbaegis has quit IRC | 05:39 | |
| openstackgerrit | Ross Krumbeck proposed openstack/kolla: Ansible-ize OpenStack Designate https://review.openstack.org/353261 | 05:41 |
| *** kbaegis has joined #openstack-kolla | 05:42 | |
| *** kbaegis1 has quit IRC | 05:42 | |
| *** kbaegis1 has joined #openstack-kolla | 05:45 | |
| *** kbaegis has quit IRC | 05:45 | |
| *** jtriley has quit IRC | 05:47 | |
| *** rkrum has joined #openstack-kolla | 05:47 | |
| *** jtriley has joined #openstack-kolla | 05:48 | |
| *** kbaegis has joined #openstack-kolla | 05:48 | |
| *** kbaegis1 has quit IRC | 05:48 | |
| *** kbaegis1 has joined #openstack-kolla | 05:51 | |
| *** kbaegis has quit IRC | 05:51 | |
| *** kbaegis has joined #openstack-kolla | 05:54 | |
| *** kbaegis1 has quit IRC | 05:54 | |
| *** kbaegis1 has joined #openstack-kolla | 05:56 | |
| *** kbaegis has quit IRC | 05:56 | |
| *** kbaegis has joined #openstack-kolla | 06:00 | |
| *** hieulq has quit IRC | 06:00 | |
| *** duonghq_ has joined #openstack-kolla | 06:00 | |
| *** jtriley has quit IRC | 06:01 | |
| *** kbaegis1 has quit IRC | 06:01 | |
| *** duonghq has quit IRC | 06:01 | |
| *** jtriley has joined #openstack-kolla | 06:01 | |
| *** kbaegis has quit IRC | 06:03 | |
| *** kbaegis has joined #openstack-kolla | 06:03 | |
| *** kbaegis1 has joined #openstack-kolla | 06:06 | |
| *** hieulq has joined #openstack-kolla | 06:07 | |
| *** kbaegis has quit IRC | 06:07 | |
| *** kbaegis has joined #openstack-kolla | 06:10 | |
| *** jtriley has quit IRC | 06:10 | |
| *** kbaegis1 has quit IRC | 06:10 | |
| *** jtriley has joined #openstack-kolla | 06:11 | |
| *** kbaegis1 has joined #openstack-kolla | 06:13 | |
| *** kbaegis has quit IRC | 06:14 | |
| *** zhurong has quit IRC | 06:15 | |
| *** kbaegis has joined #openstack-kolla | 06:16 | |
| *** kbaegis1 has quit IRC | 06:16 | |
| *** zhurong has joined #openstack-kolla | 06:16 | |
| *** zhubingbing_ has joined #openstack-kolla | 06:18 | |
| *** duonghq_ is now known as duonghq | 06:18 | |
| *** jtriley has quit IRC | 06:19 | |
| *** kbaegis1 has joined #openstack-kolla | 06:19 | |
| *** jtriley has joined #openstack-kolla | 06:20 | |
| *** kbaegis has quit IRC | 06:20 | |
| *** zhubingbing has quit IRC | 06:20 | |
| *** kbaegis has joined #openstack-kolla | 06:22 | |
| *** kbaegis1 has quit IRC | 06:23 | |
| *** kbaegis has quit IRC | 06:25 | |
| *** kbaegis1 has joined #openstack-kolla | 06:25 | |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: Pin to the Ubuntu 16.04 https://review.openstack.org/329454 | 06:26 |
| *** kbaegis1 has quit IRC | 06:28 | |
| *** kbaegis has joined #openstack-kolla | 06:28 | |
| *** kbaegis1 has joined #openstack-kolla | 06:30 | |
| *** kbaegis has quit IRC | 06:30 | |
| *** kbaegis has joined #openstack-kolla | 06:33 | |
| *** kbaegis1 has quit IRC | 06:34 | |
| *** senk_ has quit IRC | 06:35 | |
| *** david-lyle has joined #openstack-kolla | 06:36 | |
| *** kbaegis1 has joined #openstack-kolla | 06:37 | |
| *** kbaegis has quit IRC | 06:37 | |
| *** david-lyle_ has quit IRC | 06:39 | |
| *** kbaegis has joined #openstack-kolla | 06:39 | |
| *** kbaegis1 has quit IRC | 06:39 | |
| *** jtriley has quit IRC | 06:42 | |
| *** jtriley has joined #openstack-kolla | 06:43 | |
| *** belmoreira has joined #openstack-kolla | 06:48 | |
| *** phuongnh has joined #openstack-kolla | 06:49 | |
| *** jtriley has quit IRC | 06:51 | |
| *** daneyon_ has joined #openstack-kolla | 07:00 | |
| *** daneyon_ has quit IRC | 07:05 | |
| *** rkrum has quit IRC | 07:07 | |
| *** bootsha has joined #openstack-kolla | 07:11 | |
| *** senk_ has joined #openstack-kolla | 07:11 | |
| rstarmer | so how does one debug the startup of a service in a container? the neutron-linuxbridge-agent container fails right as it tries to run the agent: | 07:11 |
| rstarmer | Running command: 'neutron-linuxbridge-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini' | 07:11 |
| rstarmer | Guru mediation now registers SIGUSR1 and SIGUSR2 by default for backward compatibility. SIGUSR1 will no longer be registered in a future release, so please use SIGUSR2 to generate reports. | 07:11 |
| *** hanchao has joined #openstack-kolla | 07:15 | |
| openstackgerrit | Lu lei proposed openstack/kolla: Fix oslo.i18n in kolla project https://review.openstack.org/355342 | 07:15 |
| *** matrohon has joined #openstack-kolla | 07:28 | |
| *** kbaegis1 has joined #openstack-kolla | 07:37 | |
| *** kbaegis has quit IRC | 07:37 | |
| *** kbaegis1 has quit IRC | 07:40 | |
| *** kbaegis has joined #openstack-kolla | 07:40 | |
| *** kbaegis1 has joined #openstack-kolla | 07:42 | |
| *** kbaegis has quit IRC | 07:42 | |
| *** kbaegis has joined #openstack-kolla | 07:44 | |
| *** kbaegis1 has quit IRC | 07:44 | |
| *** kbaegis1 has joined #openstack-kolla | 07:47 | |
| *** kbaegis has quit IRC | 07:47 | |
| *** kbaegis has joined #openstack-kolla | 07:49 | |
| *** kbaegis1 has quit IRC | 07:49 | |
| *** kbaegis has quit IRC | 07:52 | |
| *** kbaegis1 has joined #openstack-kolla | 07:52 | |
| *** jtriley has joined #openstack-kolla | 07:53 | |
| *** kbaegis has joined #openstack-kolla | 07:55 | |
| bjolo | rstarmer, docker logs? | 07:56 |
| rstarmer | that is the log | 07:56 |
| rstarmer | command runs, and... nothing, contianer restarts | 07:56 |
| *** kbaegis1 has quit IRC | 07:56 | |
| rstarmer | I have a working version based on 3.0.0 tagged images, that starts, and looks identical. I just don't see why the failing container does | 07:57 |
| *** jtriley has quit IRC | 07:57 | |
| *** kbaegis1 has joined #openstack-kolla | 07:57 | |
| *** kbaegis has quit IRC | 07:58 | |
| openstackgerrit | Lu lei proposed openstack/kolla: Fix oslo.i18n in kolla project https://review.openstack.org/355342 | 08:00 |
| *** kbaegis1 has quit IRC | 08:01 | |
| *** kbaegis has joined #openstack-kolla | 08:01 | |
| *** sdake has joined #openstack-kolla | 08:01 | |
| *** kbaegis1 has joined #openstack-kolla | 08:03 | |
| *** kbaegis has quit IRC | 08:03 | |
| sdake | evening folks | 08:04 |
| rstarmer | sdake: evening | 08:06 |
| *** kbaegis has joined #openstack-kolla | 08:06 | |
| rstarmer | or morning, depending on your preference | 08:06 |
| *** kbaegis1 has quit IRC | 08:06 | |
| bjolo | start the container manually with the -D flag_ | 08:06 |
| bjolo | ? | 08:06 |
| bjolo | sdake, evening | 08:06 |
| rstarmer | bjolo: will try, don't thing I've done that. | 08:07 |
| *** kbaegis1 has joined #openstack-kolla | 08:08 | |
| *** kbaegis has quit IRC | 08:09 | |
| wznoinsk | Pavo: is your mariadb listening on the same IP as you're trying to connect to? (vip/haproxy(keepalived)) ? | 08:09 |
| *** kbaegis1 has quit IRC | 08:11 | |
| *** kbaegis has joined #openstack-kolla | 08:11 | |
| *** athomas has joined #openstack-kolla | 08:13 | |
| sdake | well off to bed | 08:14 |
| sdake | been workign on on tonearm setup all weekend | 08:14 |
| sdake | about wrapped up with "basic" setup | 08:14 |
| *** kbaegis1 has joined #openstack-kolla | 08:14 | |
| *** kbaegis has quit IRC | 08:14 | |
| sdake | 3 12 hr days of stand sit stand sit = out of steam | 08:15 |
| *** sdake has quit IRC | 08:15 | |
| *** kbaegis has joined #openstack-kolla | 08:16 | |
| *** kbaegis1 has quit IRC | 08:17 | |
| *** kbaegis has quit IRC | 08:18 | |
| *** kbaegis has joined #openstack-kolla | 08:19 | |
| *** prithiv has joined #openstack-kolla | 08:20 | |
| *** kbaegis1 has joined #openstack-kolla | 08:22 | |
| *** Serlex has joined #openstack-kolla | 08:22 | |
| *** kbaegis has quit IRC | 08:22 | |
| *** strigazi is now known as strigazi_AFK | 08:26 | |
| *** dmueller-evoila has joined #openstack-kolla | 08:27 | |
| *** dmueller-evoila has quit IRC | 08:34 | |
| *** sdake has joined #openstack-kolla | 08:35 | |
| *** bootsha has quit IRC | 08:40 | |
| prithiv | Hi. Can someone look at the BP I proposed. https://blueprints.launchpad.net/kolla/+spec/kolla-collectd-container. Need approval | 08:41 |
| *** sdake has quit IRC | 08:43 | |
| openstackgerrit | Lu lei proposed openstack/kolla: Fix oslo.i18n in kolla project https://review.openstack.org/355342 | 08:46 |
| *** daneyon_ has joined #openstack-kolla | 08:49 | |
| *** dmueller-evoila has joined #openstack-kolla | 08:50 | |
| openstackgerrit | Paul Bourke proposed openstack/kolla: Fix bandit gate jobs https://review.openstack.org/353942 | 08:52 |
| *** jtriley has joined #openstack-kolla | 08:53 | |
| *** daneyon_ has quit IRC | 08:53 | |
| openstackgerrit | Wei Cao proposed openstack/kolla: Remove duplicate items https://review.openstack.org/355377 | 08:55 |
| *** The_Ball has joined #openstack-kolla | 08:58 | |
| *** jtriley has quit IRC | 08:59 | |
| The_Ball | What process installs the ansible playbooks into /usr/share/kolla/ansible? And how can I re-run it to put the latest once in place after a "git checkout 2.0.2" for example? | 08:59 |
| *** dmueller-evoila has quit IRC | 09:00 | |
| *** bootsha has joined #openstack-kolla | 09:02 | |
| pbourke | The_Ball: pip install . | 09:03 |
| *** dmueller-evoila has joined #openstack-kolla | 09:04 | |
| The_Ball | ah, I see, thank you | 09:05 |
| *** prithiv has quit IRC | 09:05 | |
| openstackgerrit | Paul Bourke proposed openstack/kolla: Document the Dockerfile customisation mechanism https://review.openstack.org/351123 | 09:11 |
| *** dmueller-evoila has quit IRC | 09:11 | |
| *** bootsha has quit IRC | 09:14 | |
| *** bootsha has joined #openstack-kolla | 09:14 | |
| *** bootsha_ has joined #openstack-kolla | 09:16 | |
| *** bootsha__ has joined #openstack-kolla | 09:18 | |
| *** bootsha has quit IRC | 09:20 | |
| *** papacz has joined #openstack-kolla | 09:21 | |
| *** bootsha_ has quit IRC | 09:22 | |
| openstackgerrit | Prithiv proposed openstack/kolla: Add networking guide for Neutron extensions https://review.openstack.org/355386 | 09:23 |
| openstackgerrit | Lu lei proposed openstack/kolla: Fix oslo.i18n in kolla project https://review.openstack.org/355342 | 09:28 |
| openstackgerrit | Paul Bourke proposed openstack/kolla: Handle empty package list for install_packages https://review.openstack.org/354429 | 09:29 |
| *** kbaegis has joined #openstack-kolla | 09:30 | |
| openstackgerrit | Prithiv proposed openstack/kolla: Add networking guide for Neutron extensions https://review.openstack.org/355386 | 09:30 |
| *** prithiv has joined #openstack-kolla | 09:31 | |
| *** kbaegis1 has quit IRC | 09:31 | |
| *** kbaegis has quit IRC | 09:33 | |
| *** kbaegis has joined #openstack-kolla | 09:33 | |
| *** kbaegis1 has joined #openstack-kolla | 09:35 | |
| *** kbaegis has quit IRC | 09:35 | |
| *** kbaegis has joined #openstack-kolla | 09:38 | |
| *** kbaegis1 has quit IRC | 09:38 | |
| *** kbaegis has quit IRC | 09:40 | |
| *** kbaegis has joined #openstack-kolla | 09:40 | |
| *** kbaegis1 has joined #openstack-kolla | 09:43 | |
| *** kbaegis has quit IRC | 09:45 | |
| *** sdake has joined #openstack-kolla | 09:46 | |
| *** kbaegis1 has quit IRC | 09:47 | |
| *** kbaegis has joined #openstack-kolla | 09:47 | |
| *** kbaegis1 has joined #openstack-kolla | 09:49 | |
| *** hanchao has quit IRC | 09:49 | |
| *** kbaegis has quit IRC | 09:49 | |
| *** kbaegis has joined #openstack-kolla | 09:52 | |
| *** kbaegis1 has quit IRC | 09:52 | |
| *** jtriley has joined #openstack-kolla | 09:54 | |
| *** kbaegis1 has joined #openstack-kolla | 09:55 | |
| *** kbaegis has quit IRC | 09:55 | |
| openstackgerrit | Prithiv proposed openstack/kolla: Add networking guide for Neutron extensions https://review.openstack.org/355386 | 09:56 |
| *** athomas has quit IRC | 09:57 | |
| *** kbaegis1 has quit IRC | 09:58 | |
| *** kbaegis has joined #openstack-kolla | 09:58 | |
| *** jtriley has quit IRC | 09:59 | |
| *** kbaegis1 has joined #openstack-kolla | 10:00 | |
| *** kbaegis has quit IRC | 10:01 | |
| *** athomas has joined #openstack-kolla | 10:02 | |
| *** kbaegis has joined #openstack-kolla | 10:03 | |
| *** kbaegis1 has quit IRC | 10:03 | |
| The_Ball | should I change enable_glance for example in ansible/group_vars/all.yml or is it possible to override this in /etc/kolla/globals.yml? | 10:04 |
| *** kbaegis1 has joined #openstack-kolla | 10:05 | |
| *** kbaegis has quit IRC | 10:06 | |
| duonghq | You should change in /etc/.... due to kolla-binary read config from that | 10:06 |
| *** kbaegis has joined #openstack-kolla | 10:08 | |
| *** kbaegis1 has quit IRC | 10:08 | |
| *** kbaegis1 has joined #openstack-kolla | 10:10 | |
| *** kbaegis has quit IRC | 10:11 | |
| The_Ball | duonghq, that makes sense, I must have been doing something else wrong | 10:11 |
| *** prithiv has quit IRC | 10:11 | |
| *** kbaegis has joined #openstack-kolla | 10:13 | |
| *** kbaegis1 has quit IRC | 10:13 | |
| *** kbaegis1 has joined #openstack-kolla | 10:15 | |
| *** kbaegis has quit IRC | 10:16 | |
| *** prithiv has joined #openstack-kolla | 10:16 | |
| duonghq | anybody got "ImportError: No module named fixtures | 10:17 |
| duonghq | " in boostrap_neutron? | 10:17 |
| duonghq | sdake: are you there? | 10:18 |
| *** kbaegis1 has quit IRC | 10:18 | |
| *** kbaegis has joined #openstack-kolla | 10:18 | |
| *** kbaegis has quit IRC | 10:21 | |
| *** kbaegis1 has joined #openstack-kolla | 10:21 | |
| kbaegis1 | Morning sdake :) | 10:24 |
| kbaegis1 | Kolla confirmed on Gentoo | 10:24 |
| kbaegis1 | With some caveats/workarounds | 10:24 |
| duonghq | kbaegis1: everything is up? | 10:25 |
| kbaegis1 | It was. 2.0.1. Had to work around a couple of things | 10:26 |
| kbaegis1 | 3.0.0/centos-source doesn't work yet though | 10:26 |
| Serlex | Not a kolla question, but anyone know where I can download HVM images for Xen computenodes? | 10:27 |
| kbaegis1 | 2.0.1 works with 1) linuxbridging only (ovs doesn't seem to), and HAProxy/keepalived disabled or the vip manually assigned | 10:29 |
| kbaegis1 | Does the host have to have ovs installed? I know I enabled support for it in the kernel | 10:30 |
| kbaegis1 | On 3.0.0 I was able to deploy up until the neutron database bootstrap container | 10:31 |
| duonghq | kbaegis1: which error did you get? | 10:31 |
| kbaegis1 | There may be an issue with that in master | 10:31 |
| kbaegis1 | I'm reproducing now | 10:31 |
| duonghq | I have trouble with this, too | 10:32 |
| kbaegis1 | I'll paste the docker logs soon | 10:32 |
| duonghq | does it like: ImportError: No module named fixtures | 10:32 |
| openstackgerrit | Wei Cao proposed openstack/kolla: Add solum container https://review.openstack.org/355408 | 10:32 |
| duonghq | I look around source code but still cannot find anything unusual | 10:33 |
| kbaegis1 | That sounds familiar | 10:33 |
| kbaegis1 | I know I've seen that. I just remember it was neutron-db-bootstram | 10:34 |
| kbaegis1 | *p | 10:34 |
| duonghq | the bootstrap_neutron? | 10:34 |
| *** prithiv has quit IRC | 10:35 | |
| *** prithiv has joined #openstack-kolla | 10:35 | |
| kbaegis1 | I'm on nova already. I'll give you an actual paste in a minute :) | 10:36 |
| duonghq | sound good | 10:36 |
| kbaegis1 | http://hastebin.com/gikoguquje.vhdl | 10:37 |
| kbaegis1 | fixtures, yes | 10:37 |
| kbaegis1 | Last opportunities for regressions (by comments) were July 15th and Aug 12th | 10:39 |
| kbaegis1 | Ken Wronkiewicz <wh-openstack@wirewd.com> and Jenkins <jenkins@review.openstack.org> (who knows), respectively | 10:40 |
| *** kbaegis1 has quit IRC | 10:40 | |
| duonghq | The Jenkins is a bot, he merge W+1 commit to master branch | 10:40 |
| *** kbaegis has joined #openstack-kolla | 10:41 | |
| kbaegis | Weird, got dropped | 10:41 |
| kbaegis | Rolled back to de7e8bc00b8f430d37df86857877b94f96c23fe3 to check | 10:41 |
| duonghq | and Ken Wronkiewicz is wirehead_ ? afaik | 10:42 |
| sdake | you have trouble with which duonghq ? | 10:42 |
| kbaegis | neutron-server | 10:42 |
| rstarmer | yes, bootstrap_neutron and a complaint about fixtures | 10:42 |
| kbaegis | rolled back | 10:43 |
| kbaegis | testing now | 10:43 |
| sdake | centos binary will sometimes do thigns like that | 10:43 |
| sdake | are you using centos binary? | 10:43 |
| sdake | centos binary has trouble tracking their dependencies in rpm packaging | 10:43 |
| kbaegis | I guess I need to rebuild the image too :) | 10:43 |
| rstarmer | I thought it was centos/source for me | 10:43 |
| sdake | holy shit my tonearm finally works | 10:43 |
| sdake | after 20+ hours of jerking around | 10:43 |
| duonghq | sdake: I use Ubuntu/source | 10:43 |
| sdake | off by half a MM | 10:44 |
| kbaegis | centos source | 10:44 |
| *** hieulq has quit IRC | 10:44 | |
| kbaegis | two stderrs already: base:Ignoring dnspython3: markers u"python_version=='3.4'" don't match your environment INFO:kolla.image.build.neutron-base:Ignoring ovs: markers u"python_version=='3.4'" don't match your environment | 10:45 |
| sdake | please dont skip please dont skip please dont skip :) | 10:45 |
| sdake | ignore those stderrs kbaegis | 10:46 |
| sdake | those are nonsense errors | 10:46 |
| sdake | we should probabll yfilter them in the build tool | 10:46 |
| kbaegis | Okay :) | 10:46 |
| kbaegis | So I just rebuild nova-server/nova-base | 10:46 |
| kbaegis | *neutron-server/neutron-base | 10:46 |
| *** prithiv has quit IRC | 10:46 | |
| kbaegis | trying redeploy from de7e8bc00b8f430d37df86857877b94f96c23fe3 | 10:47 |
| *** mbound has joined #openstack-kolla | 10:47 | |
| *** duonghq has left #openstack-kolla | 10:50 | |
| kbaegis | I do a lot of developery things for a non-developer these days :) | 10:50 |
| *** mbound has quit IRC | 10:51 | |
| rstarmer | that's because you live in the world of Ops/Dev, just enough Development to get things Operational | 10:52 |
| *** jtriley has joined #openstack-kolla | 10:55 | |
| kbaegis | same issue with that commit | 10:55 |
| kbaegis | rolling back | 10:55 |
| *** rstarmer has quit IRC | 10:56 | |
| The_Ball | I'm hitting an error: ERROR! Unexpected Exception: No module named runner.return_data but I'm using the latest ansible (pip install -U ansible) | 10:56 |
| kbaegis | Oh hi, The_Ball. Think we were learning things on #docker together :) | 10:57 |
| The_Ball | Hehe, yeah | 10:57 |
| *** phuongnh has quit IRC | 10:58 | |
| Daviey | hey gang. | 11:00 |
| kbaegis | We're a gang? | 11:00 |
| kbaegis | Nice! I'll get my leather jacket ;) | 11:00 |
| *** jtriley has quit IRC | 11:00 | |
| Daviey | indeed | 11:00 |
| *** vbel has quit IRC | 11:00 | |
| *** vbel has joined #openstack-kolla | 11:01 | |
| kbaegis | b9e6848c5366046ee7cbd3e2ae683ecee30cc721 fails to build neutron-server from centos-source | 11:01 |
| kbaegis | Okay, I want to try something | 11:02 |
| kbaegis | How can I isolate neutron for rollback? ie- everything else on 3.0.0devxxx, only neutron components on 2.0.1? | 11:03 |
| kbaegis | Is that possible? | 11:03 |
| *** zhurong has quit IRC | 11:05 | |
| kbaegis | Anyone? OpenStack itself has great latitude with the API versions. It shouldn't care | 11:06 |
| kbaegis | Does kolla add additional requirements? | 11:06 |
| kbaegis | Well there's a wrinkle | 11:10 |
| kbaegis | I'll play around with it. See if I can fix the image | 11:10 |
| kbaegis | Bet it's just missing a package | 11:11 |
| kbaegis | @The_Ball you may not want the latest version of ansible, depending on the deployed version | 11:14 |
| *** britthouser has joined #openstack-kolla | 11:18 | |
| *** dmueller-evoila has joined #openstack-kolla | 11:19 | |
| *** dmueller-evoila has quit IRC | 11:24 | |
| kbaegis | Hey guys | 11:28 |
| kbaegis | How do I troubleshoot the container? | 11:28 |
| kbaegis | It keeps stopping before exec lets me execute any actual commands | 11:28 |
| *** dmueller-evoila has joined #openstack-kolla | 11:28 | |
| *** duonghq has joined #openstack-kolla | 11:31 | |
| *** daneyon_ has joined #openstack-kolla | 11:31 | |
| duonghq | kbaegis: anything news? | 11:32 |
| duonghq | kbaegis: I think you can replace the original binary with script to do some investigate | 11:34 |
| kbaegis | Yeah, how? | 11:34 |
| kbaegis | :) | 11:34 |
| kbaegis | It's on a loop | 11:35 |
| kbaegis | I guess I can try to reverse engineer all the -v and -e switches :/ | 11:35 |
| *** dmueller-evoila has quit IRC | 11:35 | |
| *** daneyon_ has quit IRC | 11:35 | |
| duonghq | kbaegis: I think there are 2 point to probe: neutron_server and neutron_base | 11:37 |
| *** rhallisey has joined #openstack-kolla | 11:38 | |
| *** dmueller-evoila has joined #openstack-kolla | 11:39 | |
| *** rhallisey_ has joined #openstack-kolla | 11:39 | |
| kbaegis | Okay | 11:42 |
| kbaegis | I have to figure out the whole damn command though | 11:42 |
| duonghq | and the whole process flow | 11:42 |
| kbaegis | docker run crucible.lab:4000/zerolabs/centos-source-neutron-server:3.0.0 -e KOLLA_SERVICE_NAME=bootstrap-neutron -e KOLLA_CONFIG_STRATEGY=COPY_ALWAYS -e KOLLA_BOOTSTRAP= -e PATH=/var/lib/kolla/venv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin -e KOLLA_BASE_DISTRO=centos -e KOLLA_INSTALL_TYPE=source -e KOLLA_INSTALL_METATYPE=mixed -e 'PS1=$(tput bold)($(printenv KOLLA_SERVICE_NAME))$(tput sgr0)[$(id -un)@$(hostname -s) $(pwd)]$ ' - | 11:43 |
| sdake | kbaegis the rollback you want is possible | 11:43 |
| sdake | it requires manual hacking | 11:43 |
| kbaegis | sdake how? :) | 11:43 |
| The_Ball | kbaegis, cherry-picking 491822c510ca9010d36aeecbcf7716828088cb1b to 2.0.2 fixed the ansible 2 compatibility issue btw | 11:43 |
| sdake | moment | 11:43 |
| kbaegis | Nice! | 11:44 |
| duonghq | still can't understand why this thing can be broken | 11:44 |
| sdake | kbaegis you have to do this or each one in this file - you basically override it in globals.yml | 11:45 |
| sdake | https://github.com/openstack/kolla/blob/master/ansible/roles/neutron/defaults/main.yml#L16 | 11:45 |
| sdake | take a look and let me know if you if q&a :) | 11:45 |
| kbaegis | That's not too bad :) | 11:45 |
| sdake | change tag to what you want it to be (hardcode in globals.yml) | 11:46 |
| kbaegis | Wait, so could I set those values in globals? | 11:48 |
| kbaegis | Right | 11:48 |
| kbaegis | Okay | 11:48 |
| *** rhallisey has quit IRC | 11:51 | |
| *** mbound has joined #openstack-kolla | 11:51 | |
| *** dave-mccowan has joined #openstack-kolla | 11:53 | |
| *** jtriley has joined #openstack-kolla | 11:55 | |
| kbaegis | done :) | 11:55 |
| kbaegis | Let's try that then | 11:55 |
| *** mbound has quit IRC | 11:56 | |
| *** dwalsh has joined #openstack-kolla | 11:57 | |
| *** prithiv has joined #openstack-kolla | 11:59 | |
| *** jtriley has quit IRC | 12:00 | |
| kbaegis | I didn't see 2.0.1 images for lbaas or ovs-base | 12:01 |
| kbaegis | Let's see how this works | 12:01 |
| *** sdake has quit IRC | 12:02 | |
| *** rhallisey has joined #openstack-kolla | 12:03 | |
| *** prithiv has quit IRC | 12:06 | |
| *** sdake has joined #openstack-kolla | 12:06 | |
| kbaegis | fingers crossed | 12:08 |
| *** JoseMello has joined #openstack-kolla | 12:08 | |
| kbaegis | Thanks sdake! Working! | 12:10 |
| sdake | np | 12:10 |
| *** dmueller-evoila has quit IRC | 12:12 | |
| *** dmueller-evoila has joined #openstack-kolla | 12:19 | |
| wznoinsk | hi all | 12:26 |
| kbaegis | well, it deployed successfully | 12:27 |
| kbaegis | Horizon barfed though | 12:27 |
| kbaegis | Only returning 500 | 12:28 |
| kbaegis | http://hastebin.com/imaqorejej.rb | 12:32 |
| The_Ball | Can I trigger the "upgrade" bootstrap manually? | 12:32 |
| kbaegis | There's the vomit from horizon when someone tries to connect | 12:32 |
| The_Ball | my sql schema is out of date | 12:32 |
| *** rhallisey_ has quit IRC | 12:34 | |
| The_Ball | sorry, kolla-ansible upgrade, that sounds too easy :D | 12:34 |
| kbaegis | Doesn't it? | 12:36 |
| kbaegis | Delayed hello wznoinsk | 12:37 |
| *** sdake has quit IRC | 12:38 | |
| kbaegis | Well, keystone is setup with all of the appropriate endpoints, but 3.0.0 horizon is still broken | 12:45 |
| kbaegis | I'll roll that back to 2.0.1 too | 12:45 |
| The_Ball | my cinder for oVirt is back up after the fs corruption. Learnt a lot about kolla in the process, pretty awesome! | 12:46 |
| *** rkrum has joined #openstack-kolla | 12:47 | |
| kbaegis | Yeah, neat huh? | 12:47 |
| duonghq | hmm, I think tommorrow I'll recheck the neutron error, strange bug | 12:48 |
| kbaegis | Thanks duonghq | 12:49 |
| kbaegis | Horizon 3.0.0 is also broken | 12:49 |
| kbaegis | afaict anyways | 12:49 |
| kbaegis | I'm good at finding bugs. Creating them too :) | 12:50 |
| duonghq | at these days, commit focus mainly on bp/third-party-plugin-support | 12:50 |
| duonghq | It shouldn't screw up something like that | 12:51 |
| duonghq | I'm good at make trouble too | 12:51 |
| kbaegis | redeploy #10 | 12:55 |
| kbaegis | tenth time's the charm ;) | 12:56 |
| kbaegis | The_Ball how did you end up recovering? | 13:01 |
| kbaegis | ooc? | 13:01 |
| *** dmueller-evoila has quit IRC | 13:05 | |
| duonghq | kbaegis: so up downgrade Horizon to 2.x.x? | 13:07 |
| duonghq | *up -> you | 13:07 |
| duonghq | iirc: the version from about 2 weeks ago run smoothly | 13:08 |
| *** klint has quit IRC | 13:09 | |
| *** dmueller-evoila has joined #openstack-kolla | 13:10 | |
| *** rkrum has quit IRC | 13:10 | |
| kbaegis | Yeah, I downgraded | 13:11 |
| *** rkrum has joined #openstack-kolla | 13:11 | |
| duonghq | kbaegis: does it run now? | 13:12 |
| kbaegis | Well, it runs | 13:12 |
| duonghq | nice | 13:12 |
| duonghq | everything? | 13:12 |
| kbaegis | Everything is deployed and running | 13:12 |
| kbaegis | Horizon still isn't working for me though | 13:13 |
| kbaegis | I'm getting 404s | 13:13 |
| kbaegis | 'The requested URL / was not found on this server' | 13:13 |
| duonghq | what does the log say? | 13:13 |
| duonghq | (apache2 log) | 13:13 |
| *** jtriley has joined #openstack-kolla | 13:13 | |
| kbaegis | huh' | 13:14 |
| kbaegis | 'Target WSGI script not found or unable to stat: /var/lib/kolla/venv' | 13:14 |
| duonghq | aha | 13:14 |
| kbaegis | What's up with that? | 13:15 |
| duonghq | Can you try jump in to this container and check by hand | 13:15 |
| kbaegis | That's what I did :) | 13:15 |
| duonghq | this script doesn't exist? | 13:16 |
| *** rkrum has quit IRC | 13:16 | |
| kbaegis | Doesn't appear to, no | 13:16 |
| kbaegis | http://hastebin.com/wulayokivu.js | 13:16 |
| openstackgerrit | Abe Music proposed openstack/kolla: enhancements to install_packages macro https://review.openstack.org/348567 | 13:17 |
| kbaegis | config_files | 13:17 |
| kbaegis | is the only thing in there | 13:17 |
| duonghq | no idea, sorry | 13:18 |
| duonghq | today I just run out of luck, I think I'll take a rest soon. | 13:19 |
| *** daneyon_ has joined #openstack-kolla | 13:20 | |
| *** daneyon_ has quit IRC | 13:24 | |
| wznoinsk | kbaegis: hi | 13:29 |
| *** bjolo has quit IRC | 13:30 | |
| openstackgerrit | Dennis Mueller proposed openstack/kolla: Added neutron_metering_agent to ansible tasks for neutron https://review.openstack.org/346630 | 13:31 |
| *** belmoreira has quit IRC | 13:31 | |
| *** signed8bit has joined #openstack-kolla | 13:33 | |
| *** dmueller-evoila has quit IRC | 13:41 | |
| zhubingbing_ | hi | 13:41 |
| *** rbergeron has quit IRC | 13:43 | |
| *** rbergeron has joined #openstack-kolla | 13:43 | |
| *** dmueller-evoila has joined #openstack-kolla | 13:44 | |
| *** huikang has joined #openstack-kolla | 13:48 | |
| *** belmoreira has joined #openstack-kolla | 13:50 | |
| *** ayoung has joined #openstack-kolla | 13:55 | |
| *** dasTor has joined #openstack-kolla | 13:57 | |
| *** dims has quit IRC | 13:59 | |
| *** bootsha__ has quit IRC | 14:01 | |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes: WIP Adding Mariadb shutdown command to POD delete event https://review.openstack.org/355189 | 14:01 |
| *** jtriley_ has joined #openstack-kolla | 14:02 | |
| Pavo | morning everyone | 14:03 |
| *** jtriley has quit IRC | 14:04 | |
| kbaegis | Morning | 14:05 |
| *** dims has joined #openstack-kolla | 14:05 | |
| openstackgerrit | zhubingbing proposed openstack/kolla: repair aodh image dockerfile https://review.openstack.org/350715 | 14:07 |
| *** sdake has joined #openstack-kolla | 14:10 | |
| zhubingbing_ | morning everyone | 14:12 |
| Pavo | morning | 14:13 |
| huikang | Morning | 14:14 |
| *** dmueller-evoila has quit IRC | 14:15 | |
| *** jtriley_ has quit IRC | 14:18 | |
| Jeffrey4l_ | huikang, hi please review this https://review.openstack.org/345751 ;) | 14:19 |
| *** fragatina has joined #openstack-kolla | 14:20 | |
| *** dave-mcc_ has joined #openstack-kolla | 14:21 | |
| *** fragatina has quit IRC | 14:21 | |
| *** jtriley has joined #openstack-kolla | 14:21 | |
| *** fragatina has joined #openstack-kolla | 14:21 | |
| *** dave-mccowan has quit IRC | 14:22 | |
| *** dave-mccowan has joined #openstack-kolla | 14:23 | |
| *** dmueller-evoila has joined #openstack-kolla | 14:23 | |
| wznoinsk | I got the ironic on kolla stable/mitaka working, found bugs are here http://bit.ly/2bbxNk0 , could someone triage or comment on them? | 14:24 |
| *** dave-mcc_ has quit IRC | 14:25 | |
| rhallisey | wznoinsk, sure | 14:25 |
| *** dave-mcc_ has joined #openstack-kolla | 14:26 | |
| openstackgerrit | zhubingbing proposed openstack/kolla: repair aodh image dockerfile https://review.openstack.org/350715 | 14:27 |
| rhallisey | wznoinsk, that's awesome you got it working | 14:28 |
| *** dave-mccowan has quit IRC | 14:29 | |
| wznoinsk | rhallisey: thanks, I was basing my work on ground work done by our work-mates, sean-k-mooney being one of them | 14:29 |
| *** dave-mccowan has joined #openstack-kolla | 14:29 | |
| rhallisey | nice :) | 14:29 |
| *** dave-mcc_ has quit IRC | 14:32 | |
| sean-k-mooney | rhallisey: wznoinsk: and rodolfoh (i think that his irc name) have been doing some good work on this on and off this cycle. i was really glad when they told me it was working this morning | 14:33 |
| sean-k-mooney | sdake: ^ fyi wznoinsk got ironic working with kolla locally. | 14:34 |
| *** zhurong has joined #openstack-kolla | 14:37 | |
| *** zhurong has quit IRC | 14:39 | |
| kbaegis | Huh | 14:39 |
| kbaegis | 3 minute creation time for an instance on 2.0.1 kolla | 14:39 |
| kbaegis | Still hasn't finished spawning | 14:40 |
| *** JoseMello has quit IRC | 14:40 | |
| *** ralonsoh has joined #openstack-kolla | 14:44 | |
| wznoinsk | sdake: rhallisey credits to ralonsoh too | 14:44 |
| rhallisey | nice work :) | 14:44 |
| *** sdake has quit IRC | 14:48 | |
| openstackgerrit | Paul Bourke proposed openstack/kolla: Check volume group exists for Cinder in prechecks https://review.openstack.org/354154 | 14:50 |
| *** lyncos has joined #openstack-kolla | 14:52 | |
| *** michauds has joined #openstack-kolla | 14:56 | |
| sbezverk | Jeffrey4l_, infra people keeps ignoring our requests. :-( | 15:02 |
| Jeffrey4l_ | sbezverk, yup :( | 15:03 |
| openstackgerrit | Paul Bourke proposed openstack/kolla: Add basic checking for network_interface to prechecks https://review.openstack.org/355515 | 15:03 |
| sbezverk | I am going to write nasty email to openstack mailer, asking wtf | 15:03 |
| *** mbound has joined #openstack-kolla | 15:05 | |
| *** senk_ has quit IRC | 15:06 | |
| rhallisey | wznoinsk, ok all triaged. Looks like a bunch of those are quick and painless Dockerfile changes | 15:06 |
| *** dmueller-evoila has quit IRC | 15:09 | |
| Pavo | anyone have a networking diagram on how the interfaces for nodes are suppose to be assigned best pratice | 15:11 |
| Pavo | practice* | 15:11 |
| rhallisey | Pavo, I know sdake does | 15:13 |
| rhallisey | he's not around at the moment | 15:13 |
| Pavo | yeah docs doesn't rally explain that well | 15:13 |
| Pavo | really* jesus I can't type today lol | 15:14 |
| openstackgerrit | Merged openstack/kolla: Customizations for Tgtd https://review.openstack.org/354392 | 15:16 |
| *** kbaegis has quit IRC | 15:16 | |
| rhallisey | Pavo, https://github.com/openstack/kolla/blob/master/doc/kollanet.png | 15:16 |
| Pavo | ok so from that digram I take it that eth0 is internal? | 15:17 |
| rhallisey | yes | 15:19 |
| Pavo | is that referring to the deployer node or the controller and compute nodes? | 15:20 |
| rhallisey | I believe its for controller & compute | 15:21 |
| Pavo | thats what I thought, thanks | 15:21 |
| rhallisey | no problem | 15:21 |
| Pavo | so whats assigning dhcp for internal? | 15:21 |
| Pavo | dhcp_agent? | 15:22 |
| openstackgerrit | Wei Cao proposed openstack/kolla: Add solum container https://review.openstack.org/355408 | 15:24 |
| rhallisey | until bifrost is around, an op will do it by hand | 15:24 |
| *** lyncos has quit IRC | 15:31 | |
| *** kbaegis has joined #openstack-kolla | 15:32 | |
| *** diogogmt has joined #openstack-kolla | 15:33 | |
| *** lyncos has joined #openstack-kolla | 15:35 | |
| openstackgerrit | Merged openstack/kolla: Dockerfiles for Networking-SFC Container https://review.openstack.org/344930 | 15:37 |
| openstackgerrit | Merged openstack/kolla: Volume specification for ironic-api was invalid. https://review.openstack.org/351206 | 15:37 |
| *** belmoreira has quit IRC | 15:38 | |
| *** harlowja_at_home has joined #openstack-kolla | 15:40 | |
| *** haplo37__ has joined #openstack-kolla | 15:41 | |
| wznoinsk | rhallisey: thanks, they're not hard changes, true | 15:41 |
| openstackgerrit | Merged openstack/kolla: Modify some spelling mistakes https://review.openstack.org/350544 | 15:42 |
| zhubingbing_ | TASK [aodh : Creating the aodh service and endpoint] *************************** | 15:45 |
| zhubingbing_ | fatal: [localhost]: FAILED! => {"failed": true, "msg": "'item' is undefined"} | 15:45 |
| zhubingbing_ | to retry, use: --limit @/usr/local/share/kolla/ansible/site.retry | 15:45 |
| zhubingbing_ | PLAY RECAP ********************************************************************* | 15:45 |
| zhubingbing_ | localhost : ok=141 changed=1 unreachable=0 failed=1 | 15:45 |
| zhubingbing_ | who can help me | 15:45 |
| *** vhosakot has joined #openstack-kolla | 15:47 | |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes: WIP Adding Mariadb shutdown command to POD delete event https://review.openstack.org/355189 | 15:47 |
| openstackgerrit | Prithiv proposed openstack/kolla: Add Networking-SFC roles in ansible https://review.openstack.org/346842 | 15:52 |
| *** vhosakot has quit IRC | 15:52 | |
| openstackgerrit | Merged openstack/kolla: Fix the neutron-lbaas-agent container issues https://review.openstack.org/347606 | 15:53 |
| *** vhosakot has joined #openstack-kolla | 15:53 | |
| openstackgerrit | Merged openstack/kolla: Fix 'reconfigure' command of kolla https://review.openstack.org/348120 | 15:53 |
| openstackgerrit | Merged openstack/kolla: Customizations for Ceph https://review.openstack.org/351823 | 15:54 |
| openstackgerrit | Merged openstack/kolla: Add heka custom config to define user's log pipline https://review.openstack.org/352662 | 15:55 |
| *** vhosakot has quit IRC | 15:57 | |
| *** vhosakot has joined #openstack-kolla | 15:58 | |
| sbezverk | kolla-kube gates should be restored.. Thank you Jeffrey for your help!! | 16:00 |
| rhallisey | nice | 16:00 |
| rhallisey | I didn't realize he had a patch for that :) | 16:00 |
| rhallisey | we had the exact same patch | 16:01 |
| rhallisey | glad that's fixed | 16:01 |
| *** daneyon_ has joined #openstack-kolla | 16:02 | |
| rhallisey | zhubingbing_, I'm not sure what it is, I would output item.url and item.interface and see what's going on there | 16:03 |
| zhubingbing_ | i have fixed it | 16:04 |
| zhubingbing_ | As a result of aodh_api_port no setting | 16:05 |
| zhubingbing_ | - -thank you rhallisey | 16:05 |
| *** daneyon_ has quit IRC | 16:06 | |
| *** srwilkers has joined #openstack-kolla | 16:10 | |
| *** dims has quit IRC | 16:10 | |
| *** vhosakot has quit IRC | 16:17 | |
| *** vhosakot has joined #openstack-kolla | 16:18 | |
| *** harlowja_at_home has quit IRC | 16:28 | |
| *** rstarmer has joined #openstack-kolla | 16:32 | |
| *** Jeffrey4l_ has quit IRC | 16:33 | |
| *** duonghq has quit IRC | 16:37 | |
| *** daneyon_ has joined #openstack-kolla | 16:38 | |
| *** daneyon has quit IRC | 16:40 | |
| *** dims has joined #openstack-kolla | 16:42 | |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes: WIP Adding Mariadb shutdown command to POD delete event https://review.openstack.org/355189 | 16:43 |
| *** dims has quit IRC | 16:47 | |
| Mech422 | morning | 16:48 |
| openstackgerrit | Ken Johnston proposed openstack/kolla: Minor formatting change to CONTRIBUTING.rst https://review.openstack.org/347032 | 16:49 |
| *** lrensing_ has joined #openstack-kolla | 16:50 | |
| *** dims has joined #openstack-kolla | 16:53 | |
| openstackgerrit | Ken Johnston proposed openstack/kolla: Readability Improvements to Advanced Config Doc https://review.openstack.org/347102 | 16:54 |
| *** signed8bit is now known as signed8bit_Zzz | 17:00 | |
| *** sdake has joined #openstack-kolla | 17:00 | |
| sdake | sean-k-mooney hotness!! | 17:01 |
| *** daneyon has joined #openstack-kolla | 17:03 | |
| *** signed8bit_Zzz is now known as signed8bit | 17:04 | |
| *** ralonsoh has quit IRC | 17:04 | |
| openstackgerrit | Merged openstack/kolla-kubernetes: Jinja2 pvc template error because of imcomplete if else https://review.openstack.org/354072 | 17:06 |
| *** daneyon_ has quit IRC | 17:06 | |
| *** daneyon_ has joined #openstack-kolla | 17:07 | |
| *** harlowja has joined #openstack-kolla | 17:08 | |
| openstackgerrit | Merged openstack/kolla-kubernetes: Add persistent volumes to Rabbitmq https://review.openstack.org/354073 | 17:09 |
| *** daneyon has quit IRC | 17:09 | |
| *** jrich523_ has joined #openstack-kolla | 17:11 | |
| *** jrich523_ has quit IRC | 17:12 | |
| *** jrich523 has quit IRC | 17:12 | |
| *** jrich523 has joined #openstack-kolla | 17:12 | |
| *** wirehead_ has quit IRC | 17:15 | |
| *** vhosakot has quit IRC | 17:15 | |
| *** vhosakot has joined #openstack-kolla | 17:16 | |
| sean-k-mooney | sdake: im assuming you were refering to ironic as a service :) | 17:17 |
| sdake | sean-k-mooney yup | 17:17 |
| *** Serlex has quit IRC | 17:17 | |
| sdake | oh thatis what you meant - i thought you meant bifrost deployment | 17:17 |
| sdake | that is even hotter | 17:18 |
| sdake | so you can use baremetal as a service now with newton once patches land? | 17:18 |
| *** wirehead_ has joined #openstack-kolla | 17:18 | |
| sdake | yo wirehead_ | 17:18 |
| sean-k-mooney | sdake: wznoinsk was using stable/mitaka but i dont think the iroic code has change in kolla since then so his next step is to move to master and try again | 17:19 |
| sdake | sweet | 17:19 |
| sdake | so what hapepns when you use it? | 17:19 |
| sdake | can you give me a rundown of what occurs? | 17:19 |
| sean-k-mooney | sdake: but ya after that you should be able to use it with master. | 17:20 |
| sdake | does it interfere with nova? | 17:20 |
| sean-k-mooney | sdake: yes | 17:20 |
| sdake | (nova's scheduler specifically) | 17:20 |
| sean-k-mooney | sdake: we were booting the server via horison | 17:20 |
| sean-k-mooney | sdake: it works with nova too | 17:20 |
| sdake | so its vm or baremetal but not both at the same time correct? | 17:20 |
| sean-k-mooney | sdake: currently but i would like to figure out how to enable both at the same time | 17:21 |
| sdake | yes that is what we need | 17:21 |
| sdake | but atleast we are making progress :) | 17:21 |
| sdake | yay for progress | 17:21 |
| sean-k-mooney | sdake nova can handle both in the same could as it the flavor that select if its ironic or a nova vm | 17:22 |
| Pavo | anyone have a suggested setup for 2 node deployment using VMs for testing? ie... network adapters, cpus, ram | 17:22 |
| *** daneyon has joined #openstack-kolla | 17:22 | |
| sdake | yes I would like for it to work like that but every time i asked jpeeler about it he said its not possible to work that way | 17:22 |
| sdake | pavo - 80gb disk - 2 nic - 8g ram min - try to get above that | 17:23 |
| sdake | per vm | 17:23 |
| sdake | try to get aboe the 8g ram I mean | 17:23 |
| sdake | the 80gb disk is probably overkill | 17:23 |
| sdake | I haven't measured usage on a vm for a long time | 17:23 |
| sdake | but it at one time was 80g | 17:24 |
| sdake | but we have optimized | 17:24 |
| Pavo | ok cool I have 200Gb disk, 4 cpus, 16Gb ram and 2 NICs per | 17:24 |
| sdake | but also added features | 17:24 |
| wirehead_ | heya sdake | 17:24 |
| sean-k-mooney | sdake: i think we might be able to make it work via kollas ansible inventory file. currently the enable ironic sets the virt driver to ironic on all compute nodes but that is not what we want in this case. | 17:24 |
| sdake | Pavo personally if you have never deployed kolla before I'd go for an AIO | 17:24 |
| Pavo | so should I have external nic bridged and internal nic NAT? | 17:24 |
| sdake | pavo which hypervisor are you using | 17:24 |
| Pavo | sdake well I have been trying for 2 days now | 17:24 |
| sdake | pavo the fastest path is using bare metal | 17:25 |
| sdake | kolla doesn't make a mess to clean up | 17:25 |
| sdake | its very tidy - unlike something like devstack for example | 17:25 |
| Pavo | yeah I understand that but if I can't get it deployed to 2 VMs then I doubt I can get it deployed to 2x baremetals | 17:25 |
| sean-k-mooney | Pavo: i use a 8 core 16GB vm with 40 GB of space for my dev usually but 8 GB should work too as sdake said | 17:25 |
| *** mark-casey has joined #openstack-kolla | 17:25 | |
| *** ayoung has quit IRC | 17:25 | |
| sdake | pavo can't get deployed for what reason | 17:25 |
| sdake | pavo - firt i'd focus on getting a nova vm booted | 17:26 |
| *** daneyon_ has quit IRC | 17:26 | |
| sdake | then sort out getting host networking and all that jazz setup later | 17:26 |
| Pavo | well the late attempt was I think networking issue, said that host from node1 had to be resolvable from node2 but had all nodes and IPs in /etc/hosts for all VMs so... have no clue yet | 17:26 |
| sdake | Pavo vms present all kinds of network challlenges - if you know how to use virutalbox well you could probably figure it out | 17:27 |
| sdake | otherwise, a baremetal machine with two nics is easy | 17:27 |
| wirehead_ | Your friends will love you if you nick some nics. | 17:27 |
| sdake | pavo when you get to that point again ask in channel - someone can walk you through it | 17:27 |
| sean-k-mooney | Pavo: if you are using virtmanager/libvirt to spawn the vms then yes create a new linux bridge. nat the first interface in each vm and add the second interface to the bridge with no dhcp on the bridge | 17:27 |
| sean-k-mooney | Pavo: baremetal kolla is simpler the in vms though as sdake said | 17:28 |
| Pavo | yeah using vmware fusion on my mac pro 5,1 | 17:28 |
| Pavo | yeah I think it will be easier also, just trying to get the process down right and to get all the repos and dependencies correctly so I can do it on bare metal machines completely offline | 17:30 |
| *** vhosakot has quit IRC | 17:30 | |
| Pavo | current setup requires airgap installs | 17:30 |
| Pavo | or guess I should say current policies | 17:31 |
| sean-k-mooney | Pavo: ah offline kolla is not really a thing. it can be done once the container build server has internet access but you will either have to set up a registery or copy all of the contianer to the other hosts | 17:32 |
| Pavo | yeah got all the images built and pushed to private registry | 17:32 |
| Pavo | on the deployer machine | 17:32 |
| Pavo | which is only connected to the target nodes | 17:33 |
| sean-k-mooney | Pavo: ok cool and the target machines can acess that machine over the network? | 17:33 |
| sean-k-mooney | never mind thats a dumb question as it is deploying the other nodes | 17:33 |
| *** vhosakot has joined #openstack-kolla | 17:34 | |
| Pavo | yeah, I have seen it done before just trying to replicate the process, biggest issue right now is what kolla uses for for internal and external interface configs, to understand the network design on how the nodes should be configured | 17:36 |
| *** lyncos has quit IRC | 17:40 | |
| *** ayoung has joined #openstack-kolla | 17:41 | |
| *** athomas has quit IRC | 17:45 | |
| *** senk_ has joined #openstack-kolla | 17:45 | |
| sdake | pavo you can make it easy or can make it hard | 17:46 |
| sdake | pavo i have a diagram of hard | 17:46 |
| sdake | (as in secure) | 17:46 |
| Pavo | yeah secure would be ideal | 17:46 |
| sdake | easy = eval = 2 nics | 17:46 |
| *** JoseMello has joined #openstack-kolla | 17:47 | |
| *** inc0 has joined #openstack-kolla | 17:47 | |
| inc0 | hey | 17:47 |
| inc0 | I'm back from my voyages, so I should help more' | 17:48 |
| sdake | https://www.gliffy.com/go/publish/10486755 | 17:48 |
| inc0 | vhosakot, around? | 17:48 |
| sdake | pavo ^^ | 17:48 |
| sdake | pavo before going complex i'd focus on getting somethign workign :) | 17:48 |
| sdake | baby steps | 17:48 |
| Pavo | now thats a network digram thanks sdake | 17:48 |
| *** vhosakot has quit IRC | 17:54 | |
| *** srwilkers has quit IRC | 17:54 | |
| *** srwilkers has joined #openstack-kolla | 17:55 | |
| *** Pavo_ has joined #openstack-kolla | 17:57 | |
| *** Pavo has quit IRC | 17:58 | |
| *** Pavo_ is now known as Pavo | 17:58 | |
| *** inc0 has quit IRC | 18:01 | |
| kbaegis | Pretty good one. I like the "sorage network" :) | 18:03 |
| sdake | thanks it took me about 6 hours to draw it | 18:09 |
| wirehead_ | So, it was a long drawn-out process. | 18:09 |
| sdake | most of that was sorting out how the networks should connect | 18:09 |
| openstackgerrit | zhubingbing proposed openstack/kolla: Add aodh role https://review.openstack.org/351027 | 18:09 |
| sdake | wirehead_ i don't even know what to say ;-) | 18:09 |
| *** Pavo has quit IRC | 18:10 | |
| *** kbaegis1 has joined #openstack-kolla | 18:15 | |
| *** kbaegis has quit IRC | 18:15 | |
| *** vhosakot has joined #openstack-kolla | 18:15 | |
| *** mbound has quit IRC | 18:16 | |
| *** zhubingbing_ has quit IRC | 18:20 | |
| *** egonzalez90 has joined #openstack-kolla | 18:21 | |
| *** inc0 has joined #openstack-kolla | 18:21 | |
| *** vhosakot_ has joined #openstack-kolla | 18:24 | |
| *** vhosakot has quit IRC | 18:25 | |
| *** senk_ has quit IRC | 18:27 | |
| *** Pavo has joined #openstack-kolla | 18:28 | |
| openstackgerrit | Merged openstack/kolla: Minor formatting change to CONTRIBUTING.rst https://review.openstack.org/347032 | 18:30 |
| *** vhosakot_ has quit IRC | 18:33 | |
| *** neophy has joined #openstack-kolla | 18:33 | |
| *** ayoung has quit IRC | 18:37 | |
| sdake | any progress on the networking on osic cluster? | 18:42 |
| sdake | inc0 ? | 18:43 |
| *** daneyon_ has joined #openstack-kolla | 18:44 | |
| *** vhosakot has joined #openstack-kolla | 18:47 | |
| *** vhosakot has quit IRC | 18:47 | |
| inc0 | sdake, I need to dig through it | 18:47 |
| inc0 | irtermite, around? | 18:47 |
| sdake | this problem is blocking all progress on osic work | 18:47 |
| *** vhosakot has joined #openstack-kolla | 18:48 | |
| inc0 | but what exactly is problem? | 18:48 |
| sdake | ovs dhcp doesn't work | 18:48 |
| inc0 | I have just Jeffreys mail that he gets lost packets | 18:48 |
| sdake | the next step was someone was going to try linuxbridge | 18:48 |
| sdake | i think lost packets = all packets lost | 18:48 |
| inc0 | and it seems to be TOR problem? | 18:48 |
| inc0 | not our own for sure? | 18:48 |
| sdake | no idea if its a tor problem or problem with our code | 18:49 |
| sdake | hence the desire to deploy with linuxbridge | 18:49 |
| inc0 | I doubt it's code | 18:49 |
| inc0 | more likely configuration | 18:49 |
| sean-k-mooney | sdake: wasnt it working fine when inc0 did the first somketest | 18:49 |
| *** daneyon_ has quit IRC | 18:49 | |
| sdake | sean-k-mooney he never sshed into the vm | 18:49 |
| inc0 | I pinged it | 18:49 |
| inc0 | through namespace | 18:49 |
| sean-k-mooney | sdake: ^ | 18:49 |
| inc0 | which seems ok | 18:49 |
| sean-k-mooney | sdake: since the ping worked dhcp worked | 18:49 |
| inc0 | it's our provider vlan that seems fleaky | 18:49 |
| sdake | sean-k-mooney we need it accessable outside of namesapce to operate with rally | 18:49 |
| sean-k-mooney | sdake: yes the floating ip need to work via br-ex | 18:50 |
| *** haplo37__ has quit IRC | 18:50 | |
| inc0 | ok, I'm getting into cluster | 18:50 |
| inc0 | and debugging networking | 18:50 |
| inc0 | yay | 18:50 |
| sean-k-mooney | sdake: if that is the issue its likely a route is missing | 18:50 |
| sdake | if i knew how it worked - id help :) | 18:51 |
| sdake | it took me months to get my internal test bed operational | 18:52 |
| sdake | internal as in the one in my house | 18:52 |
| sdake | the networking - always the networking | 18:52 |
| sdake | inc0 i didn't see jeffrey4l's mail | 18:52 |
| inc0 | he just asked me if switches are misconfigured | 18:53 |
| inc0 | nothing more | 18:53 |
| inc0 | so we have linuxbridge deployed | 18:53 |
| sean-k-mooney | sdake: as long as i dont have to deal with iptables,tc or linux bridge i dont mind ovs openstack networking too much. | 18:53 |
| sdake | ya deploying linuxbridge was the next step in our debug process | 18:54 |
| sean-k-mooney | sdake: the kernel stack is much harder to debug then ovs at least for me | 18:54 |
| sdake | if both are broken - its likely an environmental problem | 18:54 |
| sdake | jeffrey4l probably did that linuxbridge deploy | 18:54 |
| sdake | and sent the email as a consequence of the results | 18:55 |
| sdake | i know for certain at one point the networks were not setup via neutron properly | 18:55 |
| sdake | since it was 10./8 address | 18:55 |
| sdake | in other words, my personal network environment form init-runonce | 18:56 |
| sean-k-mooney | sdake: 10./8 or 10.0.0.0/24? | 18:56 |
| sean-k-mooney | 10.0.0.0/24 is the default in the init-runonce | 18:57 |
| *** Pavo has quit IRC | 18:57 | |
| sdake | 10.0.2.0/24 | 18:57 |
| sdake | and 10.0.0.0/24 | 18:57 |
| sdake | one was float one was management work iirc | 18:57 |
| sdake | err rather tenant network | 18:57 |
| *** lrensing_ has quit IRC | 18:58 | |
| sean-k-mooney | yes 10.0.2.0/24 is the default public/foating ip subnet | 18:58 |
| sdake | not in osic... | 18:58 |
| sean-k-mooney | 10.0.0.0/24 is the default tenant subnet in the init-runonce so on that deploy that script was probably run by mistake | 18:58 |
| sdake | running init-runonce gets evrything into shape | 18:59 |
| sdake | but then neutron needs to be conffigured after tht | 18:59 |
| sdake | pbourke iirc had solved that prolem | 18:59 |
| sdake | we have beeen blocked on this for 4-5 days | 18:59 |
| sdake | it prevents rally from running so wee can't gather benchmark results | 19:00 |
| sean-k-mooney | sdake: i wonder if when pbourke change the external network subnet,did he also adjust the routing table on the host runing rally to send it to the br-ex or onto it correct vlan network? | 19:01 |
| sdake | unlikely but dont know | 19:02 |
| inc0 | hmm | 19:02 |
| inc0 | soo | 19:02 |
| inc0 | I can ping it | 19:02 |
| *** srwilkers has quit IRC | 19:03 | |
| inc0 | let's play around it | 19:03 |
| inc0 | something is not right | 19:03 |
| *** srwilkers has joined #openstack-kolla | 19:03 | |
| sdake | can you ssh to it? | 19:03 |
| sean-k-mooney | if not it could be the mtu | 19:03 |
| sdake | could also be ingress/outgress rules | 19:03 |
| inc0 | not ssh | 19:03 |
| inc0 | unlikely | 19:03 |
| inc0 | as I'm in low level networking | 19:04 |
| inc0 | connection refused on .22 | 19:04 |
| inc0 | :22 | 19:04 |
| inc0 | which is strange | 19:04 |
| sdake | looks like firewll to me | 19:04 |
| *** haplo37__ has joined #openstack-kolla | 19:04 | |
| sdake | firewall doesn't block ping typically | 19:04 |
| sdake | but does block ssh unless enabled | 19:04 |
| sean-k-mooney | i guess check default security group rules but the init-runouce opens port 22 | 19:05 |
| inc0 | but drop on .22 would cause timeout not connection refused | 19:05 |
| inc0 | but let's see | 19:05 |
| sdake | i have found flushing iptables on the target can be of help | 19:05 |
| sdake | lunch bbiaf | 19:06 |
| sean-k-mooney | inc0: im assuming you are using cirros images so ssh should be enabeld right. | 19:08 |
| inc0 | yeah | 19:08 |
| sean-k-mooney | inc0: if the tenant is useing vxlan and the mtu on the interfaces is not reduced to <1450 | 19:08 |
| sean-k-mooney | inc0: the ssh packet will be droped | 19:08 |
| sean-k-mooney | vxlan does not allow ip fragmentation of a vxlan packet. | 19:09 |
| inc0 | well network is meant to be vlan | 19:10 |
| *** haplo37__ has quit IRC | 19:10 | |
| sean-k-mooney | the tenant network or the osic network | 19:10 |
| inc0 | ,that's bad | 19:10 |
| *** egonzalez90 has quit IRC | 19:10 | |
| inc0 | I have 80% packet drop | 19:10 |
| inc0 | 1 ping went through | 19:10 |
| sean-k-mooney | inc0: do you have a broadcast storm again? | 19:10 |
| sean-k-mooney | inc0: was the stp issue ever rootcaused | 19:11 |
| inc0 | hmm | 19:12 |
| inc0 | I don't see nay traffic now | 19:12 |
| sdake | i mean flush before deploy | 19:13 |
| *** egonzalez90 has joined #openstack-kolla | 19:14 | |
| inc0 | there is some traffic | 19:14 |
| inc0 | brb gonna walk to irtermite | 19:15 |
| sean-k-mooney | inc0: just not enough to account for the packet loss. | 19:15 |
| *** mbound has joined #openstack-kolla | 19:17 | |
| *** srwilkers has quit IRC | 19:18 | |
| inc0 | yeah stp storm allright | 19:19 |
| *** srwilkers has joined #openstack-kolla | 19:20 | |
| sean-k-mooney | :( inc0 is it coming from the osic TORs or something we did? | 19:20 |
| inc0 | hard to tell | 19:20 |
| sean-k-mooney | inc0: stp is truned off in the ovs deployment but is it on with linux bridge? | 19:21 |
| *** haplo37__ has joined #openstack-kolla | 19:22 | |
| *** mbound has quit IRC | 19:22 | |
| inc0 | so we've seen stp in ovs too | 19:22 |
| inc0 | so we might be looking at TOR with stp enabled | 19:22 |
| sean-k-mooney | inc0: could this be related to the bond in any way? | 19:23 |
| inc0 | if we have loop, yes | 19:23 |
| inc0 | however | 19:23 |
| inc0 | I'm trying to figure out where | 19:23 |
| sean-k-mooney | its ironic considering stp is intended to keep your network from exploding when you have a loop | 19:24 |
| inc0 | yeah | 19:24 |
| inc0 | however I'm not sure about loop | 19:24 |
| inc0 | more likely one of our own linuxbridges has stp enabled | 19:24 |
| sean-k-mooney | see anyting in brctl show | 19:25 |
| inc0 | none of nodes has it enabled | 19:26 |
| inc0 | I'll play till the end of the day and tomorrow I'd repave bare metal | 19:27 |
| sean-k-mooney | inc0: could cobbler maybe reconfigure the tors to disable it | 19:27 |
| inc0 | we don't have access to TOR | 19:27 |
| inc0 | and I think TOR has it disabled | 19:27 |
| sean-k-mooney | it might be worth running kolla-ansible cleanup and seeing if the traffic stops | 19:29 |
| sean-k-mooney | that will tell you if its on our end as it shoudl clean up the linux bridges. at least i think it will | 19:30 |
| *** signed8bit is now known as signed8bit_Zzz | 19:30 | |
| inc0 | yeah | 19:33 |
| inc0 | and rmmod openvswitch | 19:33 |
| *** lyncos has joined #openstack-kolla | 19:34 | |
| irtermite | inc0: send me the tcp dump and I will talk with networking when I am done with this ticket | 19:34 |
| sean-k-mooney | well if the deployment is using linux bridge im not sure that will help | 19:34 |
| inc0 | sure, let me repave whole thing and check if that helps | 19:35 |
| irtermite | sean-k-mooney: tor is configured for spanning-tree port type edge... they are automatically put into a fowarding state and do not go through the listening state << response from networking when this was asked about this environment on Aug 4th. | 19:37 |
| *** signed8bit_Zzz is now known as signed8bit | 19:38 | |
| sean-k-mooney | irtermite: so looking at http://www.cisco.com/c/en/us/td/docs/switches/datacenter/nexus5000/sw/configuration/guide/cli/CLIConfigurationGuide/SpanningEnhanced.html#20968 | 19:41 |
| sean-k-mooney | in edge mode if we then bond the interfaces would that cause what we are seeing? | 19:42 |
| inc0 | hmm I think I see what's wrong | 19:43 |
| sean-k-mooney | it says "Interfaces that are connected to hosts should not receive STP Bridge Protocol Data Units" for edge ports but that is not what we are seeing | 19:43 |
| inc0 | sean-k-mooney, so we have setup p4p1.733 iface | 19:43 |
| inc0 | and we still use p4p1 as neutron external | 19:44 |
| inc0 | which doesn't seem right | 19:44 |
| sean-k-mooney | inc0: im not certin that would break anyting. the subport should strip out vlan 733 packets but the rest should go to p4p1 tagged | 19:45 |
| sean-k-mooney | are p4p1 and p4p1.733 connected to the same bridge | 19:45 |
| inc0 | shouldn't | 19:45 |
| inc0 | so what I'm going to do is to tell neutron to use p4p1.733 as external | 19:46 |
| inc0 | and create flat network on top of it | 19:46 |
| inc0 | with address space corresponding to this vlan | 19:46 |
| inc0 | it was meant for ceph replication, but we're not doing this now | 19:46 |
| inc0 | if it works we'll repave everything and sort out vlans | 19:47 |
| sean-k-mooney | that should work correctly | 19:47 |
| sean-k-mooney | we will not be able to test tenant vlan but that does not give us much anyway | 19:47 |
| sean-k-mooney | we are already testing flat and tunneled networks | 19:47 |
| inc0 | for example I'll leave bond0 out of it and use p4p1 for all non-mgmt non-neuton traffic | 19:47 |
| ntpttr___ | hey inc0, is it possible to use the master branch of kolla to deploy services in another branch, maybe through a globals.yml config option? | 19:47 |
| inc0 | ntpttr___, you mean use master kolla to deploy mitaka cinder? | 19:48 |
| ntpttr___ | inc0: yep | 19:48 |
| sean-k-mooney | ntpttr___: it should be | 19:48 |
| inc0 | yeah, just use kolla-build.conf | 19:48 |
| inc0 | look at stable mitaka build-conf | 19:48 |
| sean-k-mooney | ntpttr___: i have used master to deploy mitaka | 19:48 |
| *** ayoung has joined #openstack-kolla | 19:48 | |
| inc0 | defaults will point to stable | 19:48 |
| inc0 | uncomment repo config from it and use it with master | 19:49 |
| inc0 | should be fien | 19:49 |
| *** senk_ has joined #openstack-kolla | 19:49 | |
| ntpttr___ | inc0: okay, do I need to copy kolla-build.conf into /etc/kolla as well then? | 19:50 |
| inc0 | off we go with deployment | 19:50 |
| inc0 | yeah | 19:50 |
| *** senk_ has quit IRC | 19:50 | |
| *** fragatina has quit IRC | 19:53 | |
| sean-k-mooney | inc0: so what is your current network config. neutron_external_interface=p4p1.733 networking_interface=p1p1 | 19:55 |
| *** daneyon_ has joined #openstack-kolla | 19:55 | |
| inc0 | sean-k-mooney, http://paste.openstack.org/show/557618/ | 19:56 |
| inc0 | not exactly | 19:56 |
| inc0 | we leave p1p1 out of it | 19:56 |
| inc0 | we have bunch of tagged networks on bond0 | 19:56 |
| inc0 | but in fact we should put a neutron there and not tag it at all | 19:57 |
| inc0 | instead use neutrons provider vlan nets | 19:57 |
| *** lyncos has quit IRC | 19:57 | |
| *** egonzalez90 has quit IRC | 19:58 | |
| *** daneyon has quit IRC | 19:58 | |
| sean-k-mooney | use neutron provider vlans for tenants or for the storage network etc? | 19:58 |
| inc0 | tenants | 19:58 |
| sean-k-mooney | ok thats what i taught | 19:58 |
| inc0 | we'll put all the storage and all on p4p1 | 19:59 |
| inc0 | with tags | 19:59 |
| sean-k-mooney | so bassical give bond0 to neutron and everything else goes via one of the other interfaces | 19:59 |
| *** lyncos has joined #openstack-kolla | 19:59 | |
| inc0 | yeah | 19:59 |
| sean-k-mooney | ya that will likely cause less issues | 19:59 |
| inc0 | if that fails, I'll ask politely to unbind ifaces on TOR | 20:00 |
| inc0 | and we'll deal with raw ifaces | 20:00 |
| inc0 | we don't really need bonding | 20:00 |
| sean-k-mooney | ya i guess even though we will be doing scale testing of the control plane not performance testing of the tenant networking | 20:01 |
| *** srwilkers has quit IRC | 20:02 | |
| sean-k-mooney | the bond is not required | 20:02 |
| sean-k-mooney | inc0: by the way one thing that kolla is missing is the ability to automatically add the neutron local ip to the ovs external brige to make tunneled traffic go over the provider bridge | 20:04 |
| inc0 | well we'd need to specify this bridge somewhere | 20:05 |
| inc0 | IP* | 20:05 |
| *** lyncos has quit IRC | 20:05 | |
| sean-k-mooney | ill take a look at adding that next cycle but im not going to start anything new until after the rcs are out | 20:05 |
| sean-k-mooney | there is enough in the review queue as it is. | 20:06 |
| *** huikang has quit IRC | 20:08 | |
| irtermite | inc0: sean-k-mooney: was this seen on more than one server? which one(s) | 20:10 |
| *** lyncos has joined #openstack-kolla | 20:10 | |
| inc0 | irtermite, all of them, I repaved and spawning new vm | 20:10 |
| inc0 | let me test it out | 20:10 |
| sean-k-mooney | inc0: oh is it redeployed? | 20:11 |
| inc0 | yeah | 20:11 |
| inc0 | configuring sectroups etc as we speak | 20:11 |
| sean-k-mooney | i have to run soon. but kind of interested if that was the issue | 20:11 |
| *** vhosakot has quit IRC | 20:12 | |
| inc0 | still bad | 20:14 |
| inc0 | no dhcp traffic | 20:14 |
| inc0 | on flat network | 20:14 |
| *** neophy has quit IRC | 20:14 | |
| sean-k-mooney | no flat network in neutron or create in ovs? | 20:15 |
| inc0 | neutron has flat network | 20:15 |
| sean-k-mooney | with the correct physnet? | 20:16 |
| sean-k-mooney | im assuming you ran basically this https://github.com/openstack/kolla/blob/master/tools/init-runonce#L54 | 20:16 |
| inc0 | so I see dhcp traffic in p4p1.733 on controller | 20:16 |
| inc0 | sean-k-mooney, not exactly | 20:16 |
| inc0 | I enabled dhcp in flat network | 20:17 |
| inc0 | and spawned vm there | 20:17 |
| sean-k-mooney | oh ok | 20:17 |
| sean-k-mooney | so on the contoler p4p1,733 is reciving the dhcp boot request form the vm correct? | 20:18 |
| sean-k-mooney | are you seeing that on the host with tcpdump or in the network namespace? | 20:18 |
| inc0 | I see it on host | 20:18 |
| inc0 | in a moment I'll go to compute node | 20:19 |
| inc0 | cuz dhcp is reaching host | 20:19 |
| inc0 | and host replies | 20:19 |
| inc0 | but vm never gets reply | 20:19 |
| inc0 | which might mean that we have arp issue | 20:19 |
| inc0 | because multicast is working, unicast seems not | 20:19 |
| inc0 | brb | 20:19 |
| *** inc0 has quit IRC | 20:19 | |
| *** kbaegis has joined #openstack-kolla | 20:20 | |
| *** kbaegis1 has quit IRC | 20:20 | |
| sean-k-mooney | inc0 hum ok maybe enableing the l2_pop mech driver would help as arp would be resolved on the node but then again that make things more complicated to debug | 20:20 |
| sean-k-mooney | ok got to go have dinner or i might eat my monitor | 20:24 |
| sean-k-mooney | sorry i cant help more. hopefully its something fixable. | 20:24 |
| *** sdake has quit IRC | 20:32 | |
| *** jtriley has quit IRC | 20:33 | |
| *** Pavo has joined #openstack-kolla | 20:38 | |
| ntpttr___ | when we enable tempest in kolla, how do we know things like tempest_image_id, tempest_public_network_id, and that sort of thing to put in globals.yml? Openstack hasn't been deployed yet | 20:38 |
| *** dwalsh has quit IRC | 20:38 | |
| *** Pavo has quit IRC | 20:39 | |
| *** fragatina has joined #openstack-kolla | 20:41 | |
| *** fragatina has quit IRC | 20:42 | |
| sean-k-mooney | ntpttr___: i think tempest need to be deployed after you have deployed your cloud | 20:42 |
| sean-k-mooney | not sure though | 20:42 |
| *** fragatina has joined #openstack-kolla | 20:43 | |
| ntpttr___ | sean-k-mooney: so I deploy, then get all that info to put in globals, then do kolla-ansible deploy again? | 20:44 |
| sean-k-mooney | ntpttr___: reading through the ansible i dont see anything that would allow it to poplate them itself | 20:44 |
| sean-k-mooney | am you could jsut run kolla-ansible deploy -e tempest_image_id=123 --tags tempest | 20:45 |
| sean-k-mooney | i dont think it needs to be in the global.yml | 20:45 |
| sean-k-mooney | but yes deploy cloud. enrole images then deploy tempest | 20:46 |
| ntpttr___ | sean-k-mooney: cool I'll give it a go, thanks | 20:47 |
| *** rstarmer_ has joined #openstack-kolla | 20:48 | |
| sean-k-mooney | ntpttr___: the current workflow is kinda flawed e.g. if you enable tempest we probably shoudl run the init_run once or something similar to setup the required networks and iamges for you but at lest the cureent workflow is scriptable | 20:49 |
| rstarmer_ | why is the nova scheduler such a hater... | 20:49 |
| ntpttr___ | sean-k-mooney: yeah that would be nice if it were automated, but at least documenting the current workflow would probably be good too. It's tough to piece it together looking at the code | 20:50 |
| rstarmer_ | VirtualBox VM based instance, neutron appears to be behaving, glance is happy, the nova-compute container is humming and letting the scheduler know that it has cores avaialble... and still: the filter returns 0 hosts... | 20:50 |
| *** kbaegis has quit IRC | 20:56 | |
| *** sdake has joined #openstack-kolla | 20:57 | |
| *** Pavo has joined #openstack-kolla | 20:59 | |
| *** JoseMello has quit IRC | 21:00 | |
| irtermite | sean-k-mooney: are you all experiencing packet loss between instances only, hosts only, both? over VPN? | 21:00 |
| irtermite | bonding? what mode? | 21:00 |
| irtermite | OVS or LinuxBridge? Think I saw you mention LinuxBridge earlier | 21:01 |
| *** haplo37__ has quit IRC | 21:03 | |
| *** sdake has quit IRC | 21:08 | |
| *** Pavo has quit IRC | 21:09 | |
| *** Pavo has joined #openstack-kolla | 21:13 | |
| *** jamesden_ has joined #openstack-kolla | 21:13 | |
| *** rhallisey has quit IRC | 21:14 | |
| *** sdake has joined #openstack-kolla | 21:17 | |
| rstarmer_ | anyone seen this error before in nova_scheduler: Require both a host and instance NUMA topology to fit instance on host | 21:17 |
| *** haplo37__ has joined #openstack-kolla | 21:18 | |
| wznoinsk | rstarmer_: it's rather question to guys on #openstack-nova but it basically means you can't spawn two numa nodes VM when you have single numa on the host | 21:21 |
| wznoinsk | or other variations | 21:21 |
| rstarmer_ | seems like other than that (and a bug where the _member_ role doesn't seem to be created), my centos-binary kolla 3.0.0 system is up and running | 21:23 |
| sdake | well my tonearm is working | 21:24 |
| sdake | unfortunately my MC catridge is bust | 21:24 |
| sdake | rstarmer_ i think there was some patch activity to enable numa | 21:25 |
| sdake | rstarmer_ perhaps we busted that inadvertently | 21:25 |
| sdake | (patch activity in kolla) | 21:25 |
| *** lyncos_ has joined #openstack-kolla | 21:26 | |
| *** lyncos_ has quit IRC | 21:26 | |
| *** lyncos has quit IRC | 21:26 | |
| rstarmer_ | sdake: any pointer to how I can turn that off? :D | 21:27 |
| sdake | rstarmer_ no clue | 21:27 |
| sdake | git blame is what i'd look at | 21:27 |
| sdake | the review said NUMA right in it | 21:27 |
| rstarmer_ | ok ,will dig in the scheduler see if I can find it. | 21:27 |
| sdake | git log kolla | 21:27 |
| sdake | search for nUMA | 21:27 |
| sdake | git blame the commit | 21:27 |
| sdake | or rathe rgit show the commit | 21:28 |
| wznoinsk | rstarmer_: numa may be requested a flavor-key | 21:28 |
| wznoinsk | s/a/as | 21:29 |
| rstarmer_ | wznoinsk: there doesn't seem to be any key related to NUMA in the flavor I'm using, perhaps this is the issue? | 21:31 |
| rstarmer_ | http://pastebin.com/quJhcg59 | 21:31 |
| wznoinsk | requesting a machine with 1cpu should work no matter what (single numa), let me check something here | 21:32 |
| wznoinsk | rstarmer_: how about image props? | 21:32 |
| rstarmer_ | http://pastebin.com/uLnEgukP | 21:35 |
| rstarmer_ | not that I created | 21:35 |
| *** sdake has quit IRC | 21:39 | |
| irtermite | sean-k-mooney: Note from networking... "If they can rerun that packet capture with the –ne that would be helpful. That will show the multicast address (hopefully) and that actual VLAN IDs." | 21:39 |
| irtermite | "What they’re likely doing is performing packet captures on an interface (trunk) that has upwards of 30+ VLANs on it. They are likely seeing BDPUs being sent from the root bridge for each of the VLANs (PVST). The normal interval is 2 seconds between messages. But when you have 30+ VLANs (which is true for OSIC interfaces), those timers vary between VLANs, and you’ll get the behavior they’re seeing." | 21:39 |
| irtermite | Heading out shortly. Buffer is running. Will see mentions later. | 21:40 |
| rstarmer_ | I just added virt_type=qemu to nova.conf for compute, and now the instance starts, so perhaps the numa "error" wasn't an error, just the lack of a defined virt_type caused the issue? | 21:42 |
| rstarmer_ | not sure, but will test more later. | 21:42 |
| rstarmer_ | now on to the next bug? | 21:43 |
| rstarmer_ | :) | 21:43 |
| wznoinsk | ah, forgot about it | 21:43 |
| rstarmer_ | Wow, I think I can finally say I have a _Working_ Kolla deployed instance. Time to launch (from a blank VM) ~ 3 hours, including image build and an hour of debug. | 21:48 |
| irtermite | Does that mean you all aren't seeing networking issues anymore? | 21:53 |
| mark-casey | irtermite: unrealted I think | 21:53 |
| mark-casey | related | 21:53 |
| irtermite | ACK | 21:53 |
| rstarmer_ | irermite: I don't have any network issues at the moment, but I'mon an All-in-one instance | 21:54 |
| *** sdake has joined #openstack-kolla | 21:54 | |
| irtermite | understod | 21:54 |
| irtermite | understood* | 21:54 |
| irtermite | ok, out for real this time | 21:54 |
| rstarmer_ | Now I get to build a multi-node system. now _that_ will be some fun :) | 21:55 |
| *** harlowja has quit IRC | 21:55 | |
| Pavo | setting up a 2 node deployment right now with 3 VMs, 1x Deployer and 2 nodes | 21:56 |
| wznoinsk | rstarmer_: out of curiosity, numa topology filter is now enabled by default? | 21:57 |
| rstarmer_ | I can't find any reference to it, the defaults that I have found didn't seem to call it out, but given that there's no schduler_default_filters defined, I'm assuming that it is now part of the default list | 21:58 |
| rstarmer_ | actually, it doesn't look like it's a filter afte rall | 21:59 |
| *** matrohon has quit IRC | 22:00 | |
| wznoinsk | rstarmer_: right, the error seems to be coming from the driver | 22:00 |
| rstarmer_ | Retry AZ Ram Disk compute Compute Cap Image Server-anti pro affinit are listed as filters, it comes out of the filter, is the only choice so it's the top weight, and then is selected and locked, and then the scheduler throws up a "Require both..." error | 22:00 |
| rstarmer_ | nova.virt.hardware process | 22:01 |
| sdake | how do you define that numa configuration option | 22:01 |
| rstarmer_ | so perhaps the virt_type not being set doesn't have an appropriate default? | 22:01 |
| *** mark-casey has quit IRC | 22:01 | |
| rstarmer_ | sdake: I don't know... | 22:01 |
| wznoinsk | sdake: via nova flavor-key | 22:02 |
| wznoinsk | https://specs.openstack.org/openstack/nova-specs/specs/juno/implemented/virt-driver-numa-placement.html | 22:02 |
| wznoinsk | and NUMATopologyFilter in filters list in nova.conf | 22:02 |
| wznoinsk | sdake: maybe a better written guide to numa - http://docs.openstack.org/developer/nova/testing/libvirt-numa.html | 22:03 |
| rstarmer_ | My 'resolution" appears to have been setting the virt_type under [libvirt] in nova.conf on nova_compute container | 22:04 |
| rstarmer_ | But to be clear, on second review, it's not the Filter that was causing issues, it's another part of the scheduler | 22:04 |
| *** fragatin_ has joined #openstack-kolla | 22:10 | |
| wznoinsk | rstarmer_: NUMA filter is not one of the default ones in nova.conf https://github.com/openstack/nova/blob/282c257aff6b53a1b6bb4b4b034a670c450d19d8/nova/conf/scheduler.py | 22:10 |
| *** jamesden_ has quit IRC | 22:10 | |
| wznoinsk | hardware.py couldn't read host_topology (or part of it) hence it was throwing that error, odd | 22:10 |
| rstarmer | yeah, wonder why setting qemu fixed it? double odd, but at least there's a fix | 22:13 |
| *** fragatina has quit IRC | 22:13 | |
| *** fragatin_ has quit IRC | 22:14 | |
| *** fragatina has joined #openstack-kolla | 22:15 | |
| *** haplo37__ has quit IRC | 22:15 | |
| openstackgerrit | Shaun Smekel proposed openstack/kolla: Add full support for fernet https://review.openstack.org/349366 | 22:20 |
| *** sdake has quit IRC | 22:20 | |
| *** daneyon has joined #openstack-kolla | 22:21 | |
| *** vhosakot has joined #openstack-kolla | 22:25 | |
| *** daneyon has quit IRC | 22:25 | |
| openstackgerrit | Shaun Smekel proposed openstack/kolla: Add full support for fernet https://review.openstack.org/349366 | 22:26 |
| openstackgerrit | Shaun Smekel proposed openstack/kolla: Add full support for fernet https://review.openstack.org/349366 | 22:29 |
| *** signed8bit is now known as signed8bit_Zzz | 22:34 | |
| *** sdake has joined #openstack-kolla | 22:35 | |
| sdake | 1800 cartridge | 22:41 |
| sdake | paperweight | 22:41 |
| * sdake growls profusely | 22:41 | |
| * sdake nerd rages some more | 22:44 | |
| openstackgerrit | Kevin Fox proposed openstack/kolla: Add jq https://review.openstack.org/355003 | 22:45 |
| sdake | smekel_ - say re https://blueprints.launchpad.net/kolla/+spec/third-party-plugin-support | 22:46 |
| *** signed8bit_Zzz is now known as signed8bit | 22:46 | |
| wirehead_ | so, it’s more cart-rage than cartridge. | 22:47 |
| sdake | smekel_ it is always good to check the bllueprint status to make sure other people aren't working on a peice of work already | 22:47 |
| * sdake does cartwheels | 22:47 | |
| *** mbound has joined #openstack-kolla | 22:47 | |
| sdake | smekel_ thats why we have those work item fields - to help distribute work amongst the community | 22:47 |
| sdake | smekel_ can yoou please get the work items there up to date with your reviews in the queue | 22:49 |
| *** MrDan has joined #openstack-kolla | 22:50 | |
| MrDan | hello | 22:50 |
| MrDan | i have an issue when I do a precheck, it fails on checking ansible version | 22:51 |
| sdake | MrDan are you using master or mitaka or liberty branch? | 22:51 |
| MrDan | master I think, default | 22:51 |
| sdake | are you doing an evaluation? | 22:51 |
| sdake | or do you intend to do development on kolla | 22:51 |
| MrDan | for my own fun for now | 22:52 |
| MrDan | and dev i think, not a dev person yet | 22:52 |
| sdake | ok - well master may or may not work based upon data just provided by rstarmer | 22:52 |
| MrDan | i am playing aroung openstack atm | 22:52 |
| *** mbound has quit IRC | 22:52 | |
| sdake | roger | 22:52 |
| sdake | which distro do you have | 22:52 |
| sdake | and type pip show ansible | 22:52 |
| MrDan | ubuntu 14.04 | 22:52 |
| sdake | sudo pip install -U ansible==2.1.0.0 | 22:52 |
| MrDan | it correctly said ansible 2.1.1.0 | 22:53 |
| *** vhosakot has quit IRC | 22:53 | |
| sdake | hrm | 22:53 |
| sdake | maybe our prechecks dont check for 2.1.1.0 | 22:53 |
| MrDan | the command runned by the playbook is 'ansible --version' | 22:53 |
| sdake | try 2.1.0.0 | 22:53 |
| sdake | same sudo command will do the job | 22:53 |
| MrDan | -U will not update to latest? | 22:54 |
| MrDan | that is 2.1.1. | 22:54 |
| MrDan | the code where it fails is the following failed_when: result.stdout | regex_replace('ansible (\\d+\\.\\d+\\.\\d+)', '\\1') | version_compare(ansible_version_min, '<') | 22:55 |
| sdake | it will update to what you specify | 22:55 |
| MrDan | ok | 22:55 |
| * sdake head hurts looking at that regex | 22:55 | |
| MrDan | haha, yes :D | 22:56 |
| MrDan | ok, did downgrade to 2.1.0 | 22:56 |
| sdake | 2.1.0.0 you mean? | 22:56 |
| MrDan | runned the script again... | 22:56 |
| MrDan | yes | 22:56 |
| sdake | ok i am off to bed | 22:57 |
| sdake | night folks | 22:57 |
| *** sdake has quit IRC | 22:57 | |
| MrDan | night, thanks for the help | 22:58 |
| MrDan | same thing | 22:59 |
| MrDan | fatal: [localhost -> localhost]: FAILED! => {"changed": false, "cmd": ["ansible", "--version"], "delta": "0:00:00.166055", "end": "2016-08-16 01:58:38.135598", "failed": true, "failed_when_result": true, "rc": 0, "start": "2016-08-16 01:58:37.969543", "stderr": "", "stdout": "ansible 2.1.0.0\n config file = \n configured module search path = Default w/o overrides", "stdout_lines": ["ansible | 22:59 |
| MrDan | 2.1.0.0", " config file = ", " configured module search path = Default w/o overrides"], "warnings": []} | 22:59 |
| MrDan | this is the error | 22:59 |
| MrDan | if I do 'ansible --version' i get the version nice and dandy, exit with 0 | 23:00 |
| *** kbaegis has joined #openstack-kolla | 23:01 | |
| *** michauds has quit IRC | 23:02 | |
| *** kbaegis has left #openstack-kolla | 23:04 | |
| *** UnSavvy has joined #openstack-kolla | 23:04 | |
| MrDan | i think it fails on this 'regex_replace('ansible (\\d+\\.\\d+\\.\\d+)', '\\1')' | 23:06 |
| *** UnSavvy has quit IRC | 23:23 | |
| *** diogogmt has quit IRC | 23:31 | |
| *** harlowja has joined #openstack-kolla | 23:36 | |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes: Adding Mariadb shutdown command to POD delete event https://review.openstack.org/355189 | 23:45 |
| *** MrDan has quit IRC | 23:49 | |
| *** zhurong has joined #openstack-kolla | 23:50 | |
| *** zhurong has quit IRC | 23:57 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!
