| *** manheim has joined #openstack-kolla | 00:02 | |
| *** japestinho has joined #openstack-kolla | 00:13 | |
| *** rwallner has joined #openstack-kolla | 00:19 | |
| openstackgerrit | Merged openstack/kolla-ansible master: Fix bootstrap-servers with invalid api_interface https://review.openstack.org/434940 | 00:21 |
|---|---|---|
| *** ddyer2 has joined #openstack-kolla | 00:21 | |
| *** ddyer2 has quit IRC | 00:22 | |
| *** ddyer has quit IRC | 00:23 | |
| *** caoyuan has joined #openstack-kolla | 00:25 | |
| *** ducttape_ has joined #openstack-kolla | 00:31 | |
| *** ducttape_ has quit IRC | 00:36 | |
| *** ddyer has joined #openstack-kolla | 00:36 | |
| *** caoyuan has quit IRC | 00:37 | |
| *** iniazi has joined #openstack-kolla | 00:39 | |
| *** iniazi_ has quit IRC | 00:42 | |
| *** tovin07_ has joined #openstack-kolla | 00:45 | |
| *** cuongnv has joined #openstack-kolla | 00:48 | |
| *** manheim has quit IRC | 00:57 | |
| openstackgerrit | Merged openstack/kolla-ansible master: Neutron: nova_metadata_ip property is deprecated https://review.openstack.org/478446 | 01:05 |
| *** goldyfruit has joined #openstack-kolla | 01:09 | |
| *** huzhengchuan has joined #openstack-kolla | 01:29 | |
| *** kevzha01 has joined #openstack-kolla | 01:31 | |
| *** kevzha01 is now known as kevinz | 01:33 | |
| *** tovin07__ has joined #openstack-kolla | 01:37 | |
| *** cuongnv_ has joined #openstack-kolla | 01:37 | |
| *** cuongnv has quit IRC | 01:39 | |
| *** tovin07_ has quit IRC | 01:39 | |
| *** caoyuan has joined #openstack-kolla | 01:41 | |
| *** goldyfruit has quit IRC | 01:50 | |
| *** manheim has joined #openstack-kolla | 02:01 | |
| openstackgerrit | Noboru Iwamatsu proposed openstack/kolla master: Add designate dashboard to horizon ubuntu/binary https://review.openstack.org/480325 | 02:01 |
| *** manheim has quit IRC | 02:05 | |
| *** Pavo has quit IRC | 02:09 | |
| *** cuongnv_ is now known as cuongnv | 02:12 | |
| *** zhurong has joined #openstack-kolla | 02:12 | |
| openstackgerrit | jangseon ryu proposed openstack/kolla-ansible master: The external_ceph_cephx_enabled option is duplicated in the global.yml. So one of them was deleted. https://review.openstack.org/480328 | 02:31 |
| *** ducttape_ has joined #openstack-kolla | 02:33 | |
| *** ducttape_ has quit IRC | 02:37 | |
| openstackgerrit | jangseon ryu proposed openstack/kolla-ansible master: The external_ceph_cephx_enabled option is duplicated in the global.yml. So one of them was deleted. https://review.openstack.org/480329 | 02:39 |
| *** japestinho has quit IRC | 02:43 | |
| *** zhubingbing has joined #openstack-kolla | 02:44 | |
| *** dixiaoli has joined #openstack-kolla | 02:52 | |
| *** ipsecguy has quit IRC | 03:07 | |
| openstackgerrit | zhubingbing proposed openstack/kolla-ansible master: Fix field data is too large https://review.openstack.org/476367 | 03:07 |
| *** ipsecguy has joined #openstack-kolla | 03:11 | |
| *** mdnadeem has joined #openstack-kolla | 03:40 | |
| openstackgerrit | shaofeng cheng proposed openstack/kolla-ansible master: Add block_device_allocate_retries config option https://review.openstack.org/479119 | 03:47 |
| zhubingbing | Hi guys | 04:03 |
| spsurya | good morning | 04:04 |
| spsurya | sup zhubingbing | 04:04 |
| zhubingbing | hi spusurya | 04:05 |
| zhubingbing | ;) | 04:05 |
| masber | hi, I need to check if the sr-iov drivers are loaded to the database, I checked the /etc/kolla/passwords.yml but can't find the password for mariadb? | 04:07 |
| duonghq | hi zhubingbing spsurya | 04:08 |
| zhubingbing | masber | 04:08 |
| zhubingbing | u can | 04:08 |
| *** lpetrut has joined #openstack-kolla | 04:09 | |
| zhubingbing | U can cat /etc/password.yml | grep mariadb | 04:09 |
| masber | zhubingbing, cat /etc/kolla/passwords.yml | grep mariadb shows nothing | 04:11 |
| *** kolla-slack1 has quit IRC | 04:11 | |
| *** kolla-slack has joined #openstack-kolla | 04:11 | |
| masber | I am checking the deployment node where I run my kolla-ansible commands, is that the right place? | 04:12 |
| zhubingbing | kolla-genpwd | 04:12 |
| *** spsurya has quit IRC | 04:12 | |
| *** lpetrut has quit IRC | 04:12 | |
| *** spsurya has joined #openstack-kolla | 04:12 | |
| zhubingbing | run this command kolla-genpwd | 04:12 |
| masber | done | 04:13 |
| masber | still same | 04:14 |
| zhubingbing | [root@control01 ~]# cat /etc/kolla/passwords.yml | grep database | 04:19 |
| zhubingbing | database_password: | 04:19 |
| zhubingbing | sorry | 04:19 |
| *** lpetrut has joined #openstack-kolla | 04:19 | |
| zhubingbing | databse_password it is mariadb password | 04:19 |
| zhubingbing | sorry | 04:19 |
| masber | thank you! | 04:20 |
| masber | found it | 04:20 |
| zhubingbing | spusurya | 04:20 |
| zhubingbing | Can u review it ? https://review.openstack.org/#/c/476367/ | 04:21 |
| *** yuanying has joined #openstack-kolla | 04:21 | |
| *** yuanying_ has quit IRC | 04:21 | |
| spsurya | zhubingbing: sure I will | 04:29 |
| zhubingbing | thanks | 04:29 |
| *** ducttape_ has joined #openstack-kolla | 04:31 | |
| *** janki has joined #openstack-kolla | 04:31 | |
| *** ducttape_ has quit IRC | 04:35 | |
| spsurya | zhubingbing: done | 04:35 |
| spsurya | sup duonghq | 04:36 |
| zhubingbing | 3q | 04:41 |
| *** jamesbenson has joined #openstack-kolla | 04:50 | |
| *** yangyapeng has joined #openstack-kolla | 04:50 | |
| openstackgerrit | jangseon ryu proposed openstack/kolla-ansible master: Removing deplicated option from global.yml file. https://review.openstack.org/480329 | 04:52 |
| *** kevinz has quit IRC | 04:53 | |
| *** yangyapeng has quit IRC | 04:57 | |
| *** yangyapeng has joined #openstack-kolla | 04:58 | |
| *** dixiaoli has quit IRC | 05:00 | |
| *** lpetrut has quit IRC | 05:02 | |
| *** yangyapeng has quit IRC | 05:05 | |
| *** yangyapeng has joined #openstack-kolla | 05:06 | |
| *** yangyape_ has joined #openstack-kolla | 05:07 | |
| *** prateek has joined #openstack-kolla | 05:07 | |
| *** jamesbenson has quit IRC | 05:09 | |
| *** jamesbenson has joined #openstack-kolla | 05:10 | |
| *** yangyapeng has quit IRC | 05:11 | |
| *** ducttape_ has joined #openstack-kolla | 05:11 | |
| openstackgerrit | Bertrand Lallau proposed openstack/kolla-ansible master: Fix outdated barbican-api-paste.ini file https://review.openstack.org/469939 | 05:14 |
| *** jamesbenson has quit IRC | 05:14 | |
| *** ducttape_ has quit IRC | 05:16 | |
| *** yangyape_ has quit IRC | 05:20 | |
| *** yangyapeng has joined #openstack-kolla | 05:21 | |
| *** skramaja has joined #openstack-kolla | 05:21 | |
| *** lpetrut has joined #openstack-kolla | 05:25 | |
| *** dixiaoli has joined #openstack-kolla | 05:27 | |
| masber | hi, is Derek Ditch around? | 05:37 |
| masber | I was wondering whether I could ask him a couple of questions regarding this document he wrote https://review.openstack.org/gitweb?p=openstack/kolla.git;a=commitdiff;h=840b73181eaaae2e29565f245ccf25b046da89be | 05:37 |
| *** yangyapeng has quit IRC | 05:40 | |
| *** yangyapeng has joined #openstack-kolla | 05:41 | |
| *** jamesbenson has joined #openstack-kolla | 05:55 | |
| *** jamesbenson has quit IRC | 05:59 | |
| *** Teck72 has joined #openstack-kolla | 06:01 | |
| *** Teck7 has quit IRC | 06:05 | |
| *** serlex has joined #openstack-kolla | 06:21 | |
| *** lpetrut has quit IRC | 06:27 | |
| *** magicboiz has joined #openstack-kolla | 06:32 | |
| *** jamesbenson has joined #openstack-kolla | 06:36 | |
| *** magicboiz has quit IRC | 06:37 | |
| openstackgerrit | shaofeng cheng proposed openstack/kolla-ansible master: Change destroy confirmation string https://review.openstack.org/480367 | 06:39 |
| *** jamesbenson has quit IRC | 06:41 | |
| *** lpetrut has joined #openstack-kolla | 06:50 | |
| openstackgerrit | Juan Badia Payno proposed openstack/kolla master: Changed fluentd repository for CentOS https://review.openstack.org/467219 | 06:53 |
| *** rwsu has joined #openstack-kolla | 07:00 | |
| *** mgoddard_ has joined #openstack-kolla | 07:09 | |
| *** ducttape_ has joined #openstack-kolla | 07:12 | |
| *** shardy has joined #openstack-kolla | 07:16 | |
| *** lpetrut has quit IRC | 07:16 | |
| *** caoyuan has quit IRC | 07:16 | |
| *** ducttape_ has quit IRC | 07:17 | |
| *** mgoddard_ has quit IRC | 07:18 | |
| *** lpetrut has joined #openstack-kolla | 07:23 | |
| *** yuanying has quit IRC | 07:31 | |
| *** lpetrut has quit IRC | 07:32 | |
| *** yuanying has joined #openstack-kolla | 07:37 | |
| *** oanson has quit IRC | 07:38 | |
| *** oanson has joined #openstack-kolla | 07:38 | |
| *** egonzalez has joined #openstack-kolla | 07:39 | |
| *** reidrac has joined #openstack-kolla | 07:42 | |
| *** serlex has quit IRC | 07:56 | |
| *** manheim has joined #openstack-kolla | 08:06 | |
| openstackgerrit | Juan J. Martínez proposed openstack/kolla-ansible master: Upgrade dev environment from docker-py to docker https://review.openstack.org/480400 | 08:08 |
| reidrac | re: https://review.openstack.org/480400 I had problems to push images in the dev environment using master and docker-py; installing docker python package fixes the issue | 08:10 |
| *** manheim has quit IRC | 08:15 | |
| *** mgoddard_ has joined #openstack-kolla | 08:18 | |
| *** shardy is now known as shtab | 08:27 | |
| *** shtab is now known as shardy | 08:27 | |
| *** gfidente has joined #openstack-kolla | 08:34 | |
| *** gfidente has quit IRC | 08:34 | |
| *** gfidente has joined #openstack-kolla | 08:34 | |
| openstackgerrit | Martin André proposed openstack/kolla master: Add new --skip-existing option to kolla-build https://review.openstack.org/479881 | 08:38 |
| openstackgerrit | Martin André proposed openstack/kolla master: Fix mock order https://review.openstack.org/480434 | 08:38 |
| openstackgerrit | Bertrand Lallau proposed openstack/kolla-ansible master: WIP remove docker-py https://review.openstack.org/480438 | 08:39 |
| *** yangyape_ has joined #openstack-kolla | 08:42 | |
| *** magicboiz has joined #openstack-kolla | 08:43 | |
| *** yangyape_ has quit IRC | 08:46 | |
| *** magicboiz has quit IRC | 08:46 | |
| *** yangyapeng has quit IRC | 08:46 | |
| *** yangyapeng has joined #openstack-kolla | 08:48 | |
| *** magicboiz has joined #openstack-kolla | 08:53 | |
| *** tvignaud has quit IRC | 08:57 | |
| *** yangyapeng has quit IRC | 08:59 | |
| *** yangyapeng has joined #openstack-kolla | 09:00 | |
| *** jamesbenson has joined #openstack-kolla | 09:01 | |
| *** yangyapeng has quit IRC | 09:04 | |
| *** yangyapeng has joined #openstack-kolla | 09:05 | |
| *** magicboiz has quit IRC | 09:05 | |
| *** jamesbenson has quit IRC | 09:06 | |
| Seyong | keystone is not installing on CentOS with this error: failed for volume "keystone-fernet-keys" : secrets "keystone-fernet-keys" not found | 09:08 |
| Seyong | but in Ubuntu works | 09:09 |
| Seyong | do you have any idea? | 09:09 |
| *** tvignaud has joined #openstack-kolla | 09:09 | |
| *** ducttape_ has joined #openstack-kolla | 09:13 | |
| egonzalez | Seyong, by installing means build images or deploy containers? | 09:15 |
| *** ducttape_ has quit IRC | 09:17 | |
| *** sambetts|afk is now known as sambetts | 09:18 | |
| *** omenv has quit IRC | 09:22 | |
| *** jamesbenson has joined #openstack-kolla | 09:22 | |
| *** Pavo has joined #openstack-kolla | 09:24 | |
| *** jamesbenson has quit IRC | 09:26 | |
| *** magicboiz has joined #openstack-kolla | 09:33 | |
| *** magicboiz has quit IRC | 09:37 | |
| *** magicboiz has joined #openstack-kolla | 09:37 | |
| *** dixiaoli has quit IRC | 09:38 | |
| *** jamesbenson has joined #openstack-kolla | 09:43 | |
| *** jamesbenson has quit IRC | 09:47 | |
| *** rwallner has quit IRC | 09:54 | |
| *** rwallner has joined #openstack-kolla | 09:54 | |
| *** yangyapeng has quit IRC | 09:55 | |
| *** yangyapeng has joined #openstack-kolla | 09:55 | |
| *** manheim has joined #openstack-kolla | 09:55 | |
| *** serlex has joined #openstack-kolla | 09:56 | |
| *** rwallner has quit IRC | 09:59 | |
| *** manheim has quit IRC | 10:00 | |
| reidrac | egonzalez: I get those "unregister_netdevice: waiting for lo to become free. Usage count = 1" quite frequently on the operator when I'm building the images | 10:02 |
| reidrac | it doesn't seem to have any effect on CentOS, but in SUSE sometimes kills container connectivity as described in https://github.com/moby/moby/issues/5618 | 10:02 |
| reidrac | when you saw those, was it in other nodes or just building images as in my case? | 10:03 |
| *** jamesbenson has joined #openstack-kolla | 10:04 | |
| *** omenv has joined #openstack-kolla | 10:04 | |
| *** Pavo has quit IRC | 10:05 | |
| *** jamesbenson has quit IRC | 10:08 | |
| *** manheim has joined #openstack-kolla | 10:10 | |
| openstackgerrit | Merged openstack/kolla master: Add IPV6 radvd package to neutron-l3-agent https://review.openstack.org/479788 | 10:11 |
| *** omenvl has joined #openstack-kolla | 10:11 | |
| openstackgerrit | Eduardo Gonzalez proposed openstack/kolla-ansible master: WIP: Change zun storage to etcd https://review.openstack.org/480497 | 10:11 |
| *** yangyapeng has quit IRC | 10:12 | |
| *** manheim has quit IRC | 10:12 | |
| *** manheim has joined #openstack-kolla | 10:12 | |
| *** cuongnv has quit IRC | 10:14 | |
| *** omenv has quit IRC | 10:14 | |
| *** bmace has quit IRC | 10:19 | |
| *** bmace has joined #openstack-kolla | 10:19 | |
| *** schwicht has quit IRC | 10:21 | |
| *** jamesbenson has joined #openstack-kolla | 10:24 | |
| *** schwicht has joined #openstack-kolla | 10:25 | |
| *** tovin07__ has quit IRC | 10:28 | |
| *** jamesbenson has quit IRC | 10:29 | |
| openstackgerrit | Merged openstack/kolla master: Add designate dashboard to horizon ubuntu/binary https://review.openstack.org/480325 | 10:39 |
| *** zhurong has quit IRC | 10:45 | |
| *** pbourke has quit IRC | 10:48 | |
| *** pbourke has joined #openstack-kolla | 10:49 | |
| *** rwallner has joined #openstack-kolla | 10:56 | |
| *** schwicht has quit IRC | 11:01 | |
| *** shardy is now known as shardy_lunch | 11:08 | |
| pbourke | egonzalez: hey eduardo, wondering have you done an upgrade of openvswitch in kolla recently? | 11:27 |
| egonzalez | pbourke, yes | 11:33 |
| openstackgerrit | sean mooney proposed openstack/kolla master: introduces support for the OVS DPDK dataplane https://review.openstack.org/342354 | 11:35 |
| *** schwicht has joined #openstack-kolla | 11:39 | |
| openstackgerrit | sean mooney proposed openstack/kolla master: enable the kolla build system to use out of tree docker files https://review.openstack.org/466132 | 11:58 |
| *** yangyapeng has joined #openstack-kolla | 12:02 | |
| *** yangyapeng has quit IRC | 12:07 | |
| *** caoyuan has joined #openstack-kolla | 12:09 | |
| *** goldyfruit has joined #openstack-kolla | 12:12 | |
| *** goldyfruit has quit IRC | 12:12 | |
| *** bmace has quit IRC | 12:16 | |
| *** bmace has joined #openstack-kolla | 12:16 | |
| *** schwicht has quit IRC | 12:20 | |
| *** hrw has quit IRC | 12:26 | |
| *** dims has quit IRC | 12:29 | |
| pbourke | egonzalez: im finding its dropping the bridges when the new container comes up | 12:30 |
| pbourke | egonzalez: like br-tun, br-ex, etc. Not sure if you've seen anything like that | 12:30 |
| egonzalez | pbourke, all bridges? | 12:31 |
| pbourke | egonzalez: yeah | 12:31 |
| egonzalez | let me try | 12:31 |
| pbourke | egonzalez: from what I can see that state is kept in /etc/openvswitch/conf.db in the openvswitch_db container. That file isn't under a volume so Im unsure how its working for others | 12:31 |
| *** hrw has joined #openstack-kolla | 12:32 | |
| egonzalez | actually, does mariadb upgrade worked for you? from ocata->master fails to me | 12:32 |
| pbourke | egonzalez: hmm have not tested that yet | 12:33 |
| *** rhallisey has joined #openstack-kolla | 12:33 | |
| *** ansmith has joined #openstack-kolla | 12:34 | |
| *** mdnadeem has quit IRC | 12:35 | |
| *** ducttape_ has joined #openstack-kolla | 12:36 | |
| *** yangyapeng has joined #openstack-kolla | 12:37 | |
| *** shardy_lunch is now known as shardy | 12:41 | |
| *** ducttape_ has quit IRC | 12:41 | |
| egonzalez | pbourke, conf.db should be at /var/lib/openvswitch which is a volume https://github.com/openstack/kolla/blob/master/docker/openvswitch/openvswitch-db-server/start_ovsdb_server.sh#L29 | 12:42 |
| egonzalez | check this change from Dec 2016 https://github.com/openstack/kolla/commit/f62b16f8a447b7c90da3844720a200fb4d3ef5d5 | 12:42 |
| pbourke | egonzalez: great find let me check that thanks | 12:42 |
| *** dims has joined #openstack-kolla | 12:44 | |
| *** omenvl has quit IRC | 12:48 | |
| *** daidv_ has joined #openstack-kolla | 12:49 | |
| *** mattmceuen has quit IRC | 12:51 | |
| *** caoyuan has quit IRC | 12:53 | |
| *** daidv_ has left #openstack-kolla | 12:56 | |
| *** skramaja has quit IRC | 12:59 | |
| *** lucasxu has joined #openstack-kolla | 13:00 | |
| *** goldyfruit has joined #openstack-kolla | 13:01 | |
| openstackgerrit | Andy Smith proposed openstack/kolla-ansible master: Add qdrouterd role for messaging infrastructure component https://review.openstack.org/469462 | 13:03 |
| *** schwicht has joined #openstack-kolla | 13:04 | |
| openstackgerrit | Merged openstack/kolla-ansible master: Fix outdated barbican-api-paste.ini file https://review.openstack.org/469939 | 13:04 |
| *** keico has joined #openstack-kolla | 13:04 | |
| openstackgerrit | Merged openstack/kolla-ansible master: Removing deplicated option from global.yml file. https://review.openstack.org/480329 | 13:09 |
| *** omenvl has joined #openstack-kolla | 13:09 | |
| *** denaitre has joined #openstack-kolla | 13:10 | |
| *** omenvlad has joined #openstack-kolla | 13:10 | |
| *** omenvl has quit IRC | 13:13 | |
| *** zhubingbing has quit IRC | 13:15 | |
| keico | Hello. I'm trying to deploy an Openstack (ocata) cloud using Kolla-ansible (version 4.0.2). Currently, on the control01 node, my mariadb Docker container fails to start. | 13:20 |
| keico | The following are the errors I received from /var/log/kolla/mariadb/mariadb.log: | 13:20 |
| *** caoyuan has joined #openstack-kolla | 13:21 | |
| keico | [ERROR] Can't open the mysql.plugin table. Please run mysql_upgrade to create it. [Warning] Failed to load slave replication state from table mysql.gtid_slave_pos: 1146: Table 'mysql.gtid_slave_pos' doesn't exist [ERROR] Can't open and lock privilege tables: Table 'mysql.servers' doesn't exist [Note] Server socket created on IP: 'x.x.x.x'. Fatal error: Can't open and lock privilege tables: Table 'mysql.user' doesn't exist | 13:21 |
| keico | mysqld_safe mysqld from pid file /var/lib/mysql/mariadb.pid ended | 13:22 |
| keico | Does anyone know what causes these errors and how to resolve it? | 13:22 |
| *** jtriley has joined #openstack-kolla | 13:25 | |
| reidrac | looks like maridb db wasn't bootstrapped | 13:25 |
| keico | So, I followed this instruction doc as my starting point: https://docs.openstack.org/project-deploy-guide/kolla-ansible/ocata/multinode.html. I was able to build the kolla Docker images with tag 4.0.2 and pushed them to my private docker registry. So, @reidrac, do you know how to bootstrap the mariadb database? | 13:30 |
| reidrac | kolla-ansible deploy should do it | 13:31 |
| reidrac | it it didn't happen, you should have errors | 13:31 |
| keico | This is the errors I received, after running: kolla-ansible deploy -i ./multinode command: | 13:33 |
| keico | TASK [mariadb : Waiting for MariaDB service to be ready] **************************************************************************************************** FAILED - RETRYING: Waiting for MariaDB service to be ready (10 retries left). FAILED - RETRYING: Waiting for MariaDB service to be ready (9 retries left). FAILED - RETRYING: Waiting for MariaDB service to be ready (8 retries left). FAILED - RETRYING: Waiting for MariaDB | 13:33 |
| reidrac | so mariadb didn't start | 13:34 |
| keico | FAILED - RETRYING: Waiting for MariaDB service to be ready (1 retries left). | 13:34 |
| keico | fatal: [control01]: FAILED! => {"attempts": 10, "changed": false, "elapsed": 60, "failed": true, "msg": "Timeout when waiting for search string MariaDB in x.x.x.x:3306"} to retry, use: --limit @/usr/share/kolla-ansible/ansible/site.retry | 13:34 |
| reidrac | go to control01 and check docker logs mariadb | 13:34 |
| sdake | morning pbourke eglute_s | 13:34 |
| sdake | egonzalez :) | 13:34 |
| keico | yes. That's exactly what I did. | 13:34 |
| reidrac | (check with docker ps that it has the expected containers) | 13:34 |
| egonzalez | morning sdake | 13:34 |
| pbourke | sdake: o/ | 13:34 |
| reidrac | and it complains about the tables not being there | 13:35 |
| keico | It looks like on the control01 node, the mariadb container keeps restarting. | 13:35 |
| reidrac | keico: yep, take a look at /var/lib/docker/volumes/kolla_logs/_data/mariadb/mariadb.log | 13:35 |
| reidrac | IIRC there's a bootstrap_mariadb container; does it showin docker ps -a? | 13:37 |
| keico | These are the last messages I see from this log: | 13:38 |
| keico | ERROR] Can't open the mysql.plugin table. Please run mysql_upgrade to create it. | 13:38 |
| keico | [Warning] Failed to load slave replication state from table mysql.gtid_slave_pos: 1146: Table 'mysql.gtid_slave_pos' doesn't exist | 13:38 |
| sdake | keico first off - if a deploy craters, you ahve to start fresh | 13:38 |
| keico | [ERROR] Can't open and lock privilege tables: Table 'mysql.servers' doesn't exist | 13:38 |
| reidrac | that sounds like a consequence of the boostrap failing | 13:38 |
| keico | [Note] Server socket created on IP: 'x.x.x.x'. | 13:39 |
| keico | [ERROR] Fatal error: Can't open and lock privilege tables: Table 'mysql.user' doesn't exist | 13:39 |
| sdake | keico ansibel has no way of knowing if the deployment you have was valid or wasn't | 13:39 |
| keico | mysqld_safe mysqld from pid file /var/lib/mysql/mariadb.pid ended | 13:39 |
| sdake | keico what i'd suggest is a) kolla-ansible destroy | 13:39 |
| sdake | b) kolla-ansible prechecks | 13:39 |
| sdake | fix whatever b turn up | 13:39 |
| sdake | kolla-ansible deploy | 13:39 |
| keico | wait a min. So, there's a bootstrap_mariadb container? Is it supposed to run on the control01 node as well? | 13:40 |
| sdake | keico there are a bunch off bootstrap_* containers | 13:40 |
| sdake | they are run per service | 13:40 |
| sdake | mariadb has one | 13:40 |
| egonzalez | keico, bootstrap_mariadb runs in the first mariadb node in inventory | 13:40 |
| egonzalez | check logs with docker logs bootstrap_mariadb | 13:41 |
| sdake | the bootstrap service doe the initial setup of the service (create databases, create users, create keystone creds, etc) | 13:41 |
| keico | Got it. | 13:41 |
| *** zhurong has joined #openstack-kolla | 13:44 | |
| keico | So, where can I find information of the bootstrap_* containers? In the inventory/multinode config? In other words, where can I find a list of these bootstrap containers? | 13:45 |
| pbourke | Jeffrey4l: hey Jeffrey, don't quite understand your comment on https://review.openstack.org/#/c/448654/6/ansible/roles/glance/tasks/deploy.yml | 13:46 |
| pbourke | Jeffrey4l: can you elaborate? | 13:46 |
| Jeffrey4l | pbourke, this should not related to glance_file_datadir_volume variable. | 13:46 |
| Jeffrey4l | if i use glance_file_datadir_volume=/glance/image, should we use single glance api host? | 13:47 |
| reidrac | keico: look in ansible/roles/mariadb/tasks/* for example for mariadb | 13:48 |
| pbourke | Jeffrey4l: the issue is that may be NFS mounted | 13:48 |
| reidrac | those bootstrap containers are destroyed once they do their job | 13:48 |
| pbourke | Jeffrey4l: in which case it should use all api hosts | 13:48 |
| Jeffrey4l | ok. you are correct. | 13:48 |
| Jeffrey4l | thanks. | 13:48 |
| reidrac | keico: I was trying to go trhough a normal troubleshooting sequence; if the bootstrap container doesn't provide a clue, kolla_toolbox can be to blame too | 13:49 |
| emccormick | anyone tried using the policy overwrite in horizon? I'm trying to replace keystone_policy.json and the reconfigure goes boom. | 13:50 |
| emccormick | fatal: [v69ctrl01]: FAILED! => {"failed": true, "msg": "the field 'args' has an invalid value, which appears to include a variable that is undefined. The error was: dict object has no element 0\n\nThe error appears to have been in '/usr/share/kolla-ansible/ansible/roles/horizon/tasks/config.yml': line 87, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears | 13:50 |
| emccormick | to be:\n\n\n- name: Copying over existing policy.json\n ^ here\n"} | 13:50 |
| emccormick | I looked at the code, but my Ansible-fu is not quite strong enough to figure out what's happening there. | 13:50 |
| emccormick | all I did here was drop a keystone_policy.json in /etc/kolla/config/horizon/ | 13:51 |
| keico | @reidrac, Thanks so much for your help. I will run kolla-ansilbe destroy and re-deploy the cluter. | 13:51 |
| Jeffrey4l | seems there a bug here. trying it. emccormick | 13:52 |
| reidrac | keico: you're welcome | 13:52 |
| emccormick | Thanks. This is latest Ocata fyi | 13:53 |
| *** rwallner has quit IRC | 13:53 | |
| egonzalez | emccormick, cany you try commenting this line https://github.com/openstack/kolla-ansible/blob/stable/ocata/ansible/roles/horizon/tasks/config.yml#L97 ? | 13:54 |
| Jeffrey4l | reoger. | 13:54 |
| sdake | keico to find a list of bootstrap containers - you willh ave to make one by readin gthe ansible code and searching for _bootstrap) | 13:55 |
| emccormick | doing it now | 13:55 |
| egonzalez | emccormick, i think that is the issue because is item.item.enabled is undefined | 13:56 |
| openstackgerrit | Bertrand Lallau proposed openstack/kolla-ansible master: Avoid launching "local" actions too many times https://review.openstack.org/480256 | 13:56 |
| emccormick | getting in there now | 13:57 |
| Jeffrey4l | egonzalez, check this http://paste.openstack.org/show/614458/ emccormick | 13:57 |
| * Jeffrey4l gonna file a bug and fix this issue. | 14:00 | |
| egonzalez | Jeffrey4l, good catch | 14:00 |
| Jeffrey4l | ;) | 14:01 |
| *** huzhengchuan has quit IRC | 14:01 | |
| *** reidrac_ has joined #openstack-kolla | 14:02 | |
| egonzalez | does anybody understand how mariadb upgrade makes the order to upgrade? is broken in master ATM | 14:02 |
| egonzalez | I don't fully understand the lookup_cluster.yml | 14:02 |
| egonzalez | https://bugs.launchpad.net/kolla-ansible/+bug/1692507 | 14:02 |
| openstack | Launchpad bug 1692507 in kolla-ansible "optimize reconfiguration breaks mariadb upgrade" [Critical,New] | 14:02 |
| Jeffrey4l | reconfigure mariadb is bust ;( | 14:02 |
| *** schwicht has quit IRC | 14:02 | |
| Jeffrey4l | it stop all mariadb instance right now | 14:03 |
| *** ducttape_ has joined #openstack-kolla | 14:03 | |
| *** reidrac has quit IRC | 14:04 | |
| emccormick | Jeffrey4l do I leave item.item.enabled commented out? | 14:06 |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla-ansible master: Use proper variable in horizon copy policy task https://review.openstack.org/480598 | 14:07 |
| emccormick | fyi just with that commented out it bombed still | 14:07 |
| *** schwicht has joined #openstack-kolla | 14:07 | |
| Jeffrey4l | emccormick, check above patch ^^ | 14:08 |
| emccormick | right I got that. was talking about egonzalez suggestion. Just asking if both should be applied or just yours | 14:09 |
| *** zhurong has quit IRC | 14:09 | |
| egonzalez | emccormick, apply Jeffrey4l's patch | 14:09 |
| egonzalez | i was wrong | 14:09 |
| emccormick | looks like it worked | 14:11 |
| emccormick | Thanks guys. You want me to file an official bug for it? | 14:12 |
| egonzalez | emccormick, already opened https://bugs.launchpad.net/kolla-ansible/+bug/1702486 | 14:13 |
| openstack | Launchpad bug 1702486 in kolla-ansible "horizon policy file won't work" [Undecided,In progress] - Assigned to Jeffrey Zhang (jeffrey4l) | 14:13 |
| emccormick | Thanks you sir. Much appreciated. | 14:15 |
| *** prateek has quit IRC | 14:18 | |
| *** schwicht has quit IRC | 14:24 | |
| openstackgerrit | Bertrand Lallau proposed openstack/kolla-ansible master: Fix keystone-paste.ini custom file management https://review.openstack.org/480608 | 14:25 |
| *** yangyape_ has joined #openstack-kolla | 14:29 | |
| *** schwicht has joined #openstack-kolla | 14:30 | |
| *** yangyapeng has quit IRC | 14:32 | |
| openstackgerrit | Bertrand Lallau proposed openstack/kolla-ansible master: Fix keystone-paste.ini custom file management https://review.openstack.org/480608 | 14:32 |
| *** bmace has quit IRC | 14:33 | |
| *** jamesbenson has joined #openstack-kolla | 14:34 | |
| *** prateek has joined #openstack-kolla | 14:34 | |
| *** jamesbenson has quit IRC | 14:39 | |
| *** jamesbenson has joined #openstack-kolla | 14:40 | |
| *** huzhengchuan has joined #openstack-kolla | 14:41 | |
| *** schwicht has quit IRC | 14:42 | |
| *** reidrac has joined #openstack-kolla | 14:43 | |
| openstackgerrit | Bertrand Lallau proposed openstack/kolla-ansible master: Fix tools/cleanup-images script error https://review.openstack.org/480628 | 14:45 |
| *** reidrac_ has quit IRC | 14:46 | |
| *** kiennt has joined #openstack-kolla | 14:53 | |
| *** caoyuan has quit IRC | 14:53 | |
| *** schwicht has joined #openstack-kolla | 14:54 | |
| *** mmehan has joined #openstack-kolla | 14:58 | |
| *** rwallner has joined #openstack-kolla | 14:58 | |
| *** blallau has joined #openstack-kolla | 15:00 | |
| *** lucasxu has quit IRC | 15:00 | |
| *** omenvlad has quit IRC | 15:01 | |
| *** rwallner has quit IRC | 15:02 | |
| openstackgerrit | Eduardo Gonzalez proposed openstack/kolla-ansible master: Add designate dashboard to horizon https://review.openstack.org/478158 | 15:04 |
| *** iniazi_ has joined #openstack-kolla | 15:05 | |
| *** blallau has quit IRC | 15:07 | |
| *** iniazi has quit IRC | 15:08 | |
| egonzalez | do we have meeting today? there is no item in the agenda | 15:11 |
| *** duonghq has quit IRC | 15:13 | |
| *** duonghq has joined #openstack-kolla | 15:14 | |
| duonghq | enevning guys | 15:14 |
| *** duonghq_ has joined #openstack-kolla | 15:14 | |
| openstackgerrit | jangseon ryu proposed openstack/kolla-ansible master: Remove wrong command in "ceph in kolla" guide. https://review.openstack.org/480640 | 15:14 |
| *** fooliouno has joined #openstack-kolla | 15:15 | |
| openstackgerrit | Merged openstack/kolla-ansible master: Use proper variable in horizon copy policy task https://review.openstack.org/480598 | 15:17 |
| spsurya | egonzalez: inc0 can tell better | 15:18 |
| spsurya | duonghq: evening | 15:18 |
| kolla-slack | <inc0> We can cancel if you feel like it | 15:18 |
| kolla-slack | <inc0> US had long weekend | 15:19 |
| kolla-slack | <inc0> So I expect day to be quiet | 15:19 |
| kolla-slack | <inc0> I'll send mail. | 15:19 |
| spsurya | inc0: roger | 15:20 |
| duonghq | do we still have meeting today? | 15:22 |
| kolla-slack | <inc0> Duong no I just sent cancel mail | 15:22 |
| duonghq | inc0, roger | 15:23 |
| *** schwicht has quit IRC | 15:23 | |
| kolla-slack | <inc0> Jeffrey4l and other Asia folk, no need to stay awake. Talk to you all later | 15:24 |
| duonghq | inc0, cya | 15:26 |
| *** schwicht has joined #openstack-kolla | 15:26 | |
| *** janki has quit IRC | 15:31 | |
| *** duonghq has quit IRC | 15:32 | |
| *** lucasxu has joined #openstack-kolla | 15:35 | |
| Jeffrey4l | roger. thanks. | 15:36 |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla-ansible stable/ocata: Use proper variable in horizon copy policy task https://review.openstack.org/480649 | 15:37 |
| *** dixiaoli has joined #openstack-kolla | 15:39 | |
| *** dixiaoli has quit IRC | 15:44 | |
| *** prateek has quit IRC | 15:45 | |
| *** rwallner has joined #openstack-kolla | 15:48 | |
| *** schwicht has quit IRC | 15:52 | |
| *** vhosakot has joined #openstack-kolla | 15:55 | |
| *** schwicht has joined #openstack-kolla | 15:56 | |
| *** jascott1 has joined #openstack-kolla | 16:01 | |
| vhosakot | no meeting today? | 16:02 |
| kfox1111 | morning. | 16:02 |
| inc0 | good morning | 16:03 |
| inc0 | no, no meeting today vhosakot | 16:03 |
| *** egonzalez has quit IRC | 16:03 | |
| vhosakot | ah ok cool, why no meeting? :) | 16:03 |
| inc0 | no agenda so I figured we'll let Asia and Europe go for a evening/night:) | 16:03 |
| vhosakot | cool cool :) | 16:03 |
| inc0 | how was Cisco live? | 16:04 |
| inc0 | what did you announce with Kolla? any more reading about it?:) | 16:04 |
| vhosakot | was great, did many Kolla sessions :) many hands-on. | 16:04 |
| inc0 | great news | 16:04 |
| vhosakot | wanted to get back to upgrade gates :) duonghq_ ping again :) | 16:05 |
| inc0 | vhosakot: I can help you with that | 16:05 |
| vhosakot | inc0: so, are we submitting any kolla talks for Sydney? If so, I'd love to be a co-speaker :) | 16:05 |
| *** coolsvap has joined #openstack-kolla | 16:06 | |
| *** schwicht has quit IRC | 16:06 | |
| inc0 | I think upgrade gate would make great talk on it;s own | 16:06 |
| vhosakot | inc0: yeah, I was thiknking about it too. | 16:06 |
| inc0 | I'm also thinkining of dockerhub publishing talk... maybe we could join these | 16:06 |
| vhosakot | inc0: cool, did you submit any talk? | 16:07 |
| inc0 | since it's all about gates and infra | 16:07 |
| vhosakot | inc0: any kolla hands-on workshops in Sydney? | 16:07 |
| kfox1111 | +1 more gates. :) | 16:07 |
| vhosakot | kfox1111: yeah, upgrade gates will be very useful. | 16:07 |
| kfox1111 | hep's finally winding down I think. so hoping I can get more time back now. | 16:08 |
| inc0 | great news kfox1111 | 16:08 |
| inc0 | we can kick off orch work with your benevolent oversight;) | 16:08 |
| vhosakot | who's winding down from what? :) | 16:08 |
| kfox1111 | sorry. me, and hep=high energy physics | 16:09 |
| kfox1111 | I need to dust off the upgrade and reboot gates. | 16:09 |
| kfox1111 | and the logging ps. | 16:10 |
| inc0 | vhosakot kfox1111 maybe you can work together? as most of infra-related issues will be generally common | 16:10 |
| inc0 | and outside of infra-related issues it's mostly kolla-ansible upgrade;) | 16:11 |
| kfox1111 | +1 for shared infra. though I think I've pretty much got all the infra in already in the kolla-kubernetes gates. | 16:11 |
| vhosakot | inc0: should the upgrade gates test both scenarios - upgrade from minor release-to-minor release, and upgrade to major release? | 16:11 |
| kfox1111 | the one ps already successfully does a reboot. just not all of the services come up. | 16:11 |
| kfox1111 | the upgrade gate doesn't need anythign special out of infra. | 16:11 |
| inc0 | kfox1111: I'm thinking in a lines of "pull old images, pull new images, download old version, deploy, download new version, upgrade" | 16:11 |
| inc0 | and some testing scenarios | 16:11 |
| kfox1111 | inc0: yeah. but that code isn't common between k8s and ansible? | 16:12 |
| inc0 | gate code to pull images from trarballs will be common | 16:12 |
| kfox1111 | ah. true. | 16:12 |
| inc0 | scenarios will be common | 16:12 |
| *** huzhengchuan has quit IRC | 16:12 | |
| kfox1111 | already have a ps up for that for kolla-kubernetes. should be adaptable for ansible side. | 16:13 |
| inc0 | and logic between "deploy old" (so figure out which one is old and where is the link), spawn vm, download/deploy new, check if vm exists | 16:13 |
| inc0 | most of it is actually common | 16:13 |
| kfox1111 | inc0: not sure I follow. | 16:13 |
| inc0 | well, in my brain it's gonna be more common than not common;) | 16:14 |
| inc0 | but we'll see | 16:14 |
| kfox1111 | your upgrading a vm, not launching a new one? | 16:14 |
| inc0 | I was thinking of checking if old vm didn't break during upgrade | 16:14 |
| kfox1111 | OH. your meaning sanity checks after the upgrade? | 16:14 |
| kfox1111 | yeah. that should all be sharable. | 16:14 |
| inc0 | persistence tests and sanity tests | 16:14 |
| inc0 | yeah | 16:15 |
| kfox1111 | +1 | 16:15 |
| inc0 | I think most of hard part for ansible is that kind of stuff | 16:15 |
| inc0 | since upgrade is already written | 16:15 |
| kfox1111 | yeah. | 16:15 |
| *** manheim has quit IRC | 16:16 | |
| *** athomas has quit IRC | 16:21 | |
| vhosakot | inc0: kfox1111: so, let me know how can I help with upgrade gates. Last time I chcecked was inc0's review with infra hooks to add upgrade gates. should I start with that? | 16:22 |
| vhosakot | I'm back from my PTO and traveling. | 16:22 |
| kfox1111 | not sure how much in common the ansible and k8s gates have in common. | 16:22 |
| kfox1111 | k8s launches two vm's, does a cinder volume, does some sanity checks around network/vms/cinder. | 16:23 |
| kfox1111 | the upgrade test for k8s should start assuming that was done, then do some more. | 16:23 |
| *** igordcard has quit IRC | 16:24 | |
| *** lucasxu has quit IRC | 16:27 | |
| *** jascott1 has quit IRC | 16:27 | |
| vhosakot | kfox1111: yes, I agree not much is common in ansible and k8s gates. | 16:27 |
| *** jascott1 has joined #openstack-kolla | 16:27 | |
| *** igordcard has joined #openstack-kolla | 16:30 | |
| *** jascott1 has quit IRC | 16:32 | |
| *** igordcard has quit IRC | 16:33 | |
| *** schwicht has joined #openstack-kolla | 16:34 | |
| *** reidrac has quit IRC | 16:34 | |
| *** igordcard has joined #openstack-kolla | 16:35 | |
| *** jascott1 has joined #openstack-kolla | 16:39 | |
| *** sambetts is now known as sambetts|afk | 16:44 | |
| *** kiennt has quit IRC | 16:44 | |
| *** daidv_ has joined #openstack-kolla | 16:48 | |
| *** jascott1 has quit IRC | 16:52 | |
| *** jascott1 has joined #openstack-kolla | 16:53 | |
| *** itlinux has joined #openstack-kolla | 16:54 | |
| *** jascott1 has quit IRC | 16:56 | |
| *** unicell has quit IRC | 16:59 | |
| *** daidv_ has quit IRC | 17:06 | |
| *** mgoddard_ has quit IRC | 17:08 | |
| *** duonghq_ has quit IRC | 17:09 | |
| *** serlex has quit IRC | 17:09 | |
| *** hieulq has quit IRC | 17:09 | |
| *** daidv has quit IRC | 17:09 | |
| *** igordcard has quit IRC | 17:10 | |
| *** igordcard has joined #openstack-kolla | 17:11 | |
| *** igordcard has quit IRC | 17:11 | |
| *** igordcard has joined #openstack-kolla | 17:12 | |
| *** hieulq has joined #openstack-kolla | 17:25 | |
| *** daidv has joined #openstack-kolla | 17:25 | |
| *** duonghq_ has joined #openstack-kolla | 17:25 | |
| *** ducttape_ has quit IRC | 17:31 | |
| *** ducttape_ has joined #openstack-kolla | 17:38 | |
| *** igordcard has quit IRC | 17:41 | |
| *** igordcard has joined #openstack-kolla | 17:42 | |
| *** igordcard has quit IRC | 17:44 | |
| *** igordcard has joined #openstack-kolla | 17:44 | |
| *** mgoddard_ has joined #openstack-kolla | 17:48 | |
| *** igordcard has quit IRC | 17:48 | |
| *** igordcard has joined #openstack-kolla | 17:48 | |
| *** jascott1 has joined #openstack-kolla | 18:00 | |
| *** unicell has joined #openstack-kolla | 18:02 | |
| *** tonanhngo has joined #openstack-kolla | 18:08 | |
| *** rwsu has quit IRC | 18:08 | |
| *** rwsu has joined #openstack-kolla | 18:09 | |
| *** coolsvap has quit IRC | 18:15 | |
| *** omenvlad has joined #openstack-kolla | 18:18 | |
| *** dave-mccowan has joined #openstack-kolla | 18:21 | |
| openstackgerrit | Michal Jastrzebski (inc0) proposed openstack/kolla-ansible master: Allow tmp path to download registry https://review.openstack.org/475940 | 18:26 |
| *** schwicht has quit IRC | 18:26 | |
| *** schwicht has joined #openstack-kolla | 18:28 | |
| jamesbenson | are all of the centos images available on docker hub for 4.0.0? | 18:33 |
| inc0 | jamesbenson: they are, tho outdated | 18:40 |
| jamesbenson | yep, is it faster to build docker images from source/binary due to pip/yum vs source tarball+pip? odd question, sorry | 18:46 |
| openstackgerrit | Michal Jastrzebski (inc0) proposed openstack/kolla master: Dockerfile publishing pipeline https://review.openstack.org/475469 | 18:47 |
| inc0 | usually source is a little faster | 18:48 |
| inc0 | not much of a difference tho | 18:48 |
| jamesbenson | okay, thanks :-) | 18:48 |
| *** daidv has quit IRC | 18:49 | |
| *** hieulq has quit IRC | 18:49 | |
| *** duonghq_ has quit IRC | 18:50 | |
| *** hongbin has joined #openstack-kolla | 18:56 | |
| hongbin | hi, i tried to follow this guide: https://docs.openstack.org/kolla-ansible/latest/quickstart.html , but couldn't manage to pass the precheck: http://paste.openstack.org/show/614497/ | 18:57 |
| hongbin | is anything i need to do to pass the precheck? | 18:58 |
| inc0 | hongbin: how did you install kolla-ansible | 18:58 |
| hongbin | inc0: install from source : git clone ... ; pip install -e ... | 18:59 |
| inc0 | go to source dir and run tools/kolla-ansible | 18:59 |
| inc0 | that should fix it | 18:59 |
| vhosakot | hongbin: "-i" must be the path to inventory instead of "all-in-one". https://github.com/openstack/kolla-ansible/blob/master/tools/kolla-ansible#L37. AIO inventory is usually in ${BASEDIR}/ansible/inventory/all-in-one per https://github.com/openstack/kolla-ansible/blob/master/tools/kolla-ansible#L107 | 19:01 |
| hongbin | inc0: thx, it seems to fix this error | 19:01 |
| inc0 | yeah, we have issue with our pip isntallation from source | 19:01 |
| hongbin | i see | 19:02 |
| *** ducttape_ has quit IRC | 19:02 | |
| *** daidv has joined #openstack-kolla | 19:04 | |
| *** hieulq has joined #openstack-kolla | 19:04 | |
| *** duonghq_ has joined #openstack-kolla | 19:05 | |
| *** schwicht has quit IRC | 19:05 | |
| *** iniazi has joined #openstack-kolla | 19:06 | |
| vhosakot | the deadline to submit talks for Sydney is July 14th. we could submit many talks about kolla (automated build, container snapshot/commit, prod-sanity checks, upgrades for which inc0 has ideas, containers with fast-data DPDK, SR-IOV).. let me know if anyone like sto submit talks and I'd love to be a co-speaker. | 19:07 |
| inc0 | vhosakot: yeah, let's submit talk this week | 19:08 |
| vhosakot | cool, inc0 can we discuss it in next week's meeting so anyone interested can submit more talks :) | 19:08 |
| inc0 | opendaylight+dpdk+kolla would also make awesome talk+demo | 19:09 |
| vhosakot | yep! | 19:09 |
| inc0 | next meeting is at 12th | 19:09 |
| inc0 | so not much time left afterwards | 19:09 |
| *** iniazi_ has quit IRC | 19:09 | |
| *** schwicht has joined #openstack-kolla | 19:09 | |
| vhosakot | ah right | 19:09 |
| inc0 | sean-k-mooney mgkwill interested?^ | 19:09 |
| vhosakot | here is my opendaylight+openstack workshop I did at Cisco Live last week. many people attended. https://www.ciscolive.com/us/learn/sessions/session-catalog/?search=DEVNET-2041. I did not include DPDK container as I could not get neutron's security groups not drop control plance traffic from fast data sender. | 19:12 |
| vhosakot | I got basic ping (icmp) working with neutron+dpdk by allowing incoming/outgoing imcp in security groups, but other trafffic from my fast data sender had random udp ports that neutron promptly dropped :) | 19:13 |
| vhosakot | I added some custom configs for Cisco boxes in https://review.openstack.org/#/c/416369/. | 19:14 |
| *** ducttape_ has joined #openstack-kolla | 19:20 | |
| rwellum | vhosakot: inc0: how about something on building your own images from source - then run them OS with them? I managed to do that last week and noticed that there was no documentation of the whole process? | 19:20 |
| *** gfidente is now known as gfidente|afk | 19:22 | |
| vhosakot | rwellum: meaning, use non-kolla DockerFiles with custom repos? | 19:23 |
| vhosakot | rwellum: source for the packages? or source for the container itself (DockerFile)? | 19:23 |
| rwellum | Creating kolla images from a custom repo, running (kolla-kubernetes) with that/those image(s). | 19:25 |
| vhosakot | rwellum: ah, I think building kolla images with custom repos will be a nice talk as many ops want to do it and do not want upstream/out-of-the-box repos that kolla provides, and want kolla to build internal/curated custom packages | 19:25 |
| *** ducttap__ has joined #openstack-kolla | 19:25 | |
| *** ducttape_ has quit IRC | 19:26 | |
| vhosakot | rwellum: did you have to change a lot apart from setting rpm_setup_config in kolla-build.conf to custom repos per https://github.com/openstack/kolla/blob/master/doc/image-building.rst#custom-repos ? | 19:28 |
| rwellum | Yeah more than that :) | 19:29 |
| vhosakot | rwellum: like, did you have to edit DockerFiles? Ansible playbooks too? | 19:29 |
| rwellum | vhosakot: This isn't complete - but I'll update it shortly: https://etherpad.openstack.org/p/kolla-kubernetes_with_your_own_source_images | 19:30 |
| rwellum | Take a look - let me know? | 19:30 |
| hongbin | hi, running into this error now when i ran deploy: http://paste.openstack.org/show/614504/ | 19:31 |
| vhosakot | rwellum: nice! so is the idea point kolla-build.conf to each service's custom repo and build custom container from source, and not touch anything else like DockerFile and Anisble playbook? | 19:31 |
| hongbin | i shouldn't enable anything about fluentd, not sure why it complained about it | 19:32 |
| rwellum | Right - exactly. And the point being exactly what you detailed above very eloquently :) | 19:32 |
| vhosakot | rwellum: yes, I think it will be a good talk. inc0 I think running kolla with custom internal repos will be a great talk for anyone/ops. | 19:33 |
| *** ducttape_ has joined #openstack-kolla | 19:34 | |
| rwellum | Cool vhosakot ! | 19:35 |
| *** eaguilar has quit IRC | 19:37 | |
| *** ducttap__ has quit IRC | 19:38 | |
| vhosakot | rwellum: after building custom keystone image, how did you start it? ran "kolla-ansible deploy"? | 19:38 |
| sbezverk | I have some interesting things going on with sriov and kubernetes, not sure if it would be a good topic for openstack talk ;) | 19:40 |
| rwellum | vhosakot: I'm using kubernetes - and I'm running it with my own tool I've developed. But basically, some additions to cloud.yaml and then using kubeadm | 19:41 |
| vhosakot | sbezverk: oooh, which cards did you use for SR-IOV? Cisco VICs 1340, 1380? or Intel/Mellanox? I've deployed SR-IOV with OpenStack but not with k8s tho... | 19:41 |
| rwellum | sbezverk: that would be more than inetersting! | 19:41 |
| vhosakot | rwellum: ah cool | 19:41 |
| sbezverk | vhosakot : it is Intel ixgbe, the best NIC of all times ;) | 19:42 |
| vhosakot | sbezverk: ah, I'm using some Intel ixgbe's recently to test VPP/DPDK compute kit, cool man :) | 19:42 |
| vhosakot | rwellum: so, you did not use rpm_setup_config to build custom packages because you wanted to build from source and use "install_type = source" in kolla-build.conf, right? | 19:43 |
| rwellum | sbezverk: I think I fixed a ton of bugs in the that driver in XR vpp :) | 19:44 |
| rwellum | Cost me more than a few months. | 19:44 |
| vhosakot | rpm_setup_config is for binary custom packages pulled from custome repo files. | 19:44 |
| sbezverk | rwellum : have you fixed in upstream driver? | 19:45 |
| rwellum | vhosakot: correct. But i really just asked the cores here for advise and this is what they told me to do. | 19:45 |
| vhosakot | rwellum: cool cool... building from custom source is a great idea.. yes,. | 19:45 |
| rwellum | sbezverk: no - at the time it was Dave Barach's proprietary version I think - no sure where it went. | 19:45 |
| vhosakot | rwellum: so, did you end up having a section for each container like [keystone-ssh], [keystone-fernet], [barbican-keystone-listener] for each container... the etherpad's example just has [keystone-base] under the section "Edit kolla-build.conf and add pointer to source" in https://etherpad.openstack.org/p/kolla-kubernetes_with_your_own_source_images. | 19:46 |
| sbezverk | rwellum : I see unfortunately it never left that group. Intel built a relatively good driver for x540 5.1.4 so far I do not see much troubles | 19:46 |
| rwellum | vhosakot: so far I have only done it for keystone, but yes that's the general idea. I am going to add in horizon and cinder next - as those are areas I'm interested in. | 19:49 |
| rwellum | sbezverk: cool | 19:49 |
| vhosakot | rwellum: cool, great work. can you please add a doc for it in our kolla repo, may add a section "building from source" under the existing header "Custom repos" in https://github.com/openstack/kolla/blob/master/doc/image-building.rst#custom-repos. | 19:50 |
| rwellum | Yes will do. | 19:51 |
| vhosakot | rwellum: thanks! | 19:51 |
| vhosakot | sbezverk: have you deployed OVS or Linuxbridge too incase a VM does not want to use SR-IOV and wants the legacy virtual L2 switch like OVS or Linuxbridge? | 19:52 |
| sbezverk | vhosakot : deployed where? | 19:52 |
| vhosakot | sbezverk: I meant, can VM also use OVS or LinuxBridge too when using SR-IOV... like in my case, my customer wanted few VMs using SR-IOV and bypass the hypervisor and other VMs using OVS.. so, I had to deploy OVS with SR-IOV for the customer. | 19:55 |
| sbezverk | vhosakot : in my old day ;) while I was still playing with sriov on openstack, that was not the case. You did not need ovs to give access to VM for sriov interfaces | 19:57 |
| *** rwallner has quit IRC | 19:57 | |
| vhosakot | right right, yeah.. I think neutron uses macv-type as direct (without OVS) with connecting a port to SR-IOV.. cool | 19:58 |
| *** rwallner_ has joined #openstack-kolla | 20:00 | |
| *** manheim has joined #openstack-kolla | 20:02 | |
| *** manheim has quit IRC | 20:04 | |
| *** manheim has joined #openstack-kolla | 20:04 | |
| *** rwallner_ has quit IRC | 20:04 | |
| inc0 | vhosakot: btw https://review.openstack.org/#/c/342354/ | 20:08 |
| *** jgruber has quit IRC | 20:10 | |
| *** jgruber has joined #openstack-kolla | 20:12 | |
| keico | Hello. I am trying to deploy an Openstack cloud (ocata) using kolla-ansible (version 4.0.2). | 20:13 |
| keico | Currently, the task "Creating admin project, user, role, service, and endpoint" fails with the exact errors: | 20:13 |
| keico | TASK [keystone : Creating admin project, user, role, service, and endpoint] ******* | 20:13 |
| vhosakot | inc0: thanks for the pointer. | 20:13 |
| keico | [WARNING]: when statements should not include jinja2 templating delimiters such as {{ }} or {% %}. Found: {{ (keystone_bootstrap.stdout | from_json).changed }} | 20:14 |
| keico | fatal: [control01]: FAILED! => {"failed": true, "msg": "The conditional check '{{ (keystone_bootstrap.stdout | from_json).changed }}' failed. The error was: Expecting ',' delimiter: line 1 column 133 (char 132)"} | 20:14 |
| keico | Does anyone know how to resolve this problem? Thank you in advance for your help! | 20:14 |
| inc0 | hmm I've never seen this | 20:16 |
| jamesbenson | could this be a jinja2 version issue? | 20:16 |
| vhosakot | ^^^ I think the same too | 20:16 |
| jamesbenson | keico, what version of jinja2 do you have? | 20:17 |
| keico | Here's the source/config file for this task: https://github.com/openstack/kolla-ansible/blob/4.0.2/ansible/roles/keystone/tasks/register.yml | 20:17 |
| keico | One second. Let me check | 20:17 |
| keico | [root@kolla-deployment inventory]# pip show Jinja2 Name: Jinja2 Version: 2.9.6 Summary: A small but fast and easy to use stand-alone template engine written in pure python. Home-page: http://jinja.pocoo.org/ Author: Armin Ronacher Author-email: armin.ronacher@active-4.com License: BSD Location: /usr/lib64/python2.7/site-packages Requires: MarkupSafe | 20:18 |
| keico | I believe I have Jinja2 version 2.9.6. | 20:20 |
| inc0 | keico: hmm, also what images are you using | 20:20 |
| inc0 | ? | 20:20 |
| keico | I was able to build all the docke images with version 4.0.2. | 20:21 |
| inc0 | ok that's good | 20:22 |
| keico | I am not sure if this is relavent. I also see this error from my keystone container, running on the control node: | 20:22 |
| inc0 | probably relevant | 20:22 |
| keico | 2017-07-05 15:46:09.986 58 ERROR keystone ProgrammingError: (pymysql.err.ProgrammingError) (1146, u"Table 'keystone.project' doesn't exist") [SQL: u'INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)'] [parameters: {'is_domain': 1, 'description': 'The default domain', 'extra | 20:23 |
| inc0 | definetly relevant:) | 20:23 |
| inc0 | seems like something broke down during database syncdb? | 20:23 |
| inc0 | did you see any issues before? | 20:23 |
| *** ansmith has quit IRC | 20:26 | |
| *** hieulq has quit IRC | 20:26 | |
| *** daidv has quit IRC | 20:26 | |
| inc0 | pbourke: will it run Kolla? https://thenewstack.io/oracle-opens-oci-container-runtime ;) | 20:27 |
| keico | No. To be honest, I am still newbie to kolla/kolla-ansible. So, this is where I have made that far :) | 20:28 |
| inc0 | keico: best advice, let's try to destroy/recreate | 20:28 |
| inc0 | kolla-ansible -i path-to-inv destroy --yes-i-really-really-mean-it | 20:28 |
| keico | ok. thanks! | 20:28 |
| inc0 | maybe something wierd happened during sync and it failed quietly for whatever reason | 20:29 |
| inc0 | if problem persists, we'll investigate more | 20:29 |
| rwellum | sbezverk: any luck with 1.7.0? - seems stable to me. Worked on centos - trying ubuntu now. | 20:32 |
| sbezverk | rwellum : look ok to me too, there was a chance in authorization, but with rbac roles/bindings we have currently, it seems to be not affecting us | 20:33 |
| *** shardy has quit IRC | 20:35 | |
| keico | inc0: thanks so much for your advice! It works! | 20:37 |
| inc0 | glad to hear | 20:37 |
| inc0 | good thing about Kolla is that it can be re-run within few minutes;) | 20:37 |
| *** jtriley has quit IRC | 20:39 | |
| *** keico has quit IRC | 20:40 | |
| *** daidv has joined #openstack-kolla | 20:41 | |
| *** hieulq has joined #openstack-kolla | 20:42 | |
| rwellum | sbezverk: nice. Good to have a stable release. | 20:47 |
| *** dciabrin has quit IRC | 20:49 | |
| *** dciabrin has joined #openstack-kolla | 20:49 | |
| *** rwallner has joined #openstack-kolla | 20:55 | |
| *** rwallner has quit IRC | 20:59 | |
| *** manheim has quit IRC | 21:04 | |
| *** gfidente|afk is now known as gfidente | 21:07 | |
| openstackgerrit | Merged openstack/kolla master: introduces support for the OVS DPDK dataplane https://review.openstack.org/342354 | 21:08 |
| openstackgerrit | Bertrand Lallau proposed openstack/kolla master: Fix searchlight_api container starting issue https://review.openstack.org/480743 | 21:14 |
| *** rhallisey has quit IRC | 21:15 | |
| *** goldyfruit has quit IRC | 21:16 | |
| *** Manheim has joined #openstack-kolla | 21:17 | |
| *** fooliouno has quit IRC | 21:21 | |
| *** mgoddard_ has quit IRC | 21:44 | |
| *** hieulq has quit IRC | 21:45 | |
| *** duonghq_ has quit IRC | 21:45 | |
| *** daidv has quit IRC | 21:46 | |
| *** Pavo has joined #openstack-kolla | 21:57 | |
| *** daidv has joined #openstack-kolla | 22:04 | |
| *** duonghq has joined #openstack-kolla | 22:05 | |
| *** hieulq has joined #openstack-kolla | 22:05 | |
| *** schwicht has quit IRC | 22:05 | |
| *** daidv has quit IRC | 22:07 | |
| *** Manheim has quit IRC | 22:13 | |
| *** jascott1 has quit IRC | 22:13 | |
| *** jascott1 has joined #openstack-kolla | 22:14 | |
| *** Pavo has quit IRC | 22:14 | |
| *** jascott1 has quit IRC | 22:15 | |
| *** rfxn has joined #openstack-kolla | 22:18 | |
| *** tonanhngo has quit IRC | 22:21 | |
| *** Teck72 has quit IRC | 22:21 | |
| *** manheim has joined #openstack-kolla | 22:21 | |
| *** daidv has joined #openstack-kolla | 22:23 | |
| *** schwicht has joined #openstack-kolla | 22:23 | |
| *** tonanhngo has joined #openstack-kolla | 22:26 | |
| *** tonanhngo has quit IRC | 22:31 | |
| *** jascott1 has joined #openstack-kolla | 22:31 | |
| *** goldyfruit has joined #openstack-kolla | 22:33 | |
| *** schwicht has quit IRC | 22:49 | |
| *** schwicht has joined #openstack-kolla | 22:49 | |
| *** itlinux has quit IRC | 22:50 | |
| *** klindgren_ is now known as klindgren | 22:52 | |
| *** Pavo has joined #openstack-kolla | 22:54 | |
| *** schwicht has quit IRC | 22:55 | |
| *** goldyfruit has quit IRC | 22:58 | |
| *** Pavo has quit IRC | 22:58 | |
| *** Pavo has joined #openstack-kolla | 23:05 | |
| *** Pavo has quit IRC | 23:07 | |
| *** ansmith has joined #openstack-kolla | 23:09 | |
| *** ducttape_ has quit IRC | 23:12 | |
| *** hongbin has quit IRC | 23:15 | |
| *** hieulq has quit IRC | 23:21 | |
| *** duonghq has quit IRC | 23:21 | |
| *** daidv has quit IRC | 23:21 | |
| *** schwicht has joined #openstack-kolla | 23:22 | |
| *** ducttape_ has joined #openstack-kolla | 23:23 | |
| *** manheim has quit IRC | 23:24 | |
| *** ducttape_ has quit IRC | 23:28 | |
| *** daidv has joined #openstack-kolla | 23:35 | |
| *** duonghq has joined #openstack-kolla | 23:35 | |
| *** hieulq has joined #openstack-kolla | 23:35 | |
| *** bmace has joined #openstack-kolla | 23:40 | |
| *** manheim has joined #openstack-kolla | 23:40 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!