| *** eaguilar has quit IRC | 00:08 | |
| *** yingjun has joined #openstack-kolla | 00:11 | |
| *** yangyapeng has joined #openstack-kolla | 00:16 | |
| *** yangyape_ has quit IRC | 00:19 | |
| *** yangyapeng has quit IRC | 00:20 | |
| *** rhallisey has quit IRC | 00:27 | |
| *** zhurong has joined #openstack-kolla | 00:33 | |
| *** krtaylor has joined #openstack-kolla | 00:39 | |
| sdake | sup spiette | 00:41 |
|---|---|---|
| sdake | sup spsurya | 00:41 |
| *** duonghq has joined #openstack-kolla | 00:42 | |
| duonghq | morning guys | 00:42 |
| sdake | sup duonghq | 00:43 |
| *** tovin07_ has joined #openstack-kolla | 00:43 | |
| duonghq | hi sdake | 00:45 |
| openstackgerrit | Steven Dake proposed openstack/kolla-kubernetes master: Placement support in nova service chart https://review.openstack.org/453846 | 00:45 |
| spsurya | duonghq: morning | 00:46 |
| duonghq | hi spsurya | 00:47 |
| duonghq | how about your weekend sdake? | 00:47 |
| sdake | busy | 00:47 |
| *** iceyao has joined #openstack-kolla | 01:07 | |
| *** sayantan_ has quit IRC | 01:11 | |
| openstackgerrit | Steven Dake proposed openstack/kolla-kubernetes master: Placement support in nova service chart https://review.openstack.org/453846 | 01:16 |
| sdake | duonghq did this: https://twitter.com/steven_dake | 01:18 |
| sdake | i also deployed my new server onto a cardbox box rack :) | 01:19 |
| sdake | wife is not pleased | 01:19 |
| sdake | "you should be putting that rack together" | 01:20 |
| sdake | there is like 100+ screws | 01:20 |
| * sdake groans | 01:20 | |
| sdake | i think i've finished the job on the placement api integration with the service chart for nova-control tho | 01:20 |
| sdake | found the last bug | 01:20 |
| sdake | gate will tell | 01:20 |
| sdake | ;-) | 01:20 |
| *** yangyapeng has joined #openstack-kolla | 01:20 | |
| duonghq | sdake, you did this by yourself, I mean only you? | 01:22 |
| duonghq | (the twitter link) | 01:22 |
| sdake | dad helped with the heavy lifting | 01:24 |
| sdake | that hose reel is about 120 pounds | 01:24 |
| sdake | the jeep thing is about 200 | 01:25 |
| sdake | but ya - otherrwise did myself | 01:25 |
| sdake | learned more about air compressors then any one person should know :) | 01:25 |
| duonghq | nice | 01:25 |
| sdake | QDC, NPT plugs | 01:26 |
| sdake | atleast it is standardized | 01:26 |
| sdake | had to tape the screws as well | 01:26 |
| sdake | pita | 01:26 |
| sdake | rather the threads | 01:26 |
| duonghq | there is little tricky here? | 01:26 |
| sdake | nah its pretty straight forward | 01:27 |
| sdake | if you know what parts to buy :) | 01:27 |
| duonghq | ya | 01:27 |
| sdake | watching zuul operate - like watching paint dry | 01:27 |
| sdake | although I have learned compltely how the gate works in the last coupel of weeks | 01:28 |
| sdake | (for kolla-kubernetes that is) | 01:28 |
| duonghq | awesome, much better for kollish | 01:29 |
| duonghq | but, can we setup > 2 nodes gate with zuul v2? | 01:29 |
| *** hrw has quit IRC | 01:30 | |
| *** hrw has joined #openstack-kolla | 01:32 | |
| openstackgerrit | Steven Dake proposed openstack/kolla-kubernetes master: Add --debug to compute-kit deploy https://review.openstack.org/455094 | 01:32 |
| openstackgerrit | Steven Dake proposed openstack/kolla-kubernetes master: Placement support in nova service chart https://review.openstack.org/453846 | 01:32 |
| *** cuongnv has joined #openstack-kolla | 01:32 | |
| sdake | duonghq indeed it is possible | 01:32 |
| sdake | duonghq although I hav eto say for kolla-kuberntes I think it is essential to gate on ocata images | 01:33 |
| sdake | rather then worry about multinode gating | 01:33 |
| duonghq | okay | 01:33 |
| sdake | duonghq if your interested in doing some of ht ework there - good way to learn how the gate works | 01:33 |
| sdake | duonghq also need reviews on the kola-kubernetes repo :) | 01:34 |
| sdake | i have like 30 reviews outstanding atm | 01:34 |
| duonghq | sdake, understood | 01:34 |
| *** dixiaoli has joined #openstack-kolla | 01:38 | |
| eanylin | sdake: Looks like you had fun this weekend :) | 01:40 |
| sdake | eanylin if you could call it fun | 01:40 |
| eanylin | haha, it really depends ;) | 01:40 |
| eanylin | sdake: Are the 4.0.0 image same as the ocata image that you referred to earlier? | 01:41 |
| sdake | eanylin indeed | 01:41 |
| eanylin | sdake: Ah, ok. I thought they are pretty stable? Why do we still need to gate on those? | 01:41 |
| eanylin | its not like they change right? Since its versioned | 01:42 |
| sdake | eanylin the containers are stable on 4.0.0 | 01:42 |
| sdake | the helm charts with 4.0.0 YMMV | 01:43 |
| eanylin | YMMV ? | 01:43 |
| sdake | hence the need for gating | 01:43 |
| sdake | your milage will vary | 01:43 |
| eanylin | oh | 01:43 |
| eanylin | ok | 01:43 |
| duonghq | sdake, eanylin will you go to Boston? | 01:50 |
| eanylin | duonghq: I am going, I believe sdake is going too. You? | 01:51 |
| sdake | duonghq indeed | 01:51 |
| duonghq | I have a VISA interview before I can confirm | 01:52 |
| eanylin | duonghq: I see. Good luck :) | 01:53 |
| eanylin | I think should be ok | 01:53 |
| duonghq | tks | 01:54 |
| sdake | if people looking to do some work - check out: https://launchpad.net/kolla-kubernetes/+milestone/0.6.0 | 01:54 |
| sdake | april 15th is deadline :) | 01:57 |
| sdake | folks could help out on this one to learn kolla-kubernetes: https://blueprints.launchpad.net/kolla-kubernetes/+spec/helm-service-cleanup | 01:58 |
| sdake | there are several TODO's in that blueprint | 01:58 |
| sdake | should mostly be C&P | 01:58 |
| *** zhurong has quit IRC | 01:58 | |
| *** yangyapeng has quit IRC | 01:59 | |
| *** yangyapeng has joined #openstack-kolla | 01:59 | |
| *** ksumit has quit IRC | 02:01 | |
| spsurya | duonghq: all the best !!! | 02:02 |
| spsurya | sdake: I thought multinode deployment will also be part of this BP https://blueprints.launchpad.net/kolla-kubernetes/+spec/deploy-guide | 02:08 |
| spsurya | then it would be completed | 02:09 |
| sdake | spsurya we can follow that blueprint up with multinode when we actually ahve multinode working :) | 02:10 |
| *** zhurong has joined #openstack-kolla | 02:10 | |
| spsurya | sdake: though... it is kind of lame query, so once multinode would be working we will add the multinode guide patch review in the same BP ? | 02:13 |
| openstackgerrit | Steven Dake proposed openstack/kolla-kubernetes master: Placement support in nova service chart https://review.openstack.org/453846 | 02:14 |
| sdake | spsurya i think we can make a new blueprint to expand that document | 02:14 |
| sdake | or just expand it naturally | 02:14 |
| *** yingjun has quit IRC | 02:15 | |
| spsurya | sdake: ok | 02:15 |
| *** yingjun has joined #openstack-kolla | 02:18 | |
| *** yingjun has quit IRC | 02:24 | |
| *** yingjun has joined #openstack-kolla | 02:29 | |
| *** zhubingbing has joined #openstack-kolla | 02:35 | |
| *** yingjun has quit IRC | 02:35 | |
| *** yangyapeng has quit IRC | 02:35 | |
| *** yangyapeng has joined #openstack-kolla | 02:36 | |
| *** yingjun has joined #openstack-kolla | 02:37 | |
| *** yingjun has quit IRC | 02:41 | |
| *** yingjun has joined #openstack-kolla | 02:41 | |
| *** iniazi has joined #openstack-kolla | 02:45 | |
| *** yingjun has quit IRC | 02:58 | |
| *** yingjun has joined #openstack-kolla | 03:00 | |
| *** zhubingbing has quit IRC | 03:11 | |
| openstackgerrit | shaofeng cheng proposed openstack/kolla-ansible master: Fix ansible_python is undefined in precheck. https://review.openstack.org/455109 | 03:12 |
| *** zhubingbing_ has joined #openstack-kolla | 03:14 | |
| *** yingjun has quit IRC | 03:14 | |
| *** yingjun has joined #openstack-kolla | 03:16 | |
| openstackgerrit | Steven Dake proposed openstack/kolla-kubernetes master: Placement support in nova service chart https://review.openstack.org/453846 | 03:18 |
| *** zhubingbing_ has quit IRC | 03:19 | |
| *** zhubingbing_ has joined #openstack-kolla | 03:21 | |
| *** shashank_t_ has joined #openstack-kolla | 03:22 | |
| *** zhubingbing_ has quit IRC | 03:23 | |
| *** zhubingbing_ has joined #openstack-kolla | 03:24 | |
| *** gkadam has joined #openstack-kolla | 03:24 | |
| *** shashank_t_ has quit IRC | 03:27 | |
| *** sayantan_ has joined #openstack-kolla | 03:29 | |
| *** fooliouno has joined #openstack-kolla | 03:42 | |
| openstackgerrit | Steven Dake proposed openstack/kolla-kubernetes master: Placement support in nova service chart https://review.openstack.org/453846 | 03:55 |
| sdake | Jeffrey4l around? | 03:59 |
| masber | My control and compute ndoes have different nic names, I tried to specify the nic names on the inventory file but for some reason it doesn't work. This is the error message, the globals.yml, the inventory file and the nics configuration for both servers (compute and control) --> http://paste.openstack.org/raw/606001/ | 04:02 |
| masber | I am using kolla-ansible | 04:03 |
| *** yingjun has quit IRC | 04:05 | |
| japestinho | masber what OS did you use? | 04:05 |
| masber | centos 7 | 04:05 |
| japestinho | masber I'm suggesting you to rename that device | 04:07 |
| sdake | anyone not attend the PTG that is attending summit? | 04:07 |
| japestinho | masber you can look at this http://unix.stackexchange.com/questions/205010/centos-7-rename-network-interface-without-rebooting/219277 for temporary change | 04:08 |
| japestinho | or https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/Networking_Guide/sec-Controlling_the_Selection_of_Network_Device_Names.html for permanent change | 04:08 |
| masber | japestinho, thanks I used to do this before I was just wondering why kolla-ansible was complaining | 04:14 |
| masber | also, I saw that the compute node also has a br-ex, why is that? I though br-ex only makes sense on neutron node? | 04:18 |
| *** yangyapeng has quit IRC | 04:20 | |
| *** yangyapeng has joined #openstack-kolla | 04:21 | |
| *** yingjun has joined #openstack-kolla | 04:43 | |
| *** jrobinson has quit IRC | 04:56 | |
| *** jrobinson has joined #openstack-kolla | 05:03 | |
| masber | I also don't understand why kolla-ansible is complaning if the neutron_external_interface is missing from the compute node? | 05:22 |
| masber | neutron_external_interface is for the br-ex which only exists on the neutron node, isn't it? | 05:23 |
| *** yingjun has quit IRC | 05:23 | |
| masber | could someone from kolla-ansible help me to understand ^^^? | 05:23 |
| *** yingjun has joined #openstack-kolla | 05:26 | |
| *** skramaja has joined #openstack-kolla | 05:41 | |
| *** yingjun has quit IRC | 05:42 | |
| *** unicell has joined #openstack-kolla | 05:55 | |
| *** unicell has quit IRC | 05:59 | |
| *** unicell has joined #openstack-kolla | 06:00 | |
| *** pcaruana has joined #openstack-kolla | 06:01 | |
| *** iceyao has quit IRC | 06:06 | |
| *** iceyao has joined #openstack-kolla | 06:06 | |
| *** hieulq has quit IRC | 06:07 | |
| *** yangyapeng has quit IRC | 06:19 | |
| *** zhuzeyu has joined #openstack-kolla | 06:24 | |
| *** g3ek has quit IRC | 06:27 | |
| *** manheim has joined #openstack-kolla | 06:29 | |
| *** g3ek has joined #openstack-kolla | 06:36 | |
| *** jascott1_ has joined #openstack-kolla | 06:37 | |
| *** jascott1_ has quit IRC | 06:42 | |
| openstackgerrit | Steven Dake proposed openstack/kolla-kubernetes master: Will need to squash this later if this works https://review.openstack.org/455132 | 06:44 |
| *** pbourke has quit IRC | 06:45 | |
| *** pbourke has joined #openstack-kolla | 06:47 | |
| *** jaosorior has joined #openstack-kolla | 06:49 | |
| *** jascott1_ has joined #openstack-kolla | 07:02 | |
| *** jascott1_ has quit IRC | 07:03 | |
| *** dmellado has joined #openstack-kolla | 07:08 | |
| *** targon has joined #openstack-kolla | 07:10 | |
| *** denaitre has quit IRC | 07:15 | |
| *** Serlex has joined #openstack-kolla | 07:16 | |
| openstackgerrit | Marcin Juszkiewicz proposed openstack/kolla master: mariadb: handle lack of external repos on non x86 https://review.openstack.org/444832 | 07:16 |
| *** MarginHu has joined #openstack-kolla | 07:16 | |
| MarginHu | hello, Kolla | 07:16 |
| manheim | hello | 07:22 |
| *** jrobinson has quit IRC | 07:24 | |
| *** deadnull has quit IRC | 07:26 | |
| openstackgerrit | Steven Dake proposed openstack/kolla-kubernetes master: Will need to squash this later if this works https://review.openstack.org/455132 | 07:31 |
| *** daidv has joined #openstack-kolla | 07:35 | |
| *** deadnull has joined #openstack-kolla | 07:35 | |
| *** shardy has joined #openstack-kolla | 07:36 | |
| *** nathharp has joined #openstack-kolla | 07:36 | |
| openstackgerrit | shaofeng cheng proposed openstack/kolla-ansible master: Fix ansible_python is undefined in precheck. https://review.openstack.org/455109 | 07:37 |
| sdake | Jeffrey4l about? | 07:38 |
| *** egonzalez has joined #openstack-kolla | 07:39 | |
| *** manheim has quit IRC | 07:42 | |
| *** manheim has joined #openstack-kolla | 07:43 | |
| *** rmart04 has joined #openstack-kolla | 07:49 | |
| *** hachi has quit IRC | 07:52 | |
| *** asmaa has quit IRC | 07:52 | |
| *** athomas has joined #openstack-kolla | 07:56 | |
| openstackgerrit | Merged openstack/kolla-ansible master: Configure coordination_url for cloudkitty processor https://review.openstack.org/455030 | 07:58 |
| *** blallau has joined #openstack-kolla | 08:03 | |
| *** Jezekus has joined #openstack-kolla | 08:04 | |
| *** asmaa has joined #openstack-kolla | 08:05 | |
| *** hachi has joined #openstack-kolla | 08:05 | |
| *** nathharp has quit IRC | 08:06 | |
| *** hieulq has joined #openstack-kolla | 08:06 | |
| *** nathharp has joined #openstack-kolla | 08:08 | |
| *** nathharp has quit IRC | 08:10 | |
| *** nathharp has joined #openstack-kolla | 08:10 | |
| openstackgerrit | Steven Dake proposed openstack/kolla-kubernetes master: Will need to squash this later if this works https://review.openstack.org/455132 | 08:12 |
| openstackgerrit | shaofeng cheng proposed openstack/kolla-ansible master: Update docker_py_version_min version https://review.openstack.org/454561 | 08:13 |
| openstackgerrit | Dai Dang Van proposed openstack/kolla-ansible master: Use utf8_general_ci collation as a default collation https://review.openstack.org/455154 | 08:23 |
| *** sayantan_ has quit IRC | 08:25 | |
| *** asmaa has quit IRC | 08:28 | |
| *** hachi has quit IRC | 08:28 | |
| *** guoguo has joined #openstack-kolla | 08:32 | |
| openstackgerrit | Dai Dang Van proposed openstack/kolla-ansible master: Use utf8_general_ci collation as a default collation https://review.openstack.org/455154 | 08:37 |
| openstackgerrit | Dai Dang Van proposed openstack/kolla-ansible master: Use utf8_general_ci collation as a default collation https://review.openstack.org/455154 | 08:39 |
| *** asmaa has joined #openstack-kolla | 08:41 | |
| *** hachi has joined #openstack-kolla | 08:41 | |
| daidv | Jeffrey4l, I saw one your commit (https://review.openstack.org/#/c/365428/) related to mariadb collation. | 08:42 |
| daidv | So, can I ask your some question around default database collation ? | 08:42 |
| daidv | s/your/you | 08:43 |
| hrw | morning | 08:45 |
| *** mgoddard has joined #openstack-kolla | 08:46 | |
| daidv | hrw, hi. | 08:46 |
| hrw | any reviewers with spare time? https://review.openstack.org/#/q/owner:%22Marcin+Juszkiewicz%22+status:open waits for extra eyes | 08:47 |
| sdake | holy batman - placement api working with condition jascott1 !! | 08:50 |
| *** nathharp has quit IRC | 08:54 | |
| hrw | duonghq: thx | 08:56 |
| openstackgerrit | Martin André proposed openstack/kolla master: Only copy plugin-archive into image for source builds https://review.openstack.org/455166 | 08:56 |
| openstackgerrit | Martin André proposed openstack/kolla master: Do not create empty archives https://review.openstack.org/455167 | 08:56 |
| openstackgerrit | Martin André proposed openstack/kolla master: Clarify documentation around plugins and additions archives https://review.openstack.org/455168 | 08:56 |
| duonghq | hrw, np | 09:03 |
| *** nathharp has joined #openstack-kolla | 09:10 | |
| openstackgerrit | Steven Dake proposed openstack/kolla-kubernetes master: Placement support in nova service chart https://review.openstack.org/453846 | 09:11 |
| openstackgerrit | Surya Prakash (spsurya) proposed openstack/kolla-ansible master: Implement Keystone zero-downtime upgrade https://review.openstack.org/425446 | 09:12 |
| openstackgerrit | Merged openstack/kolla-ansible master: Fix documentation for kolla_docker image pulling https://review.openstack.org/454266 | 09:13 |
| mandre | hey, anyone has objection me merging https://review.openstack.org/#/c/440534/? | 09:18 |
| hrw | mandre: I think that it got 3x +2 for a reason. | 09:19 |
| hrw | LGTM | 09:19 |
| mandre | hrw: I know... It just seems like good manner to check before self-approving a change :) | 09:21 |
| guoguo | Hi ,I run kolla-ansible, TASK [solum : Running Solum bootstrap container] "msg": "Container exited with non-zero return code" | 09:21 |
| guoguo | anyone can help me | 09:21 |
| spsurya | egonzalez: around ? | 09:22 |
| egonzalez | spsurya, yep | 09:23 |
| *** athomas has quit IRC | 09:23 | |
| egonzalez | guoguo, can you check logs from docker logs bootstrap_solum? | 09:23 |
| spsurya | I was remoning the merge conflict | 09:24 |
| mandre | guoguo: you should get the container logs to see why it failed, try connecting to your host then 'docker logs bootstrap_solum' | 09:24 |
| *** dixiaoli has quit IRC | 09:24 | |
| spsurya | egonzalez: will it go for pike ? | 09:24 |
| egonzalez | yep, is an upgrade task from ocata to pike | 09:24 |
| spsurya | egonzalez: i was thinking that once i will remove the conflict it will be ready for review | 09:25 |
| spsurya | is my understanding correct ? | 09:25 |
| spsurya | and once the review will be completed it can be merge | 09:26 |
| egonzalez | we need to test a bit more, since serial was removed by default. maybe we need to put serial in some tasks to have a good upgrade orchestration | 09:26 |
| *** iceyao has quit IRC | 09:26 | |
| *** dixiaoli has joined #openstack-kolla | 09:26 | |
| guoguo | i will try | 09:27 |
| guoguo | tks | 09:28 |
| *** sambetts|afk is now known as sambetts | 09:29 | |
| *** athomas has joined #openstack-kolla | 09:32 | |
| spsurya | egonzalez: so removing the merge conflict | 09:32 |
| spsurya | Should i go for this ? | 09:32 |
| egonzalez | spsurya, yep. when I have time and hw resources I will test multinode upgrades | 09:33 |
| spsurya | egonzalez: thanks | 09:34 |
| spsurya | egonzalez: if i will arrange the hw resources i will late you know. Then you can also try | 09:35 |
| *** nathharp has quit IRC | 09:35 | |
| egonzalez | spsurya, the tests I made is , deploy ocata, pull pike/master images | 09:36 |
| egonzalez | use rally/tempest or a `for` to create load in keystone, then upgrade while tests are running | 09:37 |
| *** nathharp has joined #openstack-kolla | 09:37 | |
| egonzalez | spsurya, only a small downtime must happen during db contract phase (at least in keystone) | 09:38 |
| spsurya | egonzalez: roger | 09:39 |
| openstackgerrit | Surya Prakash (spsurya) proposed openstack/kolla-ansible master: Implement Keystone zero-downtime upgrade https://review.openstack.org/425446 | 09:42 |
| spsurya | duonghq: please would be great you check it again | 09:44 |
| openstackgerrit | Merged openstack/kolla master: kubernetes: disable for architectures other than x86-64 https://review.openstack.org/445638 | 09:44 |
| *** haplo37 has quit IRC | 09:44 | |
| sdake | mandre if there are 2 +2's from a reviewer other then yourself, yo ucan self +2/+w | 09:44 |
| sdake | mandre all that means is that the reviewers were waiting on the gate | 09:44 |
| sdake | mandre although generally yo ushould recheck the job to make sure the gate is green | 09:45 |
| sdake | rather review | 09:45 |
| duonghq | spsurya, you should update according to egonzalez comment :) | 09:46 |
| *** tovin07_ has quit IRC | 09:47 | |
| *** tovin07 has quit IRC | 09:48 | |
| guoguo | I found error.INFO:__main__:Loading config file at /var/lib/kolla/config_files/config.json | 09:50 |
| guoguo | INFO:__main__:Validating config file | 09:50 |
| guoguo | INFO:__main__:Kolla config strategy set to: COPY_ALWAYS | 09:50 |
| guoguo | INFO:__main__:Copying service configuration files | 09:50 |
| guoguo | INFO:__main__:Coping file from /var/lib/kolla/config_files/solum.conf to /etc/kolla/solum.conf | 09:50 |
| guoguo | INFO:__main__:Setting file /etc/kolla/solum.conf owner to solum:solum | 09:50 |
| guoguo | INFO:__main__:Setting file /etc/kolla/solum.conf permission to 0644 | 09:50 |
| guoguo | INFO:__main__:Writing out command to execute | 09:50 |
| guoguo | Traceback (most recent call last): | 09:50 |
| guoguo | File "/var/lib/kolla/venv/bin/solum-db-manage", line 10, in <module> | 09:50 |
| guoguo | sys.exit(main()) | 09:50 |
| guoguo | File "/var/lib/kolla/venv/lib/python2.7/site-packages/solum/cmd/db_manage.py", line 99, in main | 09:50 |
| guoguo | CONF(project='solum') | 09:50 |
| guoguo | File "/var/lib/kolla/venv/lib/python2.7/site-packages/oslo_config/cfg.py", line 2352, in __call__ | 09:50 |
| guoguo | raise ConfigFilesNotFoundError(self._namespace._files_not_found) | 09:50 |
| guoguo | oslo_config.cfg.ConfigFilesNotFoundError: Failed to find some config files: /etc/solum/solum.conf | 09:50 |
| spsurya | duonghq: i am just removing merge conflict ? | 09:51 |
| spsurya | do i missing something | 09:51 |
| spsurya | ? | 09:51 |
| spsurya | kept the conflict changes too this time | 09:51 |
| spsurya | :( | 09:52 |
| sdake | guoguo pastebinit or fpaste plz :) | 09:52 |
| *** haplo37 has joined #openstack-kolla | 09:53 | |
| *** kolla-slack has quit IRC | 09:53 | |
| *** kolla-slack has joined #openstack-kolla | 09:54 | |
| *** tovin07_ has joined #openstack-kolla | 09:56 | |
| hrw | hm. can someone help me decode error? http://logs.openstack.org/00/453700/2/gate/gate-kolla-python27-ubuntu-xenial/15cd6c0/ | 09:57 |
| egonzalez | guoguo, is there a solum.conf file in /etc/kolla/solum-api/ ? | 10:00 |
| guoguo | egonzalez: no | 10:00 |
| *** iceyao has joined #openstack-kolla | 10:01 | |
| guoguo | solum.conf file in /etc/kolla/solum-api/ | 10:01 |
| guoguo | no /etc/kolla/solum.conf | 10:01 |
| guoguo | egonzalez, how can i do? | 10:02 |
| sdake | egonzalez quick q for you | 10:02 |
| sdake | egonzalez do you knwo if someone took care to use the stable rdo ocata repositories | 10:02 |
| sdake | prior to the release of ocata? | 10:03 |
| openstackgerrit | Martin André proposed openstack/kolla master: Move shebang customization into jinja2 block https://review.openstack.org/454637 | 10:04 |
| hrw | duonghq, egonzalez: can you vote/workflow for https://review.openstack.org/430940 (non-x86 support) one? I think that it is most important one for me | 10:04 |
| *** tovin07_ has quit IRC | 10:04 | |
| egonzalez | sdake, as far i know yes, ocata is using stable repos | 10:05 |
| egonzalez | let me double check | 10:05 |
| sdake | egonzalez PLZ :) | 10:06 |
| spsurya | guys btb | 10:08 |
| openstackgerrit | Merged openstack/kolla master: Defer handling of /var/lib/neutron/kolla to kolla API https://review.openstack.org/440534 | 10:10 |
| mandre | sdake: yep you're right, I should have hit recheck before approving (although the CI failure was unrelated) | 10:11 |
| duonghq | spsurya, okay | 10:12 |
| sdake | mandre its all good - just something to remember for next time | 10:12 |
| mandre | sdake: I just think cores were shy to approve the change, all votes came after the CI returned :) | 10:12 |
| sdake | mandre who knows | 10:12 |
| *** Jezekus has quit IRC | 10:19 | |
| *** hieulq has quit IRC | 10:19 | |
| *** iceyao has quit IRC | 10:21 | |
| egonzalez | sdake, yep, ocata is using stable repos http://paste.openstack.org/show/606032/ | 10:22 |
| sdake | egonzalez nice i'm glad someone else besides me figured that out :) | 10:22 |
| egonzalez | sdake, Jeffrey4l figured that out hehe | 10:23 |
| sdake | ya its pretty straightforward | 10:23 |
| *** cuongnv has quit IRC | 10:26 | |
| guoguo | how to send a public message to someone? | 10:28 |
| sdake | guoguo type their nick and your message | 10:28 |
| guoguo | I first use HexChat. | 10:28 |
| sdake | if you want to send a private msg use /query nick | 10:29 |
| sdake | and then type with them | 10:29 |
| *** gfidente has joined #openstack-kolla | 10:29 | |
| *** gfidente has quit IRC | 10:29 | |
| *** gfidente has joined #openstack-kolla | 10:29 | |
| guoguo | mandre Are you there? | 10:29 |
| guoguo | egonzalez: I mkdir /etc/solum and copy solum.conf from solum-api,and rerun deploy | 10:30 |
| guoguo | I want a fix method | 10:31 |
| egonzalez | guoguo, should be done automatically. :/ | 10:31 |
| guoguo | must be manual | 10:31 |
| egonzalez | guoguo, can you share deploy logs in the part where solum role is applied? | 10:31 |
| *** hieulq has joined #openstack-kolla | 10:33 | |
| *** daidv has quit IRC | 10:33 | |
| guoguo | egonzalez, how can i find logs. | 10:33 |
| egonzalez | the output of the deployment where it fails | 10:34 |
| guoguo | egonzalez,TASK [solum : Running Solum bootstrap container] "msg": "Container exited with non-zero return code" | 10:37 |
| guoguo | egonzalez,oslo_config.cfg.ConfigFilesNotFoundError: Failed to find some config files: /etc/solum/solum.conf | 10:37 |
| egonzalez | guoguo, previous logs, where folders should be created | 10:37 |
| guoguo | egonzalez, /etc/solum/ | 10:39 |
| guoguo | I should be /etc/kolla/solum-api/ | 10:39 |
| guoguo | egonzalez, I think /etc/kolla/solum-api/ folder shoud be used in bootstrap_solum | 10:40 |
| egonzalez | guoguo, logs like this http://paste.openstack.org/show/606033/ but for solum | 10:40 |
| openstackgerrit | XieYingYun proposed openstack/kolla master: Add Apache License Content in index.rst https://review.openstack.org/455232 | 10:43 |
| sdake | hey for folks that want to see how kolla-kubernetes works (as of february) check out: https://youtu.be/rHCCUP2odd8 | 10:45 |
| sdake | the deliverable has improved since then | 10:45 |
| sdake | need core reviewers :) | 10:45 |
| guoguo | egonzalez: TASK [solum : include] ********************************************************* | 10:46 |
| guoguo | [0;36mincluded: /home/kolla-ansible/ansible/roles/solum/tasks/bootstrap.yml for control01, control02, control03[0m | 10:46 |
| guoguo | TASK [solum : Creating Solum database] ***************************************** | 10:46 |
| guoguo | [0;33mchanged: [control01 -> control01][0m | 10:46 |
| guoguo | TASK [solum : Creating Solum database user and setting permissions] ************ | 10:46 |
| guoguo | [0;33mchanged: [control01 -> control01][0m | 10:46 |
| sdake | egonzalez does ironic with 4.0.0 images require the placement api? | 10:46 |
| guoguo | TASK [solum : include] ********************************************************* | 10:46 |
| guoguo | [0;36mincluded: /home/kolla-ansible/ansible/roles/solum/tasks/bootstrap_service.yml for control01, control02, control03[0m | 10:46 |
| guoguo | TASK [solum : Running Solum bootstrap container] ******************************* | 10:46 |
| sdake | guoguo use a past eservice plz | 10:46 |
| guoguo | [0;31mfatal: [control01 -> control01]: FAILED! => {"changed": true, "failed": true, "msg": "Container exited with non-zero return code"}[0m | 10:46 |
| *** iceyao has joined #openstack-kolla | 10:47 | |
| guoguo | sdake, yes, beore i make /etc/solum | 10:47 |
| egonzalez | guoguo, please, copy full trace for solum role in paste.openstack.org | 10:48 |
| *** dixiaoli has quit IRC | 10:48 | |
| guoguo | egonzalez, ok, i will make this error again, and paste to openstack.org | 10:49 |
| *** guoguo has quit IRC | 10:50 | |
| *** iceyao has quit IRC | 10:51 | |
| *** duonghq has quit IRC | 10:53 | |
| *** rhallisey has joined #openstack-kolla | 10:58 | |
| spsurya | egonzalez: corrected the merge conflict | 11:06 |
| *** shardy is now known as shardy_lunch | 11:06 | |
| spsurya | passed the jenkins | 11:06 |
| spsurya | :) | 11:06 |
| *** deadnull has quit IRC | 11:10 | |
| *** yangyapeng has joined #openstack-kolla | 11:19 | |
| *** yangyapeng has quit IRC | 11:20 | |
| *** asalkeld has quit IRC | 11:24 | |
| *** iceyao has joined #openstack-kolla | 11:33 | |
| *** Jezekus has joined #openstack-kolla | 11:40 | |
| berendt | has anybody already tested the new 17.03.0 and 17.04.0 release of docker with kolla? | 11:45 |
| Mech422 | sdake: dam - yer up late/early | 11:54 |
| Daviey | Hey, anyone used external iscsi (such as Netapp) with Kolla here? | 11:55 |
| berendt | Daviey: using the netapp unified cinder driver? | 11:56 |
| Daviey | berendt: yeah | 11:56 |
| berendt | Daviey: not yet, at the moment we are preparing a customer site for a similar use case (nfs instead of iscsi) | 11:56 |
| berendt | Daviey: i think there are some issue with the cinder-volume image and the netapp unified driver at the moment (at least when using source images) | 11:57 |
| openstackgerrit | Eduardo Gonzalez proposed openstack/kolla-ansible master: WIP: Implement blazar ansible role https://review.openstack.org/454752 | 12:02 |
| *** iniazi has quit IRC | 12:04 | |
| openstackgerrit | Eduardo Gonzalez proposed openstack/kolla master: Implement blazar Docker images https://review.openstack.org/454745 | 12:05 |
| Daviey | berendt: that was my expectation... doc's and bugs are not very verbose | 12:06 |
| openstackgerrit | Zeyu Zhu proposed openstack/kolla-ansible master: Remove the redundant variable https://review.openstack.org/448433 | 12:06 |
| *** iniazi has joined #openstack-kolla | 12:06 | |
| *** shardy_lunch is now known as shardy | 12:07 | |
| openstackgerrit | Martin André proposed openstack/kolla master: Skip [plugins,additions]-archive for binary builts https://review.openstack.org/455167 | 12:08 |
| openstackgerrit | Martin André proposed openstack/kolla master: Clarify documentation around plugins and additions archives https://review.openstack.org/455168 | 12:08 |
| *** Serlex has quit IRC | 12:10 | |
| openstackgerrit | Zeyu Zhu proposed openstack/kolla-kubernetes master: Update the version of kubernetes https://review.openstack.org/455296 | 12:12 |
| *** zhuzeyu has quit IRC | 12:15 | |
| *** MarginHu has quit IRC | 12:17 | |
| *** MarginHu has joined #openstack-kolla | 12:17 | |
| *** zhurong has quit IRC | 12:26 | |
| *** eaguilar has joined #openstack-kolla | 12:32 | |
| *** krtaylor has quit IRC | 12:34 | |
| *** krtaylor has joined #openstack-kolla | 12:37 | |
| *** eaguilar has quit IRC | 12:42 | |
| *** eaguilar has joined #openstack-kolla | 12:44 | |
| *** eaguilar has quit IRC | 12:48 | |
| *** rmart04_ has joined #openstack-kolla | 12:55 | |
| *** rmart04 has quit IRC | 12:56 | |
| *** rmart04_ is now known as rmart04 | 12:56 | |
| *** rwallner has joined #openstack-kolla | 13:05 | |
| *** rmart04 has quit IRC | 13:05 | |
| *** rwallner has quit IRC | 13:06 | |
| *** rwallner has joined #openstack-kolla | 13:07 | |
| *** rmart04 has joined #openstack-kolla | 13:08 | |
| *** rmart04 has quit IRC | 13:10 | |
| *** targon has quit IRC | 13:10 | |
| *** gkadam has quit IRC | 13:10 | |
| *** rhallisey has quit IRC | 13:14 | |
| *** rhallisey has joined #openstack-kolla | 13:16 | |
| *** esharao has joined #openstack-kolla | 13:16 | |
| *** rhallisey has quit IRC | 13:18 | |
| *** iceyao has quit IRC | 13:18 | |
| *** rwellum has joined #openstack-kolla | 13:22 | |
| *** rmart04 has joined #openstack-kolla | 13:23 | |
| *** Administrator_ has quit IRC | 13:23 | |
| *** Administrator_ has joined #openstack-kolla | 13:24 | |
| *** eanylin has quit IRC | 13:25 | |
| *** rmart04 has quit IRC | 13:25 | |
| *** rmart04 has joined #openstack-kolla | 13:28 | |
| *** rmart04 has quit IRC | 13:32 | |
| *** rmart04 has joined #openstack-kolla | 13:32 | |
| *** openstackgerrit has quit IRC | 13:33 | |
| mnaser | anyone ran into this before? | 13:34 |
| mnaser | 2017-04-10 13:25:44.696+0000: 23638: error : qemuProcessReportLogError:1802 : internal error: qemu unexpectedly closed the monitor: Could not access KVM kernel module: Permission denied | 13:34 |
| *** Serlex has joined #openstack-kolla | 13:34 | |
| berendt | mnaser: yes, i had this issue on a development in the past | 13:34 |
| mnaser | i assume i just need to chmod /dev/kvm | 13:34 |
| mnaser | but i feel like that isnt "right" | 13:34 |
| berendt | mnaser: i think i added docker to the kvm group as a workaround | 13:34 |
| mnaser | berendt https://github.com/openstack/kolla-ansible/blob/master/ansible/roles/baremetal/tasks/post-install.yml#L42-L47 | 13:36 |
| mnaser | i think that might be it | 13:36 |
| mnaser | bleh that path doesnt exist on my host | 13:36 |
| *** rmart04_ has joined #openstack-kolla | 13:37 | |
| *** jtriley has joined #openstack-kolla | 13:38 | |
| *** rmart04 has quit IRC | 13:38 | |
| *** rmart04_ is now known as rmart04 | 13:38 | |
| *** eanylin has joined #openstack-kolla | 13:41 | |
| *** jtriley has quit IRC | 13:41 | |
| mnaser | it seems the permissions are correct in the container | 13:41 |
| mnaser | docker exec nova_libvirt ls -alh /dev/kvm => crw-rw---- 1 root qemu 10, 232 Apr 9 21:21 /dev/kvm | 13:42 |
| *** mbruzek has joined #openstack-kolla | 13:42 | |
| *** mbruzek has joined #openstack-kolla | 13:42 | |
| berendt | perms of /dev/kvm on the host? | 13:42 |
| blallau | @mnaser I have the same issue on Ubuntu using virt_type=kvm driver | 13:43 |
| blallau | it works using qemu | 13:43 |
| mnaser | blallau exactly the same case | 13:43 |
| mnaser | yeah but then you're not using kvm which will slow things down, a lot.. | 13:43 |
| blallau | for sure ;) | 13:44 |
| mnaser | berendt does it matter what it is on the host? fwiw: crw-rw---- 1 root 42427 10, 232 Apr 9 21:21 /dev/kvm | 13:44 |
| mnaser | blallau ocata? | 13:44 |
| blallau | yes Ocata and master | 13:44 |
| blallau | I try to tackle this issue but I didn't success ... :( | 13:44 |
| mnaser | this isn't good | 13:44 |
| mnaser | because we only test for qemu | 13:45 |
| mnaser | i guess we never detected this | 13:45 |
| blallau | Yes I know | 13:45 |
| mnaser | do you have a bug filed blallau ? | 13:45 |
| blallau | No cause I think It was a config issue On my side , I though I was alone with this issue | 13:46 |
| blallau | So you can open I bug :) | 13:46 |
| blallau | I have made many test with/without fix b7f8abac296965096d770ee61f8533a162fcb7e4 (Make qemu use nova user) | 13:48 |
| blallau | without apparmor .. | 13:48 |
| blallau | changing permissions and users on /dev/kvm ... always the same issue | 13:48 |
| mnaser | i think i know what the issue is | 13:50 |
| mnaser | hm, maybe not | 13:51 |
| *** yangyapeng has joined #openstack-kolla | 13:52 | |
| esharao | morning folks | 13:52 |
| rwellum | Morning | 13:53 |
| esharao | tried with ubuntu over the weekend.. got my services running but ran into same issue with placement.. | 13:53 |
| esharao | having install_type: "source" in globals.yaml did not fix the placement issue | 13:54 |
| *** tovin07 has joined #openstack-kolla | 13:58 | |
| mnaser | esharao same issue as in /dev/kvm ? | 13:59 |
| esharao | same issue as reported by sdake on centos - the nova-placement-api is taking /usr/bin instaed of the /var/lib/kolla/venv/bin/ | 14:00 |
| *** jtriley has joined #openstack-kolla | 14:00 | |
| *** qiliang27 has joined #openstack-kolla | 14:01 | |
| *** shashank_t_ has joined #openstack-kolla | 14:01 | |
| *** openstackgerrit has joined #openstack-kolla | 14:02 | |
| openstackgerrit | Merged openstack/kolla-kubernetes master: Update to Helm v2.3.0 https://review.openstack.org/454380 | 14:02 |
| *** rhallisey has joined #openstack-kolla | 14:03 | |
| *** dave-mccowan has joined #openstack-kolla | 14:04 | |
| *** shashank_t_ has quit IRC | 14:06 | |
| *** dave-mccowan has quit IRC | 14:09 | |
| mnaser | i think this is related to the issue | 14:10 |
| mnaser | https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1637601 | 14:10 |
| openstack | Launchpad bug 1637601 in libvirt (Ubuntu Xenial) "UbuntuKVM: migration using NFS mount fails #190" [Medium,Fix released] | 14:10 |
| *** rhallisey has quit IRC | 14:12 | |
| *** rhallisey has joined #openstack-kolla | 14:13 | |
| blallau | @mnaser why do you think it's related? | 14:15 |
| mnaser | i think libvirt is trying to launch the vms under libvirt-qemu user | 14:16 |
| *** n0isyn0ise has joined #openstack-kolla | 14:16 | |
| *** jtriley has quit IRC | 14:17 | |
| *** mbruzek has quit IRC | 14:26 | |
| *** lucasxu has joined #openstack-kolla | 14:27 | |
| *** jtriley has joined #openstack-kolla | 14:28 | |
| *** Jezekus has quit IRC | 14:29 | |
| manheim | hi, if i've deployed kolla master, do I have to do any post-deploy setup related to placement? what if I add a new compute? | 14:31 |
| sdake | esharao for that case, you must set tyhe type to source | 14:32 |
| sdake | mornign folks | 14:32 |
| sdake | can I get a critical eye on https://review.openstack.org/#/c/453846/21 | 14:32 |
| esharao | sdake morning .. it is set to source | 14:32 |
| esharao | based on the latest guide | 14:32 |
| sdake | esharao in /etc/kolla/globals.yml | 14:32 |
| sdake | esharao i think the guide needs an adjustment | 14:32 |
| esharao | Hmm | 14:32 |
| sdake | esharao basiclly we hae two config files | 14:32 |
| sdake | we need to merge that into one | 14:32 |
| sdake | rwellum any chance you could review https://review.openstack.org/#/c/453846/21 plz | 14:33 |
| esharao | sdake agree | 14:33 |
| sdake | esharao cloud.yaml is the one config file to rule them all | 14:33 |
| sdake | globals.yml is temporary | 14:33 |
| esharao | also the 439740 needs a change in pathfinder to include /usr/local/share for ubuntu | 14:33 |
| *** targon has joined #openstack-kolla | 14:33 | |
| *** mbruzek has joined #openstack-kolla | 14:34 | |
| *** mbruzek has quit IRC | 14:34 | |
| *** mbruzek has joined #openstack-kolla | 14:34 | |
| esharao | sure.. i made sure both places (for now) had install type as source | 14:34 |
| mnaser | blallau berendt https://bugs.launchpad.net/kolla/+bug/1681461 | 14:34 |
| openstack | Launchpad bug 1681461 in kolla "Instance fails to boot: Could not access KVM kernel module: Permission denied" [Undecided,Confirmed] - Assigned to Mohammed Naser (mnaser) | 14:34 |
| mnaser | i think i have founded it | 14:34 |
| mnaser | s/ed// | 14:34 |
| *** shashank_t_ has joined #openstack-kolla | 14:34 | |
| *** fooliouno has quit IRC | 14:35 | |
| sdake | folks: moar gates for kolla-kubernetes: https://review.openstack.org/#/c/455170/ | 14:38 |
| sdake | esharao its next up | 14:38 |
| sdake | although I am traveling this week | 14:38 |
| *** shashank_t_ has quit IRC | 14:38 | |
| mnaser | oh | 14:39 |
| mnaser | would you look at that | 14:39 |
| mnaser | its fixed already | 14:39 |
| mnaser | it just wasnt backported to stable/ocata | 14:39 |
| mnaser | oh, it has | 14:39 |
| blallau | b7f8abac296965096d770ee61f8533a162fcb7e4 | 14:39 |
| mnaser | yep | 14:39 |
| mnaser | did you have that fix in blallau ? | 14:39 |
| blallau | this is what I told you before, I have it and it doesn't work | 14:40 |
| blallau | maybe you can test it on your side | 14:40 |
| blallau | I am not on baremetal (I use nested KVM) | 14:41 |
| *** tovin07 has left #openstack-kolla | 14:41 | |
| mnaser | sorry i didnt see that | 14:42 |
| openstackgerrit | sean mooney proposed openstack/kolla-ansible master: introduce contrib playbook for ovs-dpdk https://review.openstack.org/408872 | 14:42 |
| *** satyar has joined #openstack-kolla | 14:42 | |
| sdake | vhttps://www.wired.com/2017/04/ever-heard-startup-deis-well-microsoft-certainly/ | 14:45 |
| blallau | @mnaser If you use baremetal it worse testing b7f8abac296965096d770ee61f8533a162fcb7e4. Maybe nested KVM introduced another issue | 14:46 |
| mnaser | blallau cherry picked, its working | 14:46 |
| blallau | you don't have bug anymore ??? | 14:47 |
| *** srwilkers has joined #openstack-kolla | 14:48 | |
| sdake | srwilkers would you take a look at https://www.wired.com/2017/04/ever-heard-startup-deis-well-microsoft-certainly/ plz | 14:49 |
| sdake | srwilkers wrong link | 14:49 |
| sdake | srwilkers https://review.openstack.org/#/c/453846/21 | 14:49 |
| srwilkers | morning sdake | 14:49 |
| sdake | morning fine sir | 14:49 |
| sdake | i think specifically something is wrong with this shell code: https://review.openstack.org/#/c/453846/21/tests/bin/deploy_iscsi_common.sh | 14:51 |
| sdake | the conditional doesn't seem to be executing | 14:51 |
| hrw | dear reviewers (if you are here and/or alive): can you go to my patches and just -2 them without reading so I will know that they are not worth looking at and can move to other projects? I know that you have own lives, own schedules, deadlines etc. I am just tired of looking at that list of simple patches which change nothing for 99% of kolla users waiting for some reaction from people. | 14:54 |
| * hrw ends a day | 14:54 | |
| *** shashank_t_ has joined #openstack-kolla | 14:55 | |
| blallau | /query mnaser | 14:56 |
| mnaser | o/ blallau its working but its because i was using nova-libvirt from ocata (to get latest libvirt etc) | 14:57 |
| mnaser | and this was related to it | 14:57 |
| *** rmart04 has quit IRC | 14:58 | |
| blallau | get the latest nova-libvirt images from Kolla project, this it what you mean? | 14:59 |
| mnaser | nova-libvirt 4.0.0 with nova-compute 3.0.3 | 14:59 |
| sdake | hrw link plz | 15:00 |
| blallau | I was using the latest ones 4.0.0 for each | 15:00 |
| hrw | sdake: https://review.openstack.org/#/q/owner:marcin%2540juszkiewicz.com.pl+status:open | 15:00 |
| sdake | hrw we have a very large backlog - its not personal | 15:00 |
| hrw | sdake: I am aware. they are 2 months old now | 15:01 |
| *** jtriley has quit IRC | 15:01 | |
| sdake | hrw one thing that reviews look at when looking at a patch is "is the gate green" | 15:01 |
| sdake | in this case some are not | 15:01 |
| *** manheim has quit IRC | 15:01 | |
| sdake | can you type in recheck | 15:01 |
| hrw | sdake: I am thinking of making just a manageable fork with all my changes | 15:01 |
| sdake | i am getting on a plane in 30 minutes | 15:02 |
| sdake | I will review your changes then | 15:02 |
| sdake | hrw no reason to fork the repo - better to have in master | 15:02 |
| hrw | sdake: I am trying to recheck them not more than once per day. most of times it is set of unrelated failures due to random issues | 15:02 |
| sdake | hrw just recheck until they are green | 15:02 |
| sdake | or mostly green | 15:02 |
| hrw | sdake: and yes, I am aware that there are issues with gates and yes there are too small amount of people to look at it | 15:02 |
| sdake | i htink the ubuntu binary job is failing | 15:02 |
| mnaser | hrw i think the issue is that it's a lot harder for someone to review when it has a bunch of failures | 15:03 |
| hrw | sdake: centos failed due to RDO repos for example | 15:03 |
| mnaser | i think all devs should take ownership of fixing gate issues | 15:03 |
| mnaser | yeah im not sure why its giving a 403 | 15:03 |
| hrw | mnaser: those patches are NOT related to each other. not anymore | 15:03 |
| *** daidv has joined #openstack-kolla | 15:03 | |
| mnaser | hrw i agree with you! and i understand your frustration, i have a long queue of verified things with zero reviews that are trivial but people will get to it im sure | 15:04 |
| sdake | hrw i agree that all devs should take ownership of gates | 15:04 |
| sdake | during my stint as ptl I did extensive training on gate mainteannce | 15:04 |
| hrw | sdake: 'better to have in master' yes. but better to have something working now in fork than something in master in october | 15:04 |
| mnaser | now i just pinged folks at #rdo about the centos failures | 15:05 |
| sdake | that 403 was probalby transitory | 15:05 |
| sdake | i'd try a recheck | 15:05 |
| mnaser | sdake its not | 15:05 |
| mnaser | i just ran it now and its giving a 403 | 15:05 |
| satyar | Hi All | 15:05 |
| mnaser | (a wget on that) | 15:05 |
| *** fooliouno has joined #openstack-kolla | 15:05 | |
| satyar | i got a way to enable ssl for novnc proxy with ssl termination at HAproxy | 15:06 |
| satyar | should i log a blueprint/launchpad bug to push the code? | 15:06 |
| satyar | Please let me know | 15:07 |
| *** Pavo has joined #openstack-kolla | 15:07 | |
| * hrw -> creating fork anyway as I need it for my work and to show that this set of separate patches has sense | 15:08 | |
| *** dosaboy has quit IRC | 15:10 | |
| sdake | hrw another hting that helps get your patches merged and keeps the review queue flowing is reviews of other peoples work | 15:10 |
| hrw | sdake: I went through those which I understood | 15:11 |
| sdake | hrw cool not making assumptions - just stating for the general channel :) | 15:11 |
| hrw | sure | 15:11 |
| *** lrensing has joined #openstack-kolla | 15:12 | |
| *** srwilkers has quit IRC | 15:12 | |
| *** jtriley has joined #openstack-kolla | 15:15 | |
| *** skramaja has quit IRC | 15:16 | |
| *** shashank_t_ has quit IRC | 15:17 | |
| openstackgerrit | Martin André proposed openstack/kolla master: Skip [plugins,additions]-archive for binary builts https://review.openstack.org/455167 | 15:18 |
| openstackgerrit | Martin André proposed openstack/kolla master: Clarify documentation around plugins and additions archives https://review.openstack.org/455168 | 15:18 |
| *** qiliang27 has quit IRC | 15:20 | |
| *** jrich523_ has joined #openstack-kolla | 15:20 | |
| *** qiliang27 has joined #openstack-kolla | 15:20 | |
| *** jrich523 has quit IRC | 15:21 | |
| *** jrich523_ has quit IRC | 15:21 | |
| *** jrich523 has joined #openstack-kolla | 15:22 | |
| *** dosaboy has joined #openstack-kolla | 15:23 | |
| *** MarginHu has quit IRC | 15:24 | |
| *** qiliang27 has quit IRC | 15:24 | |
| *** srwilkers has joined #openstack-kolla | 15:35 | |
| hrw | egonzalez: https://review.openstack.org/#/c/434817/ is green | 15:45 |
| hrw | shit, not that one. sorry | 15:45 |
| hrw | bjreh reh/ | 15:45 |
| * hrw ends a day for sure. too tired and do simple mistakes | 15:46 | |
| *** manheim has joined #openstack-kolla | 15:50 | |
| *** manheim has quit IRC | 15:55 | |
| *** sayantan_ has joined #openstack-kolla | 15:59 | |
| inc0 | good morning | 15:59 |
| *** sayantan_ has quit IRC | 15:59 | |
| *** lucasxu has quit IRC | 16:00 | |
| *** sayantan_ has joined #openstack-kolla | 16:00 | |
| *** egonzalez has quit IRC | 16:06 | |
| openstackgerrit | Mohammed Naser proposed openstack/kolla master: WIP: Switch to RDO proxy mirrors https://review.openstack.org/455374 | 16:11 |
| *** jrich523 has quit IRC | 16:13 | |
| *** jrich523 has joined #openstack-kolla | 16:17 | |
| openstackgerrit | Mohammed Naser proposed openstack/kolla master: WIP: Switch to Docker registry proxy https://review.openstack.org/455378 | 16:18 |
| qwang | morning | 16:21 |
| openstackgerrit | Paul Bourke (pbourke) proposed openstack/kolla-ansible master: Only start one glance_api when using file backend https://review.openstack.org/448654 | 16:22 |
| *** blallau has quit IRC | 16:23 | |
| esharao | sdake.. got my vm active and running on ubuntu as well with minor changes to the guide | 16:26 |
| *** blallau has joined #openstack-kolla | 16:27 | |
| *** rwallner has quit IRC | 16:28 | |
| spsurya | night all | 16:29 |
| *** nathharp has quit IRC | 16:29 | |
| openstackgerrit | Bertrand Lallau proposed openstack/kolla-ansible master: Remove all Ubuntu 14.04 related documentation https://review.openstack.org/453668 | 16:30 |
| *** lamt has joined #openstack-kolla | 16:30 | |
| *** blallau has quit IRC | 16:32 | |
| *** ntpttr_ has joined #openstack-kolla | 16:41 | |
| *** lamt has quit IRC | 16:42 | |
| *** m1dev has joined #openstack-kolla | 16:44 | |
| *** asmaa has quit IRC | 16:46 | |
| *** hachi has quit IRC | 16:46 | |
| *** g3ek has quit IRC | 16:47 | |
| openstackgerrit | Merged openstack/kolla master: gnocchi-base: add some devel packages for non-x86 https://review.openstack.org/453700 | 16:48 |
| *** unicell has quit IRC | 16:50 | |
| *** jaosorior is now known as jaosorior_away | 16:51 | |
| *** g3ek has joined #openstack-kolla | 16:56 | |
| *** eaguilar has joined #openstack-kolla | 16:57 | |
| *** targon has quit IRC | 16:57 | |
| *** sayantan_ has quit IRC | 16:58 | |
| *** rwallner has joined #openstack-kolla | 16:58 | |
| *** lucasxu has joined #openstack-kolla | 17:02 | |
| rwellum | mnaser: 455374 - failing python 2.7 check? | 17:03 |
| *** mgoddard has quit IRC | 17:03 | |
| *** yangyapeng has quit IRC | 17:13 | |
| mnaser | rwellum i have no idea why it did | 17:15 |
| mnaser | rwellum i have this feeeling that it might have to do with the source i replaced and some weird variables getting introduced | 17:15 |
| rwellum | Yeah I took a look and I don't see anything glaring out at me. | 17:15 |
| mnaser | rwellum http://git.openstack.org/cgit/openstack-infra/project-config/tree/nodepool/scripts/configure_mirror.sh#n41 -- cant imagine much different here | 17:16 |
| *** mbruzek has quit IRC | 17:17 | |
| *** bogdando has joined #openstack-kolla | 17:17 | |
| mnaser | ok well looks like we wont be able to test much changes while rdo builds are still broken | 17:18 |
| bogdando | folks, I can't find docs on extend_start for config.json examples. Please any help on that? | 17:18 |
| rwellum | True | 17:18 |
| bogdando | My use case is to use a custom /usr/local/bin/kolla_extend_start | 17:19 |
| mnaser | dont think there are docs other than plenty of examples in kolla-ansible bogdando | 17:19 |
| bogdando | mnaser: actually I'm looking the way to encode initial step to preceed the command: | 17:20 |
| *** eaguilar has quit IRC | 17:20 | |
| bogdando | specifically I have to source a given env file before to run the cmd | 17:20 |
| bogdando | mnaser: that's all I can find for kolla-ansible with hound http://git.openstack.org/cgit/openstack/kolla-ansible/tree/specs/logging-with-heka.rst | 17:21 |
| bogdando | so, is it possible to encode init steps for the config.json? | 17:22 |
| bogdando | perhaps it is kolla bootstrap, but can't find simple examples either ;) | 17:23 |
| mnaser | rwellum i think i will propose that we rely on current-passed-ci rather than current which can include build failures | 17:23 |
| mnaser | bogdando you would have to rebuild the container to do that i think | 17:23 |
| bogdando | what I want is: given etcd.json, I need to preceed the command: /usr/bin/etcd with sourcing an env file! | 17:23 |
| rwellum | mnaser: that's not a bad idea imo | 17:24 |
| bogdando | no rebuilds, no hacks if possible ) | 17:24 |
| bogdando | I can see the hack is to mount the custom /usr/local/bin/kolla_extend_start perhaps directly | 17:24 |
| mnaser | i think you'd have to rebuild so that kolla_extend_start changes | 17:24 |
| mnaser | otherwise i would propose that you make a change to the ansible etcd role | 17:25 |
| mnaser | and add a way to configure it | 17:25 |
| *** unicell has joined #openstack-kolla | 17:25 | |
| mnaser | so that the container is launched with specific env variables | 17:25 |
| bogdando | oh, I don't need ansible for my case | 17:25 |
| mnaser | so you dont have t source them inside of it | 17:25 |
| mnaser | the container gets them directly from docekr | 17:25 |
| mnaser | #mondaysoimnotspellling | 17:25 |
| *** lamt has joined #openstack-kolla | 17:28 | |
| sdake | sdakes on a plane | 17:34 |
| sdake | sup | 17:34 |
| openstackgerrit | Mohammed Naser proposed openstack/kolla master: WIP: Switch to RDO proxy mirrors https://review.openstack.org/455374 | 17:39 |
| mnaser | rwellum ^ i believe this should help | 17:39 |
| mnaser | fixing centos mirrors at the very least | 17:39 |
| mnaser | still gotta find out why python checks failed lol | 17:39 |
| openstackgerrit | Mohammed Naser proposed openstack/kolla master: WIP: Switch to Docker registry proxy https://review.openstack.org/455378 | 17:41 |
| mnaser | rwellum the python check went through, that's so strange | 17:42 |
| sdake | inc0 kfox1111 you peeps around | 17:45 |
| *** spiette has quit IRC | 17:45 | |
| kfox1111 | morning. | 17:46 |
| kfox1111 | sdake: whats up? | 17:46 |
| sdake | kfox1111 i worked on the service chart quite a bit, and one of the gates is failing in an odd way | 17:46 |
| sdake | i think the shell code is wrong | 17:46 |
| sdake | can you check it please | 17:46 |
| sdake | https://review.openstack.org/#/c/453846/21 | 17:47 |
| kfox1111 | got some links? | 17:47 |
| rwellum | mnaser: transitional I guess - weird! | 17:47 |
| sdake | kfox1111 ya - on a plane so sintenret is slow | 17:47 |
| sdake | kfox1111 ironic fails to work - the placement api is not launching on the 4.y.z images | 17:48 |
| sdake | (ironic source that is) | 17:48 |
| kolla-slack | <jascott1> https://deis.com/blog/2017/deis-to-join-microsoft/ | 17:48 |
| kolla-slack | <jascott1> :S | 17:48 |
| sdake | jascott1 welcome to 3 hrs ago ;-) | 17:48 |
| kfox1111 | so... the ironic one isn't useing the service pcakages though. | 17:49 |
| kfox1111 | or did that change? | 17:49 |
| sdake | kfox1111 right its not using the service pakages | 17:50 |
| kolla-slack | <jascott1> sdake it was PSA for west coast slackers that join at 11am :slightly_smiling_face: | 17:50 |
| sdake | kfox1111 https://review.openstack.org/#/c/453846/21/tests/bin/deploy_ironic.sh | 17:50 |
| kfox1111 | something in the review broke it? | 17:50 |
| sdake | kfox1111 that review was broke nto begin with in many ways | 17:50 |
| sdake | now it works prtty well except for the ironic shell is broken I htink | 17:51 |
| sdake | conditions are pretty cool feature of helm tho | 17:51 |
| sdake | jascott1 ^^ | 17:51 |
| kolla-slack | <jascott1> :) | 17:51 |
| rwellum | I don't see anything obvious sdake | 17:52 |
| kfox1111 | ok. in general, it looks good.... | 17:54 |
| kfox1111 | let me look at the logs. | 17:54 |
| kfox1111 | sdake: thats weird... | 17:55 |
| kfox1111 | I'm not seeing gate-kolla-kubernetes-deploy-centos-source-4-ironic-nv. | 17:56 |
| kfox1111 | thats the stable ironic test. | 17:56 |
| *** lamt has quit IRC | 17:56 | |
| kfox1111 | I only see ubuntu, which i think is known broken. | 17:56 |
| *** gfidente is now known as gfidente|afk | 17:56 | |
| *** bogdando has quit IRC | 17:58 | |
| *** sambetts is now known as sambetts|afk | 18:00 | |
| sdake | kfox1111 got dced (in the air) | 18:03 |
| sdake | kfox1111 this is acutally teh code that i think is wrong: https://review.openstack.org/#/c/453846/21/tests/bin/deploy_iscsi_common.sh | 18:03 |
| sdake | not sure what is wrong with the shell | 18:03 |
| openstackgerrit | Qin Wang (qwang) proposed openstack/kolla-kubernetes master: [WIP] Condition and tags in requirements.yaml for keystone service https://review.openstack.org/444461 | 18:04 |
| kfox1111 | sdake: where is the log file you are looking at? | 18:04 |
| sdake | kfox1111 http://logs.openstack.org/46/453846/21/experimental/gate-kolla-kubernetes-deploy-centos-source-4-ironic-nv/82559b6/ | 18:06 |
| sdake | kfox1111 pretty sure helm install (placement) isn't happening | 18:07 |
| sdake | kfox1111 and I have it conditonlized out to not deploy if branch!=2 -a branch!=3 | 18:07 |
| kfox1111 | http://logs.openstack.org/46/453846/21/experimental/gate-kolla-kubernetes-deploy-centos-source-4-ironic-nv/82559b6/logs/pods/ | 18:08 |
| kfox1111 | there's placement stuff. | 18:08 |
| sdake | rathe to deploy | 18:08 |
| sdake | hmm | 18:08 |
| sdake | kfox1111 yup it is indeed helm installing placement | 18:11 |
| *** ipsecguy_ has joined #openstack-kolla | 18:18 | |
| *** ipsecguy has quit IRC | 18:19 | |
| sbezverk | sdake: curious how you can say that the shell is wrong if it was perfectly working before you started playing with it ;) ?? | 18:22 |
| sdake | sbezverk my changes are wrong - not the original code | 18:23 |
| *** asmaa has joined #openstack-kolla | 18:23 | |
| sdake | sbezverk i was hoping someone would find it in a review | 18:23 |
| *** hachi has joined #openstack-kolla | 18:23 | |
| sdake | i spent 12 hours on that patch yesterday watching paint dry with zuul | 18:24 |
| sdake | tired | 18:24 |
| openstackgerrit | Mohammed Naser proposed openstack/kolla master: Switch to RDO proxy mirrors https://review.openstack.org/455374 | 18:27 |
| dmsimard | Uh oh, Microsoft acquired Helm | 18:31 |
| dmsimard | (Deis) | 18:31 |
| *** eaguilar has joined #openstack-kolla | 18:34 | |
| *** mbruzek has joined #openstack-kolla | 18:35 | |
| *** mbruzek has quit IRC | 18:35 | |
| *** mbruzek has joined #openstack-kolla | 18:35 | |
| sdake | kfox1111 what do you make of this: http://logs.openstack.org/46/453846/21/experimental/gate-kolla-kubernetes-deploy-centos-binary-2-helm-compute-kit-nv/99ff6de/console.html#_2017-04-10_15_33_55_326405 | 18:36 |
| sdake | kfox1111 someitmes it works sometimes it spits out that error | 18:36 |
| kfox1111 | sdake: weird. bug in helm? | 18:37 |
| sdake | could be | 18:37 |
| *** eaguilar has quit IRC | 18:45 | |
| kolla-slack | <jascott1> intermittent? weird | 18:46 |
| sbezverk | kfox1111: please check http://logs.openstack.org/46/453846/21/experimental/gate-kolla-kubernetes-deploy-centos-binary-2-helm-compute-kit-nv/99ff6de/logs/pods/ it looks like pods got stuck on fernet keys | 18:49 |
| *** athomas has quit IRC | 18:49 | |
| kfox1111 | sbezverk: how do you figure? | 18:50 |
| kfox1111 | sbezverk: I see it pulling a container for keystone, but it never ran. | 18:51 |
| kfox1111 | http://logs.openstack.org/46/453846/21/experimental/gate-kolla-kubernetes-deploy-centos-binary-2-helm-compute-kit-nv/99ff6de/logs/pods/kolla-keystone-fernet-setup-job-zm0s5.txt | 18:51 |
| sbezverk | kfox1111: I check a couple of containers they were waiting for fernet keys to bo mounted | 18:52 |
| *** rwallner has quit IRC | 18:52 | |
| kfox1111 | sbezverk: that may be. but the fernet key stuff is pending on docker pulling an image. | 18:52 |
| kfox1111 | not fernet related at all. | 18:52 |
| sbezverk | kfox1111: ok, got it.. | 18:53 |
| openstackgerrit | Kevin Fox proposed openstack/kolla-kubernetes master: WIP: v4 gates. https://review.openstack.org/454841 | 18:53 |
| kfox1111 | sdake: so, its possible that error message happens if a chart takes too long to start? | 18:54 |
| kfox1111 | though I didn't think it waited.... | 18:54 |
| *** athomas has joined #openstack-kolla | 18:57 | |
| *** tonanhngo has joined #openstack-kolla | 18:57 | |
| *** tonanhngo has quit IRC | 18:57 | |
| sbezverk | kfox1111: having --debug on compute chart is near crazy action | 18:57 |
| kfox1111 | sbezverk: I agree. | 18:57 |
| sbezverk | as it generates tons and tons of output | 18:58 |
| *** tonanhngo has joined #openstack-kolla | 18:58 | |
| sbezverk | that might contribute to the issue observed.. | 18:58 |
| kfox1111 | if someone wants to set the flag temporarily, then churn thorugh the result. thats ok with me. I just won't bless it to go into the main gate. | 18:58 |
| kfox1111 | ah. could be. | 18:58 |
| *** bogdando has joined #openstack-kolla | 19:01 | |
| *** eaguilar has joined #openstack-kolla | 19:02 | |
| *** tonanhngo has quit IRC | 19:03 | |
| *** rwallner has joined #openstack-kolla | 19:03 | |
| inc0 | mnaser: hey, around? | 19:13 |
| mnaser | inc0 o/ | 19:14 |
| inc0 | so I'm back into gate business | 19:14 |
| inc0 | what's the status of pushable docker registry? | 19:14 |
| mnaser | inc0 my work is all done, it's pending/waiting for the infra team, i did all the things they requested | 19:15 |
| mnaser | https://review.openstack.org/#/c/401003/ | 19:15 |
| mnaser | (i dont have the time to follow up on "hey can you review this" but i dont mind making fixes if they have to be done) | 19:15 |
| inc0 | kk, I'll check up with them | 19:16 |
| mnaser | inc0 also re gate stuff - https://review.openstack.org/#/c/455374/ fixes the gate and gets us on the new proxy mirrors | 19:16 |
| mnaser | and we got this as well. https://review.openstack.org/#/c/455378/ | 19:16 |
| mnaser | we seem to be hitting some really weird transient failure in the python tests... | 19:16 |
| *** sayantan_ has joined #openstack-kolla | 19:17 | |
| kfox1111 | mnaser: so... have you ever heard from them when "zuul 3 will be ready" ? | 19:19 |
| mnaser | we dont need zuulv3 for the things we want to use | 19:19 |
| mnaser | also i was thinking the other day, we cant push gate generated images to dockerhub | 19:19 |
| kfox1111 | well, they want a job rather hten a machine. | 19:19 |
| *** eaguilar has quit IRC | 19:19 | |
| mnaser | they point to the openstack mirrors | 19:19 |
| mnaser | they have apt checks disabled | 19:19 |
| mnaser | etc | 19:19 |
| kfox1111 | but if its going to take months to get jobs, thats not a good plan. :/ | 19:19 |
| kfox1111 | no one's realy saying how far out zuul3 really is. :/ | 19:20 |
| mnaser | well we can just operate in the assumption that it doesnt exist and we can upgrade when things are good | 19:20 |
| *** rwallner has quit IRC | 19:21 | |
| kfox1111 | mnaser: +1. | 19:21 |
| kfox1111 | but there's been so much pushback. | 19:21 |
| *** sayantan_ has quit IRC | 19:21 | |
| *** rwallner has joined #openstack-kolla | 19:21 | |
| kfox1111 | just trying to figure out how justified the pushback is. | 19:22 |
| *** sayantan_ has joined #openstack-kolla | 19:22 | |
| mnaser | kfox1111 well after the last discussion we are going to just work with things and then rewrite some parts when v3 is a thing | 19:22 |
| kfox1111 | ok. cool. so infra's on board wit htat? | 19:22 |
| mnaser | yep, 3 part thing. | 19:24 |
| mnaser | 1) dockerhub proxy -- happening here https://review.openstack.org/#/c/455378/ | 19:24 |
| mnaser | 2) local docker registry -- https://review.openstack.org/#/c/401003/ | 19:24 |
| mnaser | and 3) trusted docker "pusher" -- https://review.openstack.org/#/c/447524/ | 19:24 |
| kfox1111 | yeah. #3's the thing I really want asap. | 19:25 |
| kfox1111 | thats the one I was talking about. | 19:25 |
| kfox1111 | does it have infra's support yet? or are they still pushing for v3 for that use case? | 19:26 |
| *** rwallner has quit IRC | 19:26 | |
| mnaser | kfox1111 i havent had the time to push it more because i think that we need to get the docker registry up first | 19:26 |
| kfox1111 | I'm the opposite. I want to get fresh/updated/tested containers out to the hub. | 19:27 |
| kfox1111 | the hub scales, so its an easier problem to solve I think. | 19:27 |
| kfox1111 | getting a local repo up in a way thats not easily pushed over I think is harder. | 19:28 |
| *** skape has joined #openstack-kolla | 19:28 | |
| mnaser | kfox1111 true, but then we need to change some workflow. our published registry cannot be published to dockerhub because it has certain things which make it not production-ready imho | 19:28 |
| mnaser | such as apt keys disabled, pointing to openstack mirrors | 19:28 |
| kfox1111 | right now the containers on kolla/* are built by a random collection of people, built on different systems, configured in different ways, and untested. | 19:28 |
| *** manheim has joined #openstack-kolla | 19:28 | |
| kfox1111 | mnaser: not quite sure I follow. there a published registry machine now? | 19:29 |
| inc0 | kfox1111: I think #2 will be good short term | 19:29 |
| *** bmace has quit IRC | 19:29 | |
| inc0 | or even long term | 19:29 |
| inc0 | as we'll have accessible registry | 19:29 |
| *** bmace has joined #openstack-kolla | 19:29 | |
| kfox1111 | inc0: think its up to the pounding we currently hit it with the gate jobs? | 19:30 |
| mnaser | i mean it'll be hit as much as tarballs.o.o does get hit | 19:30 |
| kfox1111 | maybe. | 19:31 |
| inc0 | if we do it locally - we'll have read-only mirrors in every region | 19:31 |
| kfox1111 | pulling a tarball takes a lot less processing potentially? | 19:31 |
| inc0 | that should be fine | 19:31 |
| mnaser | kfox1111 also in gate we don't deploy $world | 19:31 |
| kfox1111 | inc0: then we need infra to distribute to all the local mirrors? | 19:31 |
| mnaser | which means less things to pull down | 19:31 |
| inc0 | kfox1111: that's part of our work | 19:32 |
| kfox1111 | and still, does that let folks actually pull from them outside the gate? | 19:32 |
| inc0 | well it won't stop them but we need to be careful about suggesting that | 19:32 |
| kfox1111 | inc0: if we can just push containers to the hub, we don't have that problem. | 19:32 |
| *** eaguilar has joined #openstack-kolla | 19:32 | |
| kfox1111 | and caching the hub is easier? | 19:32 |
| inc0 | we will have different problem - uplink | 19:32 |
| inc0 | I don't think so | 19:32 |
| inc0 | at some point we want to have gating on master kolla | 19:33 |
| mnaser | well its happening here - https://review.openstack.org/#/c/455378/ | 19:33 |
| mnaser | or being tested :-P | 19:33 |
| *** manheim has quit IRC | 19:33 | |
| kfox1111 | caching's just a squid or other http proxy? | 19:33 |
| kfox1111 | distributing to all the local mirrors though is not? | 19:33 |
| inc0 | so pushing/pulling without cache is a no-no | 19:33 |
| inc0 | with cache* | 19:33 |
| mnaser | http proxy that is local to the region kfox1111 | 19:33 |
| mnaser | so cache hits can only happen per region | 19:33 |
| kfox1111 | mnaser: yeah. http proxy is good. | 19:34 |
| kfox1111 | it will automatically kick in if we push to the hub. | 19:34 |
| kfox1111 | keeping a docker repo mirriroed in each region is a much harder issue. | 19:34 |
| inc0 | kfox1111: we don't need to mirror them | 19:34 |
| inc0 | we can use cache same way as with dockerhub | 19:34 |
| kfox1111 | yeah. we just need fresh containers on the hub. | 19:34 |
| kfox1111 | right. | 19:34 |
| kfox1111 | so we need a way to bless containers as tested, and put them somewhere that some process can push with creds to the hub and we're good to go. | 19:35 |
| kfox1111 | we've got pushing to tarballs.o.o working already. so all we need is the last piece. | 19:35 |
| kfox1111 | which is this: https://review.openstack.org/#/c/447524/ | 19:37 |
| *** Serlex has left #openstack-kolla | 19:37 | |
| inc0 | right | 19:38 |
| kfox1111 | but that seems stalled out on zuul3. | 19:38 |
| inc0 | yeah, so it des | 19:39 |
| inc0 | does | 19:39 |
| inc0 | that's partially why I would be ok with semi-secure docker registry locally | 19:39 |
| kfox1111 | locally to what? | 19:39 |
| inc0 | infra | 19:39 |
| kfox1111 | I want docker pull from my boxen to work, not just in gate. | 19:39 |
| inc0 | + proxies local to region | 19:40 |
| kfox1111 | are we talking about standing up a docker hub replacement just for openstack for us to ship to everyone, or just local "mirror" repo's just for gate jobs? | 19:40 |
| inc0 | that's lower priority than gates tho in my book | 19:40 |
| kfox1111 | cause the latter is very uninteresting to me. :/ | 19:40 |
| inc0 | as people can (and I think should) build their own images | 19:40 |
| kfox1111 | -1 | 19:40 |
| kfox1111 | the lack of is why people MUST build their own images. | 19:41 |
| kfox1111 | thats a big ugly thing though. | 19:41 |
| inc0 | ok I'm not saying no - I'm saying docker registry is not dependant on zuulv3 | 19:41 |
| kfox1111 | i want to be able to use the same exact images, tested in the gate, on my systems. | 19:41 |
| kfox1111 | I can build it myself, but its untested. | 19:41 |
| *** lazyPower has left #openstack-kolla | 19:41 | |
| inc0 | where pushed I'd think it is | 19:41 |
| inc0 | pusher | 19:41 |
| inc0 | as it will be ugly beyond measure to do it without it | 19:42 |
| kfox1111 | so there's quite a bit of value in pushing. | 19:42 |
| inc0 | right | 19:42 |
| inc0 | and it should be super easy to do with zuulv3 | 19:42 |
| kfox1111 | yeah. and hence the quiestion I keep asking/not hearing an answer to. | 19:42 |
| kfox1111 | how far out is that? | 19:42 |
| inc0 | as we will just add pushing to gates after merge (zuulv3 is supposed to have better secret mgmt) | 19:43 |
| kfox1111 | cause if its 2 months, I'd rather just build it without zuulv3 and rewrite it for v3 later. | 19:43 |
| kfox1111 | if its a week or two, then I can wait. | 19:43 |
| inc0 | never got concrete answer, but I'd say we still are talking months | 19:43 |
| kfox1111 | yeah. then we soulnd't wait on zuulv3. | 19:43 |
| kfox1111 | we need to figure something else out in the mean time. | 19:43 |
| inc0 | secrets are issue | 19:44 |
| kfox1111 | yeah. so we probably need a vm for that. | 19:44 |
| kfox1111 | I think thats the only way today. which was what the spec was about. | 19:44 |
| inc0 | secure vm which will get images and push them | 19:44 |
| inc0 | on the other hand | 19:45 |
| inc0 | if we have vm at our disposal | 19:45 |
| inc0 | we can do both | 19:45 |
| inc0 | local registry and pushing to dockerhub | 19:45 |
| kfox1111 | still not really seeing the value of a local registry if we have a cache? | 19:45 |
| kfox1111 | its just one more thing to maintain. | 19:45 |
| kfox1111 | it has all the same auth issues that the hub has? | 19:46 |
| inc0 | dockerhub uses https | 19:46 |
| inc0 | tho | 19:46 |
| kfox1111 | thats... good. | 19:46 |
| inc0 | we can use http for pulls in local registry | 19:46 |
| inc0 | makes caching easier | 19:46 |
| *** magicboiz has joined #openstack-kolla | 19:46 | |
| kfox1111 | very very margionally. | 19:46 |
| kfox1111 | if you set the docker proxy to a squid or other caching server, | 19:47 |
| kfox1111 | it caches https good too. | 19:47 |
| inc0 | ok, I surrender;) let's focus on dockerhub | 19:47 |
| kfox1111 | :) | 19:48 |
| kfox1111 | I think on the surface, it sounds cool to docker pull containers.openstack.org/kolla/... but its a high cost when we need up to date container building first. | 19:49 |
| *** mgoddard has joined #openstack-kolla | 19:49 | |
| inc0 | ok, I'll go east sth and talk to infra | 19:51 |
| inc0 | eat | 19:52 |
| kfox1111 | thx. :) | 19:52 |
| inc0 | I don't need to go east, I'm feeling good in the west | 19:52 |
| kfox1111 | :) | 19:52 |
| kfox1111 | failing to install python-novaclient... fun. | 19:52 |
| openstackgerrit | Kevin Fox proposed openstack/kolla-kubernetes master: WIP: v4 gates. https://review.openstack.org/454841 | 19:53 |
| *** blallau has joined #openstack-kolla | 19:54 | |
| *** sayantan_ has quit IRC | 19:54 | |
| *** sayantan_ has joined #openstack-kolla | 19:54 | |
| *** magicboiz has quit IRC | 19:56 | |
| *** satyar has quit IRC | 19:56 | |
| *** magicboiz has joined #openstack-kolla | 19:57 | |
| kfox1111 | oh. had to pip install pip. | 19:57 |
| *** bogdando has quit IRC | 20:00 | |
| *** ssurana has joined #openstack-kolla | 20:01 | |
| *** magicboiz has quit IRC | 20:01 | |
| *** Pavo has quit IRC | 20:03 | |
| *** shardy has quit IRC | 20:03 | |
| *** lamt has joined #openstack-kolla | 20:08 | |
| *** eaguilar has quit IRC | 20:09 | |
| mnaser | [15:47:11] <kfox1111>it caches https good too. < you cant cache https :x | 20:14 |
| sdake | sup peeps | 20:15 |
| *** g3ek has quit IRC | 20:16 | |
| sdake | sbezverk that patch doesn't have --debug on computekit - I rebased off the debug patches | 20:17 |
| sdake | sbezverk you see - rebase on - rebase off | 20:19 |
| *** fooliouno has quit IRC | 20:20 | |
| sdake | kfox1111 i made more 4.0.0 gates | 20:21 |
| sdake | kfox1111 note whateveryou are doing with 4.0.0 will probably have to merge with the patch that is being done on ironic | 20:22 |
| sdake | rather the service chart enablement of nova | 20:22 |
| sdake | that is the most major change inthe 4.0.0 image series | 20:22 |
| sdake | have to create db properly and have to start a placement api | 20:22 |
| *** g3ek has joined #openstack-kolla | 20:26 | |
| sdake | kfox1111 the docs on docs.oo deploy master and work with 4.0.0 images | 20:26 |
| *** sbezverk_ has joined #openstack-kolla | 20:26 | |
| *** mgoddard has quit IRC | 20:26 | |
| kfox1111 | sdake: yeah, thats all pretty much there in trunk. | 20:27 |
| kfox1111 | you restricted v3 from running I think but v4 seems to have always deployed it. | 20:27 |
| sdake | from runnig hwhich? | 20:27 |
| kfox1111 | the main issue I think is running placement before nova-compute. | 20:27 |
| sbezverk_ | kfox1111: only cell depends on nova-compute afail | 20:27 |
| sdake | cells depends on nova-compute, nova-compute dependso n placement | 20:28 |
| kfox1111 | sry. mixing terms. | 20:28 |
| sdake | to make that work, i think we need some entrypoint stuff | 20:28 |
| *** skape has quit IRC | 20:28 | |
| kfox1111 | for service package? yeah. | 20:29 |
| sdake | yup | 20:29 |
| kfox1111 | its a bit worse. we need conditional deps. :/ | 20:29 |
| sdake | for the non entrypoint gates there is already synchronization | 20:29 |
| kfox1111 | and upgrading will be ugly. | 20:29 |
| *** sbezverk has quit IRC | 20:29 | |
| kfox1111 | no, 4 doesn't quite work with the non services yet... | 20:29 |
| sbezverk_ | kfox1111: if we move to 4.0.0 images then we can uncoditionally use placement ;) | 20:29 |
| sdake | i think the proper appraoch is to make a enable_ocata flag | 20:29 |
| kfox1111 | looks like the lack of cells. | 20:29 |
| sdake | rather then an enalbe placmeent and enable cells | 20:30 |
| kfox1111 | sbezverk_: we really need to learn how to do an upgrade. ;) | 20:30 |
| kfox1111 | becuase 5.x isn't as far out as you think. ;) | 20:30 |
| sbezverk_ | kfox1111: man, we do not have prod version :) we can make 4.0.0 as first prod version | 20:30 |
| kfox1111 | yeah, that may be. | 20:30 |
| sdake | sbezverk_ the big problem in my mind is all of the gating is on 2.0 images tam | 20:31 |
| sdake | atm | 20:31 |
| kfox1111 | sbezverk_: yeah, but we really need to figure out how to do these sorts of things before we set in stone some bad design we can't easily change. just saying. | 20:31 |
| sdake | sbezverk_ so wwe can't just simply "shut-off" 2.0.0 | 20:31 |
| kfox1111 | adding 4 doesn't mean we need to cut off 2 or 3. | 20:31 |
| sdake | i didn't say that kfox1111 | 20:32 |
| kfox1111 | sdake: I know. I'm just saying. | 20:32 |
| sdake | kfox1111 what I said was it is impossible to remove 2 as that is where all the maingating is | 20:32 |
| mnaser | https://review.openstack.org/#/c/455374/ <-- id appreciate if some folks can put eyes on this, this unbreaks (and speeds up) the gate for rdo-based deploys | 20:32 |
| sdake | i hoep to have that repaired soon - although travelling this week | 20:32 |
| kfox1111 | sdake: right. we need to add 4 testing that rivals the 2 testing. | 20:32 |
| kfox1111 | which is what the v4 is the start of. | 20:32 |
| sdake | kfox1111 yup - sbezverk_ has already sorted out 4.0.0 deployment | 20:33 |
| sdake | kfox1111 just a matter of spreading it through the various workflows that need it | 20:33 |
| sdake | gotta shutdown - landing | 20:33 |
| kfox1111 | some of it. | 20:33 |
| openstackgerrit | Kevin Fox proposed openstack/kolla-kubernetes master: WIP: v4 gates. https://review.openstack.org/454841 | 20:35 |
| kfox1111 | never has been tested with ceph. | 20:35 |
| kfox1111 | and I think last I tried, that was broken. | 20:35 |
| *** mgoddard has joined #openstack-kolla | 20:36 | |
| sbezverk_ | kfox1111: it should not be, since ceph is not something unusual, I think it has been tested in ocata by cinder folks before releasing it.. | 20:38 |
| *** gfidente|afk has quit IRC | 20:38 | |
| kfox1111 | sbezverk_: no, I mean, when I tried trunk builds of kolla shortly before ocata, ceph backed volumes failed to attach. | 20:39 |
| kfox1111 | not sure if that was an image build rpoblem, a config problem, or something else. just notiing, it didn't "just work" as it should have. | 20:39 |
| sbezverk_ | kfox1111: I see, hopefully it was just a local glitch and nothing major.. | 20:40 |
| kfox1111 | I hope so. I fought with it for a few days before moving on to other things. | 20:41 |
| kfox1111 | sbezverk_: I just talked with the nova folks, and cells v2 actualy only supports 2 cells. | 20:41 |
| *** eaguilar has joined #openstack-kolla | 20:41 | |
| kfox1111 | requires/supports. | 20:41 |
| kfox1111 | 0 and 1. | 20:41 |
| *** lamt has quit IRC | 20:43 | |
| *** srwilkers has quit IRC | 20:45 | |
| *** mgoddard has quit IRC | 20:46 | |
| *** lrensing has quit IRC | 20:47 | |
| sbezverk_ | kfox1111: I heard it will change in near future | 20:47 |
| kfox1111 | for pike. | 20:47 |
| kfox1111 | its just a lot of operator pain, for something that seems to have no real benifit. | 20:47 |
| sbezverk_ | kfox1111: agree, I do not see any advantage atm other than cool name ;) | 20:48 |
| kfox1111 | yeah. :/ | 20:48 |
| *** ssurana has quit IRC | 20:51 | |
| *** pomac_ is now known as pomac | 20:57 | |
| *** sayantan_ has quit IRC | 21:01 | |
| *** jtriley has quit IRC | 21:01 | |
| *** manheim has joined #openstack-kolla | 21:02 | |
| *** unicell has quit IRC | 21:03 | |
| *** unicell has joined #openstack-kolla | 21:05 | |
| *** ssurana has joined #openstack-kolla | 21:06 | |
| openstackgerrit | Kevin Fox proposed openstack/kolla-kubernetes master: WIP: v4 gates. https://review.openstack.org/454841 | 21:08 |
| mnaser | i think we'll have to backport placement api to newton | 21:09 |
| mnaser | it would be nice to have that deployed so that the upgrade is a lot less scary with a missing component | 21:09 |
| *** pcaruana has quit IRC | 21:15 | |
| *** shashank_t_ has joined #openstack-kolla | 21:20 | |
| *** blallau has quit IRC | 21:22 | |
| *** shashank_t_ has quit IRC | 21:24 | |
| *** pramodrj07 has joined #openstack-kolla | 21:32 | |
| *** MasterOfBugs has joined #openstack-kolla | 21:32 | |
| *** spiette has joined #openstack-kolla | 21:34 | |
| *** jrobinson has joined #openstack-kolla | 21:40 | |
| *** lucasxu has quit IRC | 21:40 | |
| *** eaguilar has quit IRC | 21:43 | |
| *** shashank_t_ has joined #openstack-kolla | 21:43 | |
| *** shashank_t_ has quit IRC | 21:47 | |
| *** sayantan_ has joined #openstack-kolla | 21:51 | |
| *** TxGirlGeek has joined #openstack-kolla | 21:54 | |
| *** TxGirlGeek has quit IRC | 21:57 | |
| *** lamt has joined #openstack-kolla | 22:02 | |
| *** sayantan_ has quit IRC | 22:08 | |
| *** manheim has quit IRC | 22:08 | |
| kfox1111 | mnaser: its optional in newton. Haven't had a problem in gate without it. | 22:10 |
| kfox1111 | yeah. its a lot to swallow in one upgrade. | 22:10 |
| kfox1111 | plus it has no benifit. :/ | 22:10 |
| kfox1111 | /bin/cp: cannot remove '/etc/nova/nova.conf': Permission denied | 22:15 |
| kfox1111 | http://logs.openstack.org/41/454841/5/experimental/gate-kolla-kubernetes-deploy-centos-binary-4-ceph-multi-nv/deb8462/logs/pods/kolla-nova-api-create-cell-72r6z-main.txt | 22:15 |
| *** m1dev has quit IRC | 22:15 | |
| kfox1111 | sbezverk_: any ideas? | 22:15 |
| mnaser | i haev an idea | 22:15 |
| mnaser | overlayfs on centos is broken. | 22:15 |
| mnaser | and stuff like this happen all the time | 22:16 |
| kfox1111 | sbezverk_: in nova-api-create-cell. | 22:16 |
| kfox1111 | mnaser: ;) | 22:16 |
| mnaser | kolla gate uses btrfs | 22:16 |
| mnaser | so it might be beneficial to do the same | 22:16 |
| kfox1111 | mnaser: its loopback lvm thin provisioned. | 22:16 |
| kfox1111 | yeah, I use btrfs on the production systems. | 22:16 |
| kfox1111 | haven't had a thin provision issue though in gate. | 22:16 |
| kfox1111 | the real issue is, whats trying to delete /etc/nova/nova.conf... shouldn't happen. | 22:17 |
| openstackgerrit | Merged openstack/kolla master: ironic-pxe: handle non-x86 architectures https://review.openstack.org/434817 | 22:18 |
| kfox1111 | oh.... its trying to overwrite the containers etc/nova/nova.conf... | 22:18 |
| kfox1111 | but it may not have permissions in the 4.0.x container to do so. :/ | 22:18 |
| kfox1111 | how fun... | 22:19 |
| kfox1111 | confirmed... | 22:19 |
| kfox1111 | :/ | 22:19 |
| *** manheim has joined #openstack-kolla | 22:19 | |
| kfox1111 | so, the ubuntu and centos images are much different here. :/ | 22:20 |
| kfox1111 | well... could just run this as root I guesss. :/ | 22:20 |
| *** MarginHu has joined #openstack-kolla | 22:22 | |
| *** MarginHu has quit IRC | 22:22 | |
| *** MarginHu has joined #openstack-kolla | 22:22 | |
| openstackgerrit | Kevin Fox proposed openstack/kolla-kubernetes master: WIP: v4 gates. https://review.openstack.org/454841 | 22:23 |
| *** srwilkers has joined #openstack-kolla | 22:27 | |
| *** japestinho has quit IRC | 22:30 | |
| *** lamt has quit IRC | 22:31 | |
| *** MarginHu has quit IRC | 22:40 | |
| *** MarginHu has joined #openstack-kolla | 22:41 | |
| *** mbruzek has quit IRC | 22:41 | |
| *** MarginHu has quit IRC | 22:42 | |
| *** esharao has quit IRC | 22:50 | |
| *** jascott1_ has joined #openstack-kolla | 22:51 | |
| sdake | mnaser re backporting placement, that is a nonstarter - as the placement API is a feature | 22:52 |
| mnaser | i agree, it just feels like there's less likelyness of crap hitting the fan if its there beforeo | 22:53 |
| sdake | we are on the cusp of actually having a PMT (stable:follows-policy) | 22:53 |
| sdake | that would delay that activity for 12 months | 22:53 |
| sdake | not going to approve of it ;) | 22:54 |
| kfox1111 | its one less thing to have to deal with for upgrades. | 22:54 |
| kfox1111 | if you assume all deployments didn't have placement, like they dont today. | 22:54 |
| sdake | still not going to approve of such a backport | 22:54 |
| kfox1111 | sdake: no, I'm arguing your case. :) | 22:54 |
| sdake | ok :) | 22:54 |
| *** kbaegis has joined #openstack-kolla | 22:54 | |
| kfox1111 | I'm saying, if you backported, now you have to test 2 things during upgrades. | 22:55 |
| kfox1111 | with and without. | 22:55 |
| sdake | gah tha would be a nightmare | 22:55 |
| jascott1_ | hey guys, do we still have to do backflips for 1.6.1? im still getting the networking CNI not ready errorz... | 22:55 |
| kfox1111 | jascott1_: yeah, a few. :/ | 22:55 |
| sdake | jascott1_ backflips yes | 22:55 |
| sdake | jascott1 its documented here: | 22:55 |
| kfox1111 | they are being really slow to get fixes into 1.6. :/ | 22:55 |
| sdake | https://docs.openstack.org/developer/kolla-kubernetes/deployment-guide.html | 22:56 |
| kfox1111 | sdake: sbezverk_: still seeing the same error I was before when I tried ocata. :/ | 22:56 |
| kfox1111 | http://logs.openstack.org/41/454841/6/experimental/gate-kolla-kubernetes-deploy-centos-binary-4-ceph-multi-nv/79db8e7/console.html | 22:56 |
| kfox1111 | centos + cinder backed ceph is broken for volume attachment. :/ | 22:56 |
| kfox1111 | so, v4's still broken. :/ | 22:57 |
| sdake | this one? http://logs.openstack.org/41/454841/6/experimental/gate-kolla-kubernetes-deploy-centos-binary-4-ceph-multi-nv/79db8e7/console.html#_2017-04-10_22_53_16_261970 | 22:57 |
| jascott1_ | sdake thanks! ive been going thru the 'broken 1.6' issue but this looks legit | 22:57 |
| kfox1111 | sdake: yeah. | 22:57 |
| sdake | jascott1 ya - authored by the community - its been tested and confirmed by 5 different peole | 22:57 |
| sdake | kfox1111 wierd, ironic doesn't show such issue | 22:57 |
| kfox1111 | sdake: everything but the volume attachment works. but that fails. :/ | 22:57 |
| kfox1111 | sdake: yeahk, but ironic's not testing volumes. | 22:58 |
| sdake | oh - so yo umean cinder is broken | 22:58 |
| kfox1111 | yeah, something in the cinder layer's broken. | 22:58 |
| sdake | iscsi or ceph? | 22:58 |
| kfox1111 | ceph | 22:58 |
| sdake | iscsi works? | 22:58 |
| kfox1111 | works with 2 and 3 though. | 22:58 |
| kfox1111 | not sure. I think so though. | 22:59 |
| sbezverk_ | sdake: yes iscsi works | 22:59 |
| *** jrobinson has quit IRC | 22:59 | |
| sdake | so really a problem with cinder + ceph integration | 23:00 |
| kfox1111 | or nova ceph | 23:00 |
| sdake | sbezverk_ I know iscsi works with 4.0.0 images -Ive personally tested it outside the gate, but not with a volume attach in cinder | 23:00 |
| kfox1111 | or libvirt ceph. | 23:00 |
| *** manheim has quit IRC | 23:00 | |
| sdake | sbezverk_ are you sure the volume attach works with cinder? | 23:00 |
| kfox1111 | sdake: if you din't test it with attach, its not really tested. | 23:01 |
| sdake | kfox1111 you assume i care about cinder ;) | 23:01 |
| kfox1111 | thats where it usually breaks. :/ | 23:01 |
| kfox1111 | sdake: as part of compute kit, we should. ;) | 23:01 |
| sdake | kfox1111 oh - well if you mean th eproject shoudl care, i a gree | 23:01 |
| sdake | we should fix it | 23:01 |
| *** manheim has joined #openstack-kolla | 23:01 | |
| kfox1111 | 2017-04-10 22:52:50.255 7 ERROR oslo_messaging.rpc.server libvirtError: Secret not found: no secret with matching uuid '79efb894-61c9-478b-9d16-6a3598a53871' | 23:02 |
| sdake | i have a much better handle on how the gate works now | 23:02 |
| kfox1111 | hmm.. | 23:02 |
| sdake | as well as much of the code base | 23:02 |
| kfox1111 | 47e08504-5bfc-4c45-bbf0-af02049b7623 ceph client.nova secret | 23:02 |
| kfox1111 | secret uuid's different... | 23:02 |
| kfox1111 | thats really strange. | 23:02 |
| sdake | we are releasing a tag on the 15th | 23:03 |
| sdake | any chance folks can burn through the backlog of patches that are easy to merge | 23:03 |
| sdake | it takees 5 minutes | 23:03 |
| kfox1111 | would be nice if it worked with 4.x... | 23:03 |
| sdake | maybe 15 ;) | 23:03 |
| jascott1_ | why do we change this? s/10.96.0.10/10.3.3.10/g | 23:03 |
| sdake | kfox1111 agree - I am going to help there as much I can - although I am on the road in training during the day | 23:03 |
| kfox1111 | sdake: k. | 23:04 |
| sdake | jascott1_ oh you want ot know why? | 23:04 |
| jascott1_ | haha | 23:04 |
| sdake | jascott1_ shall I drop knowledge ;) | 23:04 |
| jascott1_ | do I? | 23:04 |
| jascott1_ | :) | 23:04 |
| kfox1111 | inc0: any chance we can get the ansible stuff finished by the 15th? | 23:04 |
| sdake | jascott1_ basically 10.96.0.10 is the default kubeadm ships iwth | 23:04 |
| jascott1_ | right. | 23:04 |
| sdake | we change that 16 to a 24 at 10.3.3.0/24 | 23:05 |
| sdake | 10.96 works on many networks but not all networks | 23:05 |
| sdake | the reason it is documented that way is to explain that you may want to change the pod and serveice cidrs | 23:05 |
| jascott1_ | ah ok thanks | 23:05 |
| kfox1111 | 10.3.x doesn't work on all networks either. its a rather arbitrary change, | 23:05 |
| sdake | kfox1111 agreed | 23:06 |
| kfox1111 | but does show how to change it for your site if need be. | 23:06 |
| sdake | kfox1111 it shows how to change it | 23:06 |
| sdake | not what to change it to :) | 23:06 |
| kfox1111 | right. :) | 23:06 |
| sdake | jascott1_ for example if you use 10.96/16 and someone else inside intel is using kubadm in the lab | 23:07 |
| sdake | I suspect bad things may happen | 23:07 |
| sdake | but I can't confirm it | 23:07 |
| sdake | as i haven't tried it | 23:07 |
| sdake | jascott1_ if the documentation is unclear, would appreciate a patch to clarify it | 23:08 |
| sdake | jascott1_ explaining that bit (the pod and service cidrs) was extermely challenging | 23:08 |
| kfox1111 | depends on if your org is routing rfc 1918 space. | 23:08 |
| sdake | as I first had to understand it at a basic level | 23:08 |
| kfox1111 | rfc 1918 is only unroutable on the internet. but says nothing about at the org level. | 23:09 |
| sdake | we route 10 addresses internally | 23:09 |
| sdake | atleast I think so :) | 23:09 |
| jascott1_ | i have no idea but we def have 10.x network | 23:09 |
| kfox1111 | I made my org cough up a /16 as psudo rfc 1918 address space. | 23:09 |
| *** lamt has joined #openstack-kolla | 23:09 | |
| kfox1111 | then people at the lab can use that /16 just like it was rfc 1918 and know its not routable in the org. | 23:10 |
| sdake | jascott1_ ask your network admin for two /24 networks | 23:10 |
| jascott1_ | but im deploying on cloud1 anyway | 23:10 |
| sdake | jascott1_ if you have access to your own switches, you can setup vlans on your own | 23:10 |
| sdake | this is what I do - sort of :) | 23:10 |
| sdake | networking = hard | 23:10 |
| jascott1_ | agreed! | 23:10 |
| sdake | best left to the experts I think :) | 23:11 |
| kfox1111 | sdake: there's few that acually understand sdn yet. ;) | 23:11 |
| kfox1111 | so usually I have to explain to them how it all works. | 23:11 |
| sdake | I dont understand it | 23:11 |
| kfox1111 | and they turn green and run away half way through. :) | 23:11 |
| jascott1_ | haha | 23:12 |
| sdake | i mean at a basic level i get it | 23:12 |
| sdake | but far from an expert in the area | 23:12 |
| sdake | nor do I care to be :) | 23:12 |
| jascott1_ | ya thats really the thing, caring enough to invest the time to 'get it' | 23:12 |
| sdake | i have many expert level specialties | 23:13 |
| sdake | networking isn't one of them | 23:13 |
| jascott1_ | Im a certified Netware 4.0 Expert | 23:14 |
| jascott1_ | get calls all the time | 23:14 |
| jascott1_ | (i kid) | 23:14 |
| kfox1111 | really weirded out that the uuid changed.... | 23:14 |
| kfox1111 | jascott1_: hehe. I actually kind of liked netware 4. | 23:14 |
| kfox1111 | so much better then nt4's networking at the time. | 23:14 |
| jascott1_ | loading tha dos network stack was a treat | 23:15 |
| *** srwilkers has quit IRC | 23:15 | |
| kfox1111 | ipx was pretty stable and plug and play. | 23:15 |
| kfox1111 | ip, not so much. | 23:15 |
| jascott1_ | yeah they had the directory server going before ms | 23:15 |
| kfox1111 | ad was a shallow copy of nds for a long time. | 23:15 |
| jascott1_ | no doubt | 23:16 |
| jascott1_ | ah to bisect thin net while the whole factory is down. #good-ol-days | 23:16 |
| kfox1111 | I was really disapointed when folks pushed to windwos file shares from it. the management tools in windows really hurt. :/ | 23:16 |
| kfox1111 | hehe. | 23:16 |
| *** kbaegis has left #openstack-kolla | 23:17 | |
| jascott1_ | funny the instructor for our golang class a week back was also a CNE | 23:17 |
| kfox1111 | I barely did not get a CNE. | 23:17 |
| kfox1111 | I saw the writing on the wall, and gave up on it before wasting more time. | 23:18 |
| kfox1111 | microsoft was obviously going to get the monopoly, so I moved on. | 23:18 |
| jascott1_ | i was about 18 so it helped me get my foot in the door at some jobs | 23:18 |
| kfox1111 | yeah. well worth it in that case. | 23:18 |
| kfox1111 | ever play with... | 23:18 |
| kfox1111 | hate to say the word in plublic, | 23:19 |
| jascott1_ | Lantastic? | 23:19 |
| kfox1111 | sco's (before they were bought and went evil) | 23:19 |
| kfox1111 | linux novel server? | 23:19 |
| jascott1_ | i thought it was SUSE | 23:19 |
| jascott1_ | SUSE bought em I thought no? | 23:19 |
| kfox1111 | I think it was like caldera's, then sco bought them. | 23:19 |
| jascott1_ | ah | 23:19 |
| kfox1111 | then they went bankrupt and someone else took the name sco? | 23:20 |
| kfox1111 | can't quite remember. | 23:20 |
| kfox1111 | anyway. it worked really well too. | 23:20 |
| kfox1111 | there's still thinkgs I really miss about netware. | 23:20 |
| kfox1111 | their acl's were awesome. | 23:20 |
| kfox1111 | linuxs acl's really are painful. :/ | 23:20 |
| jascott1_ | ya I almost brought up ACLs | 23:20 |
| sdake | the dude that ran netware into the ground joined google as their ceo :) | 23:21 |
| jascott1_ | Schmidt? | 23:21 |
| sdake | thats the one | 23:21 |
| jascott1_ | yea hes from SUN | 23:21 |
| jascott1_ | iirc | 23:21 |
| sdake | another co run into the ground :) | 23:21 |
| sdake | such a shame - they used to make such nice workstations | 23:22 |
| sdake | I have an hp z820 workstation - very nice | 23:22 |
| sdake | not sure HPE going to be making them anymore | 23:22 |
| sdake | its the last good business class workstation you can buy | 23:22 |
| jascott1_ | yeah caught a random video of someone from SUN talking about the whole Oracle transition and how absolutely horrible it was | 23:23 |
| jascott1_ | it sounded worse than I had imagined | 23:23 |
| kfox1111 | hmm... ceph is using the uuid in the config file... | 23:23 |
| kfox1111 | where did that other uuid come from.... | 23:23 |
| jascott1_ | alright im going to convert these instructiones to 'buntu | 23:24 |
| kfox1111 | weird... its whats in libvirt here: http://logs.openstack.org/41/454841/6/experimental/gate-kolla-kubernetes-deploy-centos-binary-4-ceph-multi-nv/79db8e7/console.html#_2017-04-10_22_31_33_567311 | 23:24 |
| kfox1111 | so its wrong somehow in cinder... | 23:25 |
| kfox1111 | the rbd_secret_uid is not in nova.conf though... | 23:28 |
| kfox1111 | what's going on? | 23:28 |
| *** lamt has quit IRC | 23:29 | |
| kfox1111 | here we go... | 23:30 |
| kfox1111 | its only in nova-compute's configmap. | 23:30 |
| kfox1111 | but they are out of sync... | 23:30 |
| *** iniazi has quit IRC | 23:31 | |
| kfox1111 | inc0: ping | 23:32 |
| kolla-slack | <inc0> What's up? | 23:32 |
| kfox1111 | inc0: not sure how, but in this run: http://logs.openstack.org/41/454841/6/experimental/gate-kolla-kubernetes-deploy-centos-binary-4-ceph-multi-nv/79db8e7/ | 23:32 |
| *** iniazi has joined #openstack-kolla | 23:32 | |
| kfox1111 | rbd_secret_uuid is different between nova and cinder. | 23:33 |
| kfox1111 | see: http://logs.openstack.org/41/454841/6/experimental/gate-kolla-kubernetes-deploy-centos-binary-4-ceph-multi-nv/79db8e7/logs/configmaps.yaml | 23:33 |
| kfox1111 | could this be something that broke during cinder config merging into kolla-kubernetes? | 23:33 |
| *** lamt has joined #openstack-kolla | 23:33 | |
| kolla-slack | <inc0> Hmm hard to tell, would be strange because it did work before | 23:34 |
| kfox1111 | yeah. and I think it workes for 3.x and 2.x | 23:34 |
| kfox1111 | this is with 4.x. | 23:34 |
| kfox1111 | but genconfig shuldn't have changed at all there. | 23:34 |
| kfox1111 | could the cinder regen code be using a different value for passwords.yml? | 23:36 |
| kfox1111 | inc0: you can see here: http://logs.openstack.org/41/454841/6/experimental/gate-kolla-kubernetes-deploy-centos-binary-4-ceph-multi-nv/79db8e7/console.html#_2017-04-10_22_31_33_567311 | 23:36 |
| kfox1111 | its reading out the uuid that starts with a 4. | 23:37 |
| kfox1111 | but if you grep for rbd_secret_uuid here: | 23:37 |
| kfox1111 | http://logs.openstack.org/41/454841/6/experimental/gate-kolla-kubernetes-deploy-centos-binary-4-ceph-multi-nv/79db8e7/logs/configmaps.yaml | 23:37 |
| kfox1111 | you can see it starts with a 7. | 23:37 |
| kfox1111 | and came from the kolla-kubernetes ansible genconfig. | 23:38 |
| kolla-slack | <inc0> We copied ansible from master, so it's 3.0 that's further away | 23:38 |
| *** swinn has joined #openstack-kolla | 23:39 | |
| kolla-slack | <inc0> Thing is, do we use different ceph users for services? | 23:39 |
| kfox1111 | its a pure config thing right now... | 23:39 |
| kfox1111 | here's what came out of genconfig for cinder: | 23:40 |
| kfox1111 | http://logs.openstack.org/41/454841/6/experimental/gate-kolla-kubernetes-deploy-centos-binary-4-ceph-multi-nv/79db8e7/logs/kolla/cinder-volume/cinder.conf | 23:40 |
| kfox1111 | has the 7 uuid in it there too. | 23:40 |
| kolla-slack | <inc0> I need to check code for cinder (I didn't write this one) | 23:40 |
| kfox1111 | so for some reason whats in passwords.yml is not what got put into cinder.conf. | 23:40 |
| Reepicheep | I see kolla-ansible has taken my additional drives on at least the compute nodes and created a lvm volume group called "vm" | 23:41 |
| swinn | Hey all, first time deploying ocata with kolla-ansible. All services appear up except cinder-api which reports in the docker logs that the cinder-api executable is not found. I’m using ceph as a backend, all other backends are set to “no” in globals.yaml. Anyone have any experience with this error? | 23:41 |
| Reepicheep | I would like to use that for ephimeral but I'm having trouble figuring out how | 23:41 |
| inc0 | swinn: can you show me your globals.yml? | 23:43 |
| kfox1111 | inc0: this is weird... | 23:43 |
| kfox1111 | inc0: in the cinder template, its puling from var cinder_rbd_secret_uuid | 23:43 |
| kfox1111 | but in passwords.yml, its just rbd_secret_uuid. | 23:43 |
| kfox1111 | well, in my, decidedly older passwords.yml... | 23:43 |
| inc0 | hold on | 23:43 |
| kfox1111 | ah.... got a gate snapshot here: http://logs.openstack.org/41/454841/6/experimental/gate-kolla-kubernetes-deploy-centos-binary-4-ceph-multi-nv/79db8e7/logs/kolla/passwords.yml | 23:44 |
| kfox1111 | it is 79 there.. | 23:44 |
| kfox1111 | yup.. and there's the 47 one too. | 23:44 |
| inc0 | hmm interesting | 23:44 |
| kfox1111 | there is an rbd_secret_uuid and a cinder_rbd_secret_uuid now... | 23:45 |
| swinn | inc0: https://pastebin.com/qa7RY9kU | 23:45 |
| kfox1111 | why is that? | 23:45 |
| inc0 | well, I thought it was done differently | 23:45 |
| inc0 | will need to check if it's a bug in ansible | 23:45 |
| inc0 | swinn: and docker logs cinder-api shows...? | 23:46 |
| sdake | kfox1111 it matters not if we can just merge inc0's cinder change | 23:47 |
| inc0 | Reepicheep: ephemerals lands in /var/lib/docker/volumes/nova-libvirt | 23:47 |
| swinn | inc0: Running command: 'cinder-api --config-file /etc/cinder/cinder.conf' | 23:47 |
| kfox1111 | sdake: his change did merge. | 23:47 |
| sdake | kfox1111 which i guess means fixing it so it passes the gate | 23:47 |
| kfox1111 | sdake: I'm trying to understand why its broken. | 23:47 |
| sdake | kfox1111 i see - well should be easy to fix then | 23:47 |
| swinn | .. /usr/local/bin/kolla_start: line 24: exec: cinder-api: not found | 23:47 |
| inc0 | swinn: so thats looks good, what was the error again? | 23:47 |
| kfox1111 | what is the difference here: | 23:47 |
| inc0 | hmm | 23:47 |
| kfox1111 | http://logs.openstack.org/41/454841/6/experimental/gate-kolla-kubernetes-deploy-centos-binary-4-ceph-multi-nv/79db8e7/logs/kolla/passwords.yml | 23:47 |
| inc0 | did you build your own images or download from dockerhub? | 23:47 |
| kfox1111 | between rbd_secret_uuid and cinder_rbd_secret_uuid? | 23:48 |
| Reepicheep | inc0: thanks, would it be safe mounting the "vm" volume group that was created there? or is that designated for smothing else? | 23:48 |
| swinn | built, run from a local registry | 23:48 |
| swinn | insecure... | 23:48 |
| inc0 | swinn: distro and type plz:) | 23:48 |
| swinn | ubuntu binary | 23:48 |
| *** eaguilar has joined #openstack-kolla | 23:48 | |
| inc0 | looks like images failed to build correctly | 23:48 |
| kfox1111 | nova sems to use the one, and cinder uses another... but they have to be the same... | 23:48 |
| inc0 | try switching to source and redeploy plz | 23:48 |
| kfox1111 | I don't get it. | 23:48 |
| swinn | k will give it a shot | 23:49 |
| inc0 | Reepicheep: I'm trying to figure out what creates any lvm shares, cinder only comes to mind, but that's for volumes | 23:49 |
| swinn | eventually I’ll have dozens of nodes though, so need to solve for why it won’t work locally | 23:49 |
| inc0 | and you shouldn't use this share | 23:49 |
| inc0 | swinn: good thing about images is that you build them once and distribute | 23:49 |
| inc0 | I recommend source builds generally because from my experience they work better (less dependencies on stuff like distro maintainers) | 23:50 |
| inc0 | fyi source build only uses sources for openstack | 23:50 |
| inc0 | things like mariadb or libvirt are still apt-got | 23:51 |
| swinn | ok well that’s easy to test out, will have to do tomorrow since it’s almost quitting time | 23:51 |
| inc0 | swinn: sure, I'll be here if you need help:) | 23:51 |
| *** g3ek has quit IRC | 23:51 | |
| Reepicheep | inc0: I'm not sure what did it .. I have been testing kolla and by the time I got down my list to local ephimeral storage I noticed that my 3 additional unused HD's where already in that volume group | 23:51 |
| sdake | swinn ubuntu binary has bee nbroken for ages | 23:51 |
| sdake | swinn recommend ubuntu source | 23:51 |
| swinn | thanks sdake | 23:51 |
| inc0 | Reepicheep: are you using iscsi and cinder? | 23:52 |
| inc0 | sdake: not "for ages", it breaks from time to time | 23:52 |
| inc0 | but true that - ubutnu source is much more stable | 23:52 |
| Reepicheep | inc0: nope and nope, I have not enabled any storage services yet | 23:53 |
| sdake | inc0 ok | 23:53 |
| inc0 | strange, then lvm shouldn't be touched at all | 23:53 |
| Reepicheep | that is kinda what I would expect | 23:53 |
| inc0 | yeah that's what should happen, can you paste your globals.yml? | 23:53 |
| Reepicheep | I'm not sure when they got created .. I have done quite a bit of deploys on it | 23:54 |
| inc0 | I mean shouldn't happen;) expecially that we don't really "name" stuff like vm | 23:54 |
| inc0 | in fact I don't even touch lvm at all since I run all on ceph | 23:54 |
| Reepicheep | hmm.. I don't remember doing it myself | 23:54 |
| Reepicheep | oh.. I bet I know what it is .. | 23:55 |
| *** lamt has quit IRC | 23:55 | |
| swinn | would a source build also help clear up some of the issues I saw with horizon panels failing to load for additonally enabled services? | 23:55 |
| Reepicheep | prior to deploying openstack with Kolla-ansible I had used the same hardware and deployed using Fuel | 23:55 |
| inc0 | swinn: quite possibly | 23:55 |
| *** jrobinson has joined #openstack-kolla | 23:56 | |
| swinn | nice | 23:56 |
| inc0 | Reepicheep: yeah... | 23:56 |
| Reepicheep | I bet the VG's are left over from the old deployments | 23:56 |
| Reepicheep | ok.. I feel better about it now.. | 23:56 |
| inc0 | well, in Kolla-ansible we don't touch host | 23:56 |
| inc0 | as a rule | 23:56 |
| inc0 | so when you run cleanup, it should be pristine as before deploy | 23:56 |
| openstackgerrit | Kevin Fox proposed openstack/kolla-kubernetes master: WIP: v4 gates. https://review.openstack.org/454841 | 23:57 |
| *** swinn has quit IRC | 23:57 | |
| inc0 | I don't think fuel follows same policy | 23:57 |
| Reepicheep | lol, yeah that is on me.. I installed centos on the nodes but did not touch the data drives which already had the VGs on them | 23:57 |
| Reepicheep | yeah.. fuel installs and deploys the entire system | 23:57 |
| inc0 | fyi if you want to put stuff like that on volumes | 23:57 |
| inc0 | after you deploy do docker volume ls | 23:58 |
| *** manheim has quit IRC | 23:58 | |
| inc0 | you'll see quite a few | 23:58 |
| inc0 | that will correspond with /var/lib/docker/volumes | 23:58 |
| Reepicheep | yeah.. a docker volume seems like a good way to do it | 23:58 |
| inc0 | so you can mount your lvm volume to /var/lib/docker/volumes/nova-libvirt | 23:59 |
| inc0 | and that should be it | 23:59 |
| Reepicheep | inc0: I can handle that thanks | 23:59 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!