| *** neilus has quit IRC | 00:00 | |
| *** zhurong has quit IRC | 00:04 | |
| openstackgerrit | Kevin Fox proposed openstack/kolla: Allow genconfig to work with kolla-kubernetes and ceph https://review.openstack.org/372811 | 00:04 |
|---|---|---|
| *** duonghq has joined #openstack-kolla | 00:21 | |
| *** jtriley has joined #openstack-kolla | 00:30 | |
| *** huikang has joined #openstack-kolla | 00:34 | |
| *** bjolof__ has joined #openstack-kolla | 00:41 | |
| *** bjolo__ has joined #openstack-kolla | 00:41 | |
| *** bjolo has quit IRC | 00:41 | |
| *** bjolo_ has quit IRC | 00:44 | |
| openstackgerrit | zhongshengping proposed openstack/kolla: Deprecate old auth parameters of ironic in nova.conf https://review.openstack.org/372306 | 00:44 |
| *** caowei has joined #openstack-kolla | 01:02 | |
| *** phuongnh has joined #openstack-kolla | 01:05 | |
| *** zhurong has joined #openstack-kolla | 01:12 | |
| *** zhubingbing has joined #openstack-kolla | 01:17 | |
| *** Marcellin__ has quit IRC | 01:17 | |
| *** gfhellma has quit IRC | 01:20 | |
| *** rstarmer has joined #openstack-kolla | 01:25 | |
| duonghq | sdake: so we deprecate both fedora and debian support? | 01:25 |
| duonghq | (just to confirm) | 01:25 |
| openstackgerrit | dupengfei proposed openstack/kolla: Support shared path for glance images and nova instances disk https://review.openstack.org/372420 | 01:32 |
| *** huikang has quit IRC | 01:35 | |
| *** huikang has joined #openstack-kolla | 01:35 | |
| openstackgerrit | dupengfei proposed openstack/kolla: Fix keystone,database,memcache for k8s. https://review.openstack.org/372089 | 01:38 |
| openstackgerrit | caoyuan proposed openstack/kolla: Remove the redundant command when deployment https://review.openstack.org/371608 | 01:39 |
| *** huikang has quit IRC | 01:39 | |
| *** salv-orl_ has joined #openstack-kolla | 01:42 | |
| *** neilus has joined #openstack-kolla | 01:43 | |
| *** salv-orlando has quit IRC | 01:45 | |
| *** fguillot has quit IRC | 01:46 | |
| openstackgerrit | caoyuan proposed openstack/kolla: Remove the redundant command when deployment https://review.openstack.org/371608 | 01:47 |
| *** neilus has quit IRC | 01:48 | |
| sdake | duonghq there are seprate votes for each since they are seprate distributions | 01:52 |
| sdake | duonghq depends on the voting of the core reviewer team | 01:52 |
| duonghq | sdake: but current trending is going to remove those? | 01:53 |
| *** signed8bit has joined #openstack-kolla | 01:54 | |
| *** signed8bit has quit IRC | 01:54 | |
| duonghq | wonder if I misunderstand | 01:54 |
| *** rstarmer has quit IRC | 02:00 | |
| *** rstarmer has joined #openstack-kolla | 02:05 | |
| *** unicell has quit IRC | 02:05 | |
| *** sdake has quit IRC | 02:06 | |
| *** sdake has joined #openstack-kolla | 02:06 | |
| *** schwicht has quit IRC | 02:08 | |
| sdake | duonghq i can't predict votes - votes require a mjaority pproval | 02:08 |
| duonghq | roger sdake | 02:11 |
| *** schwicht has joined #openstack-kolla | 02:12 | |
| *** rstarmer has quit IRC | 02:12 | |
| *** sdake has quit IRC | 02:18 | |
| *** lulei has quit IRC | 02:21 | |
| *** rstarmer has joined #openstack-kolla | 02:24 | |
| *** daneyon has joined #openstack-kolla | 02:25 | |
| *** rstarmer has quit IRC | 02:29 | |
| *** rstarmer has joined #openstack-kolla | 02:29 | |
| *** daneyon has quit IRC | 02:29 | |
| openstackgerrit | Wei Cao proposed openstack/kolla: Delete extra spaces in baremetal install.yml https://review.openstack.org/372842 | 02:32 |
| *** neilus has joined #openstack-kolla | 02:38 | |
| *** dave-mccowan has quit IRC | 02:41 | |
| *** huikang has joined #openstack-kolla | 02:41 | |
| *** neilus has quit IRC | 02:44 | |
| *** schwicht has quit IRC | 02:54 | |
| *** huikang has quit IRC | 02:57 | |
| *** williamcaban has joined #openstack-kolla | 03:03 | |
| *** david-lyle has quit IRC | 03:03 | |
| *** williamcaban has quit IRC | 03:07 | |
| *** huikang has joined #openstack-kolla | 03:08 | |
| *** coolsvap has joined #openstack-kolla | 03:22 | |
| *** tonanhngo has joined #openstack-kolla | 03:35 | |
| *** unicell has joined #openstack-kolla | 03:39 | |
| *** tonanhngo has quit IRC | 03:39 | |
| *** tonanhngo has joined #openstack-kolla | 03:45 | |
| *** daneyon has joined #openstack-kolla | 04:13 | |
| zhubingbing | every boys | 04:15 |
| *** daneyon has quit IRC | 04:18 | |
| *** bdaca has joined #openstack-kolla | 04:18 | |
| bdaca | morning | 04:18 |
| openstackgerrit | Merged openstack/kolla: Update EPEL version in base https://review.openstack.org/372797 | 04:24 |
| bjolof__ | morning | 04:25 |
| *** neilus has joined #openstack-kolla | 04:26 | |
| bjolof__ | http://paste.openstack.org/show/581647/ | 04:27 |
| bjolof__ | about the same time for reconfigure as before. Seems like TLS did not have anything to do with it | 04:27 |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: Mark include-footer and include-header deprecated for removal https://review.openstack.org/372880 | 04:28 |
| openstackgerrit | Merged openstack/kolla: Fix using rhel as a base build image https://review.openstack.org/372570 | 04:28 |
| Jeffrey4l_ | bjolof__, reconfiguration run in serial, i.e. reconfigure 30% node at one time. so it is slow. | 04:29 |
| bjolof__ | i have asked before, but i do it again. Is anyone doing any active testing on minimal/docker host os's. i.e. coreos, ranchos, atomic, etc? | 04:30 |
| Jeffrey4l_ | no idea | 04:30 |
| *** neilus has quit IRC | 04:30 | |
| bjolof__ | Jeffrey4l_, ok, but still why should a reconfigure take 92min? | 04:31 |
| Jeffrey4l_ | it shouldn't. we need some optimize in the future. | 04:31 |
| Jeffrey4l_ | a trick method is: if you know which configuration is change, you can use `--tag` parameter. | 04:32 |
| Jeffrey4l_ | like `kolla-ansible reconfigure -t neutron` | 04:32 |
| Jeffrey4l_ | bjolof__, ^ | 04:33 |
| Jeffrey4l_ | how many node are you deploying? | 04:33 |
| bjolof__ | 3 net, 3 cont, 48 compute | 04:33 |
| openstackgerrit | Merged openstack/kolla: Install extra packages for gnocchi using setuptools extra feature https://review.openstack.org/372529 | 04:34 |
| Jeffrey4l_ | ok. did u use a high ansible fork parameter? it may make the task faster. | 04:35 |
| Jeffrey4l_ | But reconfigure is not idea for sure. | 04:35 |
| bjolof__ | 30 | 04:35 |
| *** v1k0d3n has quit IRC | 04:35 | |
| Jeffrey4l_ | idea/ideal | 04:36 |
| *** Jeffrey4l_ is now known as Jeffrey4l | 04:36 | |
| bjolof__ | Jeffrey4l_, we had a discussion here a couple of hours ago (sdake, inc0) they asked me to rerun the deploy, and reconfigure and report in | 04:36 |
| Jeffrey4l | ok. | 04:37 |
| *** zhurong has quit IRC | 04:43 | |
| *** bjolof__ has quit IRC | 04:45 | |
| *** sdake has joined #openstack-kolla | 04:46 | |
| *** tonanhngo has quit IRC | 04:50 | |
| *** tonanhngo has joined #openstack-kolla | 04:51 | |
| *** williamcaban has joined #openstack-kolla | 04:51 | |
| *** williamcaban has quit IRC | 04:56 | |
| *** huikang has quit IRC | 04:57 | |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: Change Mariadb default character set to utf8 https://review.openstack.org/371869 | 05:04 |
| *** jroll has quit IRC | 05:09 | |
| *** jroll has joined #openstack-kolla | 05:09 | |
| *** jroll has quit IRC | 05:09 | |
| *** jroll has joined #openstack-kolla | 05:09 | |
| *** aswadr_ has joined #openstack-kolla | 05:29 | |
| *** neilus has joined #openstack-kolla | 05:35 | |
| *** zhurong has joined #openstack-kolla | 05:35 | |
| *** hieulq has quit IRC | 05:36 | |
| *** phuongnh has quit IRC | 05:37 | |
| *** duonghq has quit IRC | 05:38 | |
| *** phuongnh has joined #openstack-kolla | 05:38 | |
| *** neilus has quit IRC | 05:40 | |
| openstackgerrit | dupengfei proposed openstack/kolla: Support shared path for glance images and nova instances disk https://review.openstack.org/372420 | 05:43 |
| *** jtriley has quit IRC | 05:46 | |
| *** zhurong has quit IRC | 05:47 | |
| *** zhurong has joined #openstack-kolla | 05:47 | |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: Remove no-cache option in build.py https://review.openstack.org/372901 | 05:48 |
| *** msimonin has joined #openstack-kolla | 05:51 | |
| *** hieulq has joined #openstack-kolla | 05:51 | |
| sdake | Jeffrey4l on the no-cache removal | 05:53 |
| Jeffrey4l | sdake, yep | 05:53 |
| sdake | can you explain the reasoning | 05:53 |
| Jeffrey4l | we have two option about this now: --cache and --no-cache | 05:54 |
| Jeffrey4l | it is historical issue. | 05:54 |
| Jeffrey4l | before we using oslo.config, we have --no-cache parameter. | 05:55 |
| sdake | what does --cache do? | 05:55 |
| Jeffrey4l | after we start using oslo.config, we add --cache parameter. | 05:55 |
| Jeffrey4l | the root cause for this change is: oslo.config support automatically negative option for boolean type. | 05:56 |
| *** msimonin has quit IRC | 05:56 | |
| Jeffrey4l | if we still use --no-cache parameter in oslo.config, it will gernerate a option named --nono-cache automatically. | 05:56 |
| Jeffrey4l | therefore, we moved to --cache and --nocache will be generated automatically. | 05:57 |
| Jeffrey4l | and marked --no-cache deprecated for removal . | 05:57 |
| Jeffrey4l | sdake, ^ | 05:57 |
| Jeffrey4l | this is done in newton cycle. | 05:57 |
| Jeffrey4l | so i think it is time to remove it. | 05:57 |
| Jeffrey4l | sorry, this is done in mitaka cycle. | 05:58 |
| Jeffrey4l | sdake, ^^ | 05:58 |
| *** senk has joined #openstack-kolla | 06:00 | |
| Jeffrey4l | make sense? | 06:03 |
| sdake | sorry was busy | 06:03 |
| sdake | let me read | 06:03 |
| Jeffrey4l | np | 06:04 |
| sdake | ok sounds good | 06:04 |
| sdake | my amp fuse blew, and thank god nothing was damaged | 06:04 |
| sdake | een jerking around with that for the elast 5 hours | 06:04 |
| Jeffrey4l | need a better one :) | 06:06 |
| *** egonzalez90 has joined #openstack-kolla | 06:07 | |
| egonzalez90 | morning | 06:10 |
| *** shardy has joined #openstack-kolla | 06:12 | |
| sdake | a better fuse? agree | 06:13 |
| sdake | my amp is chinese made | 06:13 |
| sdake | its pretty much top of the line for the price point | 06:13 |
| sdake | line magnetic | 06:13 |
| sdake | hey egonzalez90 | 06:13 |
| openstackgerrit | Merged openstack/kolla: Add pulling ceilometer-collector image https://review.openstack.org/372727 | 06:14 |
| zhubingbing | ping huikang | 06:16 |
| *** rstarmer has quit IRC | 06:17 | |
| sdake | zhubingbing - this morning berendt mentioned barbicana no workey | 06:19 |
| sdake | zhubingbing the implication here is it wasn't tested | 06:19 |
| zhubingbing | ok | 06:19 |
| sdake | can you dig into testing it and making it work properly? | 06:19 |
| zhubingbing | yes | 06:19 |
| zhubingbing | i testing it to working | 06:19 |
| sdake | in additin there is a problem that the default config uses a non-ha file-backed store | 06:20 |
| sdake | it needs to be ha | 06:20 |
| zhubingbing | hmm | 06:20 |
| zhubingbing | sureļ¼ i will contact berendt | 06:20 |
| *** rstarmer has joined #openstack-kolla | 06:21 | |
| *** jtriley has joined #openstack-kolla | 06:25 | |
| sdake | souns good | 06:25 |
| sdake | might research the ha thing | 06:25 |
| sdake | and the file backed storage | 06:25 |
| zhubingbing | ok | 06:26 |
| zhubingbing | barbican ha ? | 06:26 |
| zhubingbing | your mean is barbican ha? | 06:26 |
| zhubingbing | i see | 06:27 |
| *** huhaoran has joined #openstack-kolla | 06:28 | |
| *** jtriley has quit IRC | 06:30 | |
| *** senk has quit IRC | 06:31 | |
| sdake | i mean can ou run the barbican server on 3 nodes and it not implode | 06:36 |
| sdake | and wor kif one o the nodes fail | 06:36 |
| sdake | work if | 06:36 |
| *** duonghq has joined #openstack-kolla | 06:37 | |
| *** tonanhngo has quit IRC | 06:39 | |
| *** tonanhngo has joined #openstack-kolla | 06:39 | |
| *** williamcaban has joined #openstack-kolla | 06:39 | |
| openstackgerrit | Md Nadeem proposed openstack/kolla: Remove all bytcode & objectcode before running tox https://review.openstack.org/372944 | 06:41 |
| *** unicell1 has joined #openstack-kolla | 06:43 | |
| *** tonanhngo has quit IRC | 06:43 | |
| *** unicell has quit IRC | 06:43 | |
| *** williamcaban has quit IRC | 06:44 | |
| *** bjolo__ is now known as bjolo | 06:53 | |
| bjolo | morning | 06:53 |
| duonghq | morning bjolo | 06:53 |
| bjolo | new day, new builds, new issues to resolve | 06:54 |
| bjolo | sdake, did you see my post above about deploy and reconfigure? | 06:54 |
| sdake | no my alptop was closed s i was out to dinne with my son | 06:55 |
| sdake | what wer the results? | 06:55 |
| bjolo | about the same | 06:55 |
| sdake | with tls disaled? | 06:55 |
| bjolo | http://paste.openstack.org/show/581647 | 06:55 |
| sdake | ok fair enough -its not tls | 06:56 |
| sdake | theory disproved | 06:56 |
| sdake | Jeffrey4l - re reconfigure | 06:56 |
| sdake | bjolo is running into some odd behavior with nova | 06:56 |
| bjolo | yes tls disabled | 06:57 |
| Jeffrey4l | yep | 06:57 |
| sdake | Jeffrey4l can you fill me in on what problems you are aware of with reconfigure? | 06:57 |
| bjolo | no changes what so ever between deploy and reconfiure | 06:57 |
| sdake | bjolo roger - thts the data point i wanted | 06:57 |
| sdake | Jeffrey4l see bjolo's paste | 06:58 |
| Jeffrey4l | 1. more task is running. 2. we run task in serial, which will increase the running time. | 06:58 |
| sdake | expnd on 1 | 06:58 |
| Jeffrey4l | 1. more tasks are ran in reconfigure | 06:58 |
| sdake | all tasks are run in serial? | 06:58 |
| Jeffrey4l | yep. | 06:58 |
| sdake | why the performanee regresoin from 2.0.2 to master? | 06:59 |
| Jeffrey4l | issue 2 is more serious. | 06:59 |
| sdake | bjolo Jeffrey4l is the real expert on reconfigure | 06:59 |
| Jeffrey4l | re from 2.0.2 to master, is it? any test result for this? | 06:59 |
| sdake | the rest of us ust shoot in the dark | 06:59 |
| sdake | Jeffrey4l - manjeeets ran deploy then reconfigure backt o back | 07:00 |
| sdake | and reconfigure was about 5% slower | 07:00 |
| sdake | then 200% slower | 07:00 |
| sdake | he had a 64 ndoe cluster | 07:00 |
| Jeffrey4l | do u mean: 5% slower on 2.0.2 and 200 slower on master? | 07:00 |
| sdake | vs deploy | 07:00 |
| sdake | yes | 07:00 |
| Jeffrey4l | sdake, serial do not work on 2.0.2 . | 07:01 |
| sdake | 200% | 07:01 |
| Jeffrey4l | we use serial in wrong. | 07:01 |
| sdake | serial broken on ansible 1.9.4? | 07:01 |
| Jeffrey4l | on master, we move ther serial key word from tasks to playbooks. | 07:01 |
| Jeffrey4l | we use serial in wrong. do not mater 1.9 or 20 | 07:02 |
| Jeffrey4l | 2.0 | 07:02 |
| sdake | so this change makes serial work | 07:02 |
| Jeffrey4l | yes. | 07:02 |
| Jeffrey4l | but it become slower. | 07:02 |
| sdake | and serial results in dramatic performance impactin a negative way | 07:02 |
| *** duonghq has quit IRC | 07:02 | |
| sdake | bjolo does that answer your quetion | 07:03 |
| *** bjolo_ has joined #openstack-kolla | 07:03 | |
| Jeffrey4l | could u paste the /tmp/reconfigure and /tmp/deploy file out? | 07:03 |
| sdake | bjolo _ does that answer your question | 07:03 |
| Jeffrey4l | bjolo, ^^ | 07:03 |
| sdake | Jeffrey4l bjolo is the one running the tests | 07:03 |
| Jeffrey4l | yep. | 07:03 |
| sdake | Jeffrey4l he seemed to notice that nova took 5 minutes per task to run | 07:03 |
| sdake | with 48 compute nodes | 07:04 |
| sdake | thatsort of make sense to me | 07:04 |
| sdake | but not entirely | 07:04 |
| Jeffrey4l | sdake, i am thinking do we really need serial when reconfiguration? | 07:04 |
| sdake | Jeffrey4l i am not sure | 07:04 |
| sdake | your the rocket scientist behind reconfigure - you tell me | 07:04 |
| Jeffrey4l | hmm. this is related to the reconfiguration impact in openstack. | 07:05 |
| sdake | lets reframe the questoin this way - what does serial get us with reconfigure | 07:05 |
| Jeffrey4l | if you accept nova-api service is down in few seconds, so you may not need serial at all. | 07:05 |
| Jeffrey4l | the same with upgrade. | 07:05 |
| sdake | so serial buys us full uptime during reconfigure and upgrades | 07:06 |
| sdake | maximum uptime ;) | 07:06 |
| sdake | I have a pair of boxers with that printed on it ;) | 07:06 |
| Jeffrey4l | it is. | 07:07 |
| sdake | some swag from some conference | 07:07 |
| sdake | can serial be conditional | 07:07 |
| Jeffrey4l | but in some case, we may not need that. and should run the reconfigure/upgrade as fast as possible. | 07:07 |
| Jeffrey4l | yep. | 07:07 |
| sdake | i.e. run-with-serial=true or run-with-serial=flase | 07:08 |
| sdake | and let the operators decide on the tradeoff | 07:08 |
| Jeffrey4l | sdake, check this implementation https://review.openstack.org/371866 | 07:08 |
| Jeffrey4l | it is merged recently. | 07:08 |
| Jeffrey4l | we can export the serial parameter to a shell script. | 07:09 |
| Jeffrey4l | (check the kolla-ansible script changes ) | 07:09 |
| *** neilus has joined #openstack-kolla | 07:09 | |
| Jeffrey4l | bjolo, could u try reconfigure with the latest code and remove the serial=30% in kolla-ansible? | 07:10 |
| sdake | Jeffrey4l where is the serial % overeride in your patch? | 07:12 |
| Jeffrey4l | sdake, we need change the script | 07:12 |
| Jeffrey4l | https://review.openstack.org/#/c/371866/2/tools/kolla-ansible | 07:12 |
| Jeffrey4l | export the -e serial=30% to be the script options. | 07:12 |
| sdake | i see for upgrade and reconfgiure its set to 30% | 07:12 |
| sdake | so i think the short answer is it isnt configurable | 07:13 |
| *** msimonin has joined #openstack-kolla | 07:13 | |
| Jeffrey4l | so if we want support reconfigure/upgrade without serial, we need change the kolla-ansible script. | 07:13 |
| *** neilus has quit IRC | 07:14 | |
| openstackgerrit | zhubingbing proposed openstack/kolla: add etcd reconfigure file https://review.openstack.org/372968 | 07:14 |
| *** matrohon has joined #openstack-kolla | 07:15 | |
| sdake | Jeffrey4l i think we want to give opertors a choice | 07:15 |
| Jeffrey4l | cool i think so :) | 07:15 |
| Jeffrey4l | i will push a PS to implement this. | 07:15 |
| sdake | wfm | 07:15 |
| sdake | get into bjolo 's hans if he is willing to apply to his tree and test it out | 07:16 |
| sdake | and see if it satisfies him | 07:16 |
| sdake | bjolo what we are talking about here is a tradeoff | 07:16 |
| sdake | performance vs maximum availability | 07:16 |
| sdake | can't have both | 07:16 |
| sdake | have to pick which one you prefer | 07:17 |
| sdake | Jeffrey4l push some docs with teh change | 07:17 |
| sdake | i'll clean em up if needed | 07:17 |
| Jeffrey4l | ok. | 07:17 |
| sdake | (the docs part) | 07:17 |
| Jeffrey4l | np. thanks. | 07:18 |
| *** shardy has quit IRC | 07:19 | |
| *** jtriley has joined #openstack-kolla | 07:19 | |
| sdake | two victories in one day | 07:19 |
| sdake | i'd call that earning some leep | 07:20 |
| sdake | night all :) | 07:20 |
| Jeffrey4l | good night :) | 07:20 |
| bjolo_ | gnight sdake | 07:20 |
| bdaca | good night sdake :) | 07:21 |
| Jeffrey4l | bjolo_, did u see my talk with sdake ? | 07:21 |
| bjolo_ | reading right now | 07:22 |
| Jeffrey4l | bjolo_, 1. could u paster you /tmp/deploy and /tmp/reconfiguration file out? 2. could u try re-run reconfiguration without serial ( need change the one script ) | 07:22 |
| Jeffrey4l | thanks. | 07:22 |
| *** senk has joined #openstack-kolla | 07:23 | |
| *** jtriley has quit IRC | 07:24 | |
| *** msimonin has quit IRC | 07:25 | |
| *** msimonin has joined #openstack-kolla | 07:27 | |
| *** mickt has quit IRC | 07:30 | |
| bjolo_ | in a meeting | 07:30 |
| bjolo_ | give me a few | 07:30 |
| bjolo_ | http://paste.openstack.org/show/581715/ | 07:31 |
| bjolo_ | http://paste.openstack.org/show/581716/ | 07:31 |
| bjolo_ | i had two nodes that failed on the deploy, but should not make any difference | 07:32 |
| Jeffrey4l | got. | 07:32 |
| bjolo_ | one thing that gets me with reconfigure is that it does not seem to "fail fast" i.e. skip TASK for services that are not running/deployed | 07:32 |
| bjolo_ | i.e. reconfigure spend several minutes on vpnaas | 07:33 |
| bjolo_ | i dont run vpnaas | 07:33 |
| bjolo_ | not enabled | 07:33 |
| Jeffrey4l | ok. | 07:33 |
| Jeffrey4l | you paste is not complemented. | 07:33 |
| bjolo_ | ? | 07:33 |
| Jeffrey4l | it is only 1300 lines. | 07:33 |
| bjolo_ | ok could be too long | 07:33 |
| Jeffrey4l | I think there will be more. | 07:34 |
| Jeffrey4l | could u sent it to me? | 07:34 |
| bjolo_ | i used lodgeit.py | 07:34 |
| bjolo_ | sure | 07:34 |
| Jeffrey4l | ok. | 07:34 |
| bjolo_ | whats your email | 07:34 |
| Jeffrey4l | zhang.lei.fly@gmail.com bjolo_ | 07:34 |
| Jeffrey4l | bjolo_ there is another tool which can anlysis the playbooks running time. https://ara.readthedocs.io/en/latest/ | 07:37 |
| Jeffrey4l | the usage is easy. | 07:37 |
| bjolo_ | cool tnx, you read my mind :) | 07:38 |
| Jeffrey4l | it will be helpful to generate a ara report for deploy and reconfiguration if u can. | 07:38 |
| bjolo_ | was just about to ask if there was a way to get ansible to print timestamps etc | 07:38 |
| Jeffrey4l | bjolo_, ara is more powerful | 07:39 |
| Jeffrey4l | here is a example http://logs.openstack.org/01/372901/1/check/gate-kolla-dsvm-deploy-centos-binary-centos-7-nv/ccb924c/logs/playbook_reports/ | 07:39 |
| bjolo_ | cool | 07:40 |
| *** salv-orlando has joined #openstack-kolla | 07:42 | |
| *** salv-orl_ has quit IRC | 07:44 | |
| egonzalez90 | Jeffrey4l, thanks for the resource :) | 07:45 |
| *** zhurong has quit IRC | 07:55 | |
| bjolo | Jeffrey4l, ara installed | 07:55 |
| bjolo | running a reconfigure now | 07:55 |
| openstackgerrit | zhubingbing proposed openstack/kolla: add etcd reconfigure content https://review.openstack.org/372968 | 07:55 |
| bjolo | but now on to my real problem. Enable TLS does not work for me. On horizon login page i get "Unable to establish connection to keystone endpoint." | 07:56 |
| bjolo | TLS worked in 2.0.2. same certificate, inventory and globals file | 07:56 |
| *** athomas has joined #openstack-kolla | 08:00 | |
| *** zhurong has joined #openstack-kolla | 08:13 | |
| *** jtriley has joined #openstack-kolla | 08:13 | |
| *** Serlex has joined #openstack-kolla | 08:13 | |
| *** berendt has joined #openstack-kolla | 08:17 | |
| *** jtriley has quit IRC | 08:19 | |
| zhubingbing | ping berendt | 08:21 |
| berendt | hello zhubingbing | 08:21 |
| zhubingbing | have you free time? | 08:21 |
| zhubingbing | we talk about barbican /? | 08:21 |
| berendt | yes of course | 08:22 |
| *** gfidente has joined #openstack-kolla | 08:24 | |
| *** bradjones has quit IRC | 08:33 | |
| *** bradjones has joined #openstack-kolla | 08:36 | |
| *** bradjones has quit IRC | 08:36 | |
| *** bradjones has joined #openstack-kolla | 08:36 | |
| *** v1k0d3n has joined #openstack-kolla | 08:37 | |
| *** v1k0d3n has quit IRC | 08:42 | |
| *** rstarmer has quit IRC | 08:47 | |
| dupengfei_ | Hello, english speakers, who can help me to translate this message:" OpenFlow Agent (ofagent) Mechanism driver is deprecated in favor of OpenvSwitch mechanism driver with ānativeā of_interface in the Mitaka release and will be removed in the next release. " | 08:57 |
| dupengfei_ | thanks | 08:57 |
| *** dupengfei_ has quit IRC | 08:57 | |
| *** dupengfei_ has joined #openstack-kolla | 08:57 | |
| dupengfei_ | Hello, english speakers, who can help me to translate this message:" OpenFlow Agent (ofagent) Mechanism driver is deprecated in favor of OpenvSwitch mechanism driver with ānativeā of_interface in the Mitaka release and will be removed in the next release. | 08:57 |
| *** neilus has joined #openstack-kolla | 08:58 | |
| dupengfei_ | Is native is deprecated or reserved? | 08:58 |
| *** neilus has quit IRC | 09:02 | |
| *** ajpower has quit IRC | 09:02 | |
| *** ajpower has joined #openstack-kolla | 09:03 | |
| openstackgerrit | Li Yingjun proposed openstack/kolla: Fix murano auth issue https://review.openstack.org/372279 | 09:05 |
| *** duonghq has joined #openstack-kolla | 09:05 | |
| *** dwalsh has joined #openstack-kolla | 09:06 | |
| *** jtriley has joined #openstack-kolla | 09:07 | |
| *** ajpower has quit IRC | 09:10 | |
| bdaca | dupengfei_: I understand this as: OpenFlow Agent (ofagent) Mechanism driver will be removed in the next release, so use OpenvSwitch mechanism driver with ļæ½nativeļæ½ of_interface | 09:10 |
| dupengfei_ | bdaca: thanks | 09:12 |
| *** jtriley has quit IRC | 09:12 | |
| *** jmccarthy has quit IRC | 09:15 | |
| *** jmccarthy has joined #openstack-kolla | 09:15 | |
| *** ajpower has joined #openstack-kolla | 09:29 | |
| *** prithiv has joined #openstack-kolla | 09:33 | |
| prithiv | hi. anyone facing issue with template-override option when building images | 09:36 |
| *** dwalsh has quit IRC | 09:36 | |
| openstackgerrit | Waldemar Znoinski proposed openstack/kolla: map /dev from the host inside the ironic-conductor https://review.openstack.org/372119 | 09:37 |
| openstackgerrit | Waldemar Znoinski proposed openstack/kolla: use ironic_conductor volume for conductor's /var/lib/ironic https://review.openstack.org/372118 | 09:37 |
| *** daneyon has joined #openstack-kolla | 09:38 | |
| openstackgerrit | zhubingbing proposed openstack/kolla: add etcd reconfigure content https://review.openstack.org/372968 | 09:40 |
| *** daneyon has quit IRC | 09:43 | |
| *** bdaca has quit IRC | 09:47 | |
| openstackgerrit | Merged openstack/kolla: Repair swift reconfiguration https://review.openstack.org/372631 | 09:47 |
| openstackgerrit | Merged openstack/kolla: Set ownership of barbican configuration files to barbican https://review.openstack.org/372705 | 09:48 |
| bjolo | Jeffrey4l, reconfigure completed. how do you want the ara log? | 09:48 |
| *** prithiv has quit IRC | 09:50 | |
| *** bdaca has joined #openstack-kolla | 09:50 | |
| *** shardy has joined #openstack-kolla | 09:53 | |
| *** prithiv has joined #openstack-kolla | 09:58 | |
| *** prithiv has quit IRC | 09:58 | |
| *** jtriley has joined #openstack-kolla | 10:02 | |
| *** zhurong has quit IRC | 10:03 | |
| *** phuongnh has quit IRC | 10:05 | |
| *** jtriley has quit IRC | 10:07 | |
| bjolo | latest reconfigure took 106min. Nova part took about 70min of that | 10:08 |
| *** williamcaban has joined #openstack-kolla | 10:16 | |
| pbourke | bjolo: how many nodes is that? | 10:18 |
| Jeffrey4l | bjolo, wow. could u send me the result? | 10:18 |
| bjolo | 54 hosts. 3 network, 3 controller, 48 compute | 10:19 |
| bjolo | all machines are highend | 10:19 |
| bjolo | compute BL460 G8, dual E5-2690v2, 192GB ram, 10Gb | 10:20 |
| bjolo | single nic | 10:20 |
| Jeffrey4l | there must be some bug in nova. | 10:20 |
| *** williamcaban has quit IRC | 10:20 | |
| bjolo | network, controllers: HP DL360 G7, dual xeon 5690, 92 GB memory, SSD, 2x10Gb lacp bond | 10:21 |
| bjolo | kolla master is a virtual machine on vmware, but it has 8Gb memory, 8 cores (host cpu is xeon 5675), 1gb nic but host has 2x10 | 10:22 |
| *** tyrola has joined #openstack-kolla | 10:23 | |
| *** duonghq has left #openstack-kolla | 10:23 | |
| openstackgerrit | Christian Berendt proposed openstack/kolla: Improve index of docs https://review.openstack.org/373136 | 10:24 |
| openstackgerrit | Christian Berendt proposed openstack/kolla: Improve index of docs https://review.openstack.org/373136 | 10:26 |
| *** williamcaban has joined #openstack-kolla | 10:26 | |
| openstackgerrit | Christian Berendt proposed openstack/kolla: Improve index of docs https://review.openstack.org/373136 | 10:28 |
| *** ntpttr has quit IRC | 10:31 | |
| bjolo | Jeffrey4l, whats your email again. i mail you the generated ara log | 10:34 |
| *** williamcaban has quit IRC | 10:35 | |
| *** tyrola has quit IRC | 10:37 | |
| *** ntpttr has joined #openstack-kolla | 10:39 | |
| coolsvap | bjolo: its zhang.lei.fly@gmail.com | 10:39 |
| bjolo | tnx coolsvap | 10:39 |
| pbourke | it should not take anywhere near that long | 10:40 |
| *** neilus has joined #openstack-kolla | 10:46 | |
| *** tyrola has joined #openstack-kolla | 10:47 | |
| *** rhallisey has joined #openstack-kolla | 10:47 | |
| *** tonanhngo has joined #openstack-kolla | 10:47 | |
| *** eaguilar has joined #openstack-kolla | 10:49 | |
| *** neilus has quit IRC | 10:50 | |
| *** tonanhngo has quit IRC | 10:51 | |
| *** tyrola has quit IRC | 10:54 | |
| *** jtriley has joined #openstack-kolla | 10:56 | |
| pbourke | bjolo: any idea which step is taking the majority of the time? e.g. docker restart, contacting the nodes, something else? | 10:59 |
| Jeffrey4l | sorry a little busy. | 11:00 |
| Jeffrey4l | ya, that's my mail. bjolo | 11:00 |
| *** jtriley has quit IRC | 11:01 | |
| Jeffrey4l | Got it. thanks. | 11:01 |
| *** zhubingbing has quit IRC | 11:04 | |
| bjolo | pbourke, no idea really. seem like it goes in circles (could be the serial approach) | 11:05 |
| *** ccesario has joined #openstack-kolla | 11:06 | |
| openstackgerrit | Md Nadeem proposed openstack/kolla: Test: make enforce_type=True in CONF.set_override https://review.openstack.org/366818 | 11:07 |
| *** schwicht has joined #openstack-kolla | 11:08 | |
| *** tyrola has joined #openstack-kolla | 11:20 | |
| openstackgerrit | Jiri Stransky proposed openstack/kolla: Fix base CentOS repository keys https://review.openstack.org/373190 | 11:27 |
| *** mliima has joined #openstack-kolla | 11:29 | |
| *** schwicht has quit IRC | 11:32 | |
| *** tyrola has quit IRC | 11:32 | |
| *** tyrola has joined #openstack-kolla | 11:33 | |
| *** tyrola has quit IRC | 11:33 | |
| *** huhaoran has quit IRC | 11:33 | |
| *** huhaoran has joined #openstack-kolla | 11:33 | |
| mliima | morning guys | 11:35 |
| Jeffrey4l | bjolo, do you tried disable the serial during the reconfiguration/ | 11:35 |
| bjolo | not yet | 11:35 |
| Jeffrey4l | i think it is the serial cause this issue. | 11:35 |
| Jeffrey4l | ok. | 11:36 |
| *** caowei has quit IRC | 11:36 | |
| bjolo | did you have a patch for that? | 11:36 |
| Jeffrey4l | let me push a PS to disable the PS. | 11:36 |
| Jeffrey4l | no yet. wait 1 sec | 11:36 |
| bjolo | cool | 11:36 |
| bjolo | im troubleshooting TLS right now | 11:36 |
| bjolo | does not work for me | 11:36 |
| Jeffrey4l | ok. | 11:37 |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: DO NOT MERGE: disable serial in reconfiguration and ugprade https://review.openstack.org/373199 | 11:37 |
| *** huhaoran has quit IRC | 11:37 | |
| Jeffrey4l | bjolo, check above PS. I disable serial in it ^^ | 11:37 |
| *** huhaoran has joined #openstack-kolla | 11:37 | |
| bjolo | cool | 11:38 |
| bjolo | will run that later | 11:38 |
| bjolo | actually i can run it now | 11:39 |
| bjolo | need to work on the ceph storage anyway | 11:39 |
| bjolo | context switching | 11:39 |
| Jeffrey4l | OK. thanks. | 11:40 |
| *** Menthos has joined #openstack-kolla | 11:40 | |
| *** neilus has joined #openstack-kolla | 11:41 | |
| Menthos | Hey everyone | 11:42 |
| Menthos | I'm struggling to deploy with kolla on machines that have a single NIC (with a veth + bridge) | 11:42 |
| Jeffrey4l | i predict that the reconfigure will be finished in 25 minute. | 11:42 |
| Menthos | Is there some doc somewhere on how to do this? | 11:43 |
| bjolo | ok its away | 11:43 |
| bjolo | Menthos, i use vlan interfaces. Does your switch support that? think that would be easier to do | 11:43 |
| Menthos | bjolo: Probably not, I have very few control on network equipment | 11:44 |
| Jeffrey4l | Menthos, single node? or multi node? | 11:45 |
| Menthos | multi node. | 11:45 |
| *** eaguilar has quit IRC | 11:46 | |
| Menthos | Actually it might be yes to bjolo since I use a vlan (it is provided by a particular API of the platform, I have no direct access to the switch). I am not sure what you mean by "vlan interface" | 11:46 |
| Jeffrey4l | Menthos, how about vxlan? | 11:46 |
| *** neilus has quit IRC | 11:46 | |
| Jeffrey4l | eth0 -> eth0.10 then eth0.10 used for api internal and eth0 use for neutron_external_network_interface | 11:47 |
| Menthos | Jeffrey4l: that's not supported by the infrastructure I use | 11:47 |
| Jeffrey4l | Menthos, what' neutron typo will u use? | 11:47 |
| Jeffrey4l | topological, i means. | 11:48 |
| Jeffrey4l | only one flat network? or vlan? or vxlan? | 11:48 |
| Menthos | only one flat network | 11:48 |
| Jeffrey4l | hmm. that's hard :( | 11:49 |
| *** jtriley has joined #openstack-kolla | 11:50 | |
| Menthos | To be fair, this is not production, I'm just deploying a small OS with compute to measure energy consumption. | 11:50 |
| Menthos | So any network topology works for me. | 11:51 |
| Menthos | I'm just looking for the shortest path to deployment :-) | 11:51 |
| Jeffrey4l | i think you can try vxlan first( at least you can test everything except ssh into the vm directly ) | 11:51 |
| Jeffrey4l | if so, you can create a dummy interface and set the neutron_external_interface to that dummy interface( it is useless actually ) | 11:52 |
| *** v1k0d3n has joined #openstack-kolla | 11:52 | |
| *** dwalsh has joined #openstack-kolla | 11:52 | |
| Menthos | At first glance, it will be difficult because I have no control over the network. The way it works is I request a vlan from a API, and in return I get a range of IPs that I can use. | 11:53 |
| Menthos | But what you're saying should work with any vlan, shouldn't it? | 11:53 |
| *** zhurong has joined #openstack-kolla | 11:53 | |
| Jeffrey4l | i am saying use vxlan. | 11:54 |
| Jeffrey4l | if you can not touch the switcher or get vlan range by api. it is impossible to use vlan. | 11:54 |
| *** coolsvap is now known as coolsvap_ | 11:55 | |
| Menthos | A dummy interface assigned to neutron_external_interface will not work with a regular vlan? | 11:55 |
| *** coolsvap_ has quit IRC | 11:55 | |
| *** jtriley has quit IRC | 11:55 | |
| *** coolsvap_ has joined #openstack-kolla | 11:55 | |
| Jeffrey4l | yes. vlan package can not be out from the phsically node. | 11:55 |
| *** jtriley has joined #openstack-kolla | 11:55 | |
| Jeffrey4l | vlan packets | 11:55 |
| *** v1k0d3n has quit IRC | 11:57 | |
| mliima | Jeffrey4l, which version you use to deploy kolla? | 11:58 |
| Jeffrey4l | mliima, master? | 11:58 |
| mliima | :/ | 11:58 |
| mliima | so the rabbitmq container is working fine? | 11:58 |
| Menthos | Jeffrey4l: okay thanks, I'll look into it | 11:58 |
| Jeffrey4l | mliima, i am develop :) | 11:58 |
| Jeffrey4l | mliima, no.. i hit rabbitmq today. | 11:58 |
| mliima | ok Jeffrey4l | 11:59 |
| Jeffrey4l | checking the root cause. | 11:59 |
| Jeffrey4l | mliima, what error message you got? | 11:59 |
| mliima | np Jeffrey4l | 11:59 |
| *** dave-mccowan has joined #openstack-kolla | 11:59 | |
| mliima | i don't remember | 12:00 |
| Jeffrey4l | OK. | 12:00 |
| mliima | the container wasn't uping | 12:00 |
| *** jtriley has quit IRC | 12:00 | |
| Jeffrey4l | mliima, i will appreciate if you can provide docker logs rabbitmq | 12:00 |
| openstackgerrit | caoyuan proposed openstack/kolla: Delete the unnecessary space https://review.openstack.org/373218 | 12:01 |
| *** srwilkers has joined #openstack-kolla | 12:01 | |
| mliima | wait | 12:01 |
| *** bdaca has quit IRC | 12:02 | |
| *** dwalsh has quit IRC | 12:02 | |
| mliima | Jeffrey4l, docker images -> http://paste.openstack.org/show/581145/ | 12:03 |
| mliima | Jeffrey4l, docker logs rabbitmq -> http://paste.openstack.org/show/581150/ | 12:03 |
| Jeffrey4l | Got. thanks | 12:04 |
| *** rstarmer has joined #openstack-kolla | 12:04 | |
| *** srwilkers_ has joined #openstack-kolla | 12:04 | |
| mliima | Jeffrey4l, http://paste.openstack.org/show/582227/ | 12:05 |
| mliima | docker run 10.102.20.225:4000/kolla/ubuntu-source-rabbitmq:3.0.0 dpkg -l | grep rabbitmq ^^ | 12:05 |
| Jeffrey4l | got. thanks a lot. | 12:05 |
| Jeffrey4l | :) | 12:05 |
| openstackgerrit | caoyuan proposed openstack/kolla: Add "---" into bootstrap_service.yml https://review.openstack.org/373222 | 12:07 |
| *** rstarmer has quit IRC | 12:07 | |
| *** schwicht has joined #openstack-kolla | 12:14 | |
| *** williamcaban has joined #openstack-kolla | 12:15 | |
| openstackgerrit | Merged openstack/kolla: Change Mariadb default character set to utf8 https://review.openstack.org/371869 | 12:15 |
| openstackgerrit | Christian Berendt proposed openstack/kolla: Document that YAML data files should start with three dashes https://review.openstack.org/373232 | 12:17 |
| openstackgerrit | Merged openstack/kolla: Allow genconfig to work with kolla-kubernetes and ceph https://review.openstack.org/372811 | 12:17 |
| *** dupengfei_ has quit IRC | 12:19 | |
| openstackgerrit | Md Nadeem proposed openstack/kolla: Add support to use pdb with tox https://review.openstack.org/373233 | 12:19 |
| *** dupengfei_ has joined #openstack-kolla | 12:19 | |
| *** williamcaban has quit IRC | 12:20 | |
| *** shardy is now known as shardy_lunch | 12:20 | |
| *** daneyon has joined #openstack-kolla | 12:20 | |
| bjolo | ok Jeffrey4l looks like it will take more than 25min :( | 12:22 |
| *** srwilkers has quit IRC | 12:22 | |
| bjolo | Jeffrey4l, hanging on this one | 12:22 |
| bjolo | TASK [nova : Remove the nova libvirt, ssh, conductor, api, consoleauth and scheduler containers action=remove_container, name={{ item[0]['name'] }}] *** | 12:22 |
| openstackgerrit | Md Nadeem proposed openstack/kolla: Add support to use pdb with tox https://review.openstack.org/373233 | 12:22 |
| bjolo | have about 20 ansible threads consuming a lot of CPU, but the output is not moving | 12:23 |
| *** matrohon has quit IRC | 12:23 | |
| *** schwicht has quit IRC | 12:23 | |
| bjolo | ok now its moving again, but point is that 40 min intor the run and just started with nova tasks. Probably looking at a quite long runtime even w/o serial | 12:24 |
| bjolo | but lets see the numbers first :) | 12:24 |
| *** daneyon has quit IRC | 12:25 | |
| bjolo | so have anyone done a successful multinode multinetwork TLS deployment? | 12:25 |
| openstackgerrit | Merged openstack/kolla: Fix Keystone CADF notifications https://review.openstack.org/372738 | 12:26 |
| openstackgerrit | Merged openstack/kolla: fix horizon fernet container https://review.openstack.org/372730 | 12:26 |
| openstackgerrit | Merged openstack/kolla: Deprecate old auth parameters of ironic in nova.conf https://review.openstack.org/372306 | 12:26 |
| openstackgerrit | Merged openstack/kolla: run_once Checking mongodb backend for ceilometer https://review.openstack.org/372709 | 12:26 |
| openstackgerrit | Merged openstack/kolla: Install cinder related package for glance https://review.openstack.org/372561 | 12:27 |
| openstackgerrit | Waldemar Znoinski proposed openstack/kolla: use ironic_conductor volume for conductor's /var/lib/ironic https://review.openstack.org/372118 | 12:28 |
| *** caowei has joined #openstack-kolla | 12:29 | |
| *** ayoung has quit IRC | 12:30 | |
| *** srwilkers has joined #openstack-kolla | 12:30 | |
| *** schwicht has joined #openstack-kolla | 12:32 | |
| openstackgerrit | Waldemar Znoinski proposed openstack/kolla: map /dev from the host inside the ironic-conductor https://review.openstack.org/372119 | 12:34 |
| berendt | bjolo TLS for external HAProxy? | 12:35 |
| mliima | Jeffrey4l, i'm using master branch with 3.0.0 tag for images | 12:36 |
| mliima | should i use the latest tag? | 12:36 |
| bjolo | no normal haproxy | 12:37 |
| *** matrohon has joined #openstack-kolla | 12:38 | |
| bjolo | only thing i get in the log [Tue Sep 20 09:55:06.933287 2016] [:error] [pid 32] Unable to establish connection to https://foglight.mydoman.net:5000/v3/auth/tokens | 12:43 |
| bjolo | horizon.log | 12:43 |
| openstackgerrit | Kevin Fox proposed openstack/kolla-kubernetes: WIP: basic ceph deployment support https://review.openstack.org/373256 | 12:44 |
| *** shardy_lunch is now known as shardy | 12:50 | |
| *** huhaoran has quit IRC | 12:52 | |
| *** schwicht has quit IRC | 12:53 | |
| *** rstarmer has joined #openstack-kolla | 12:53 | |
| *** david-lyle has joined #openstack-kolla | 12:56 | |
| *** eaguilar has joined #openstack-kolla | 12:57 | |
| Jeffrey4l | bjolo, OK. | 13:04 |
| Jeffrey4l | what's the total times it takes? | 13:04 |
| bjolo | still runing | 13:05 |
| bjolo | started 1:25h ago | 13:05 |
| Jeffrey4l | hmm.. | 13:05 |
| *** mliima_ has joined #openstack-kolla | 13:09 | |
| *** mliima has quit IRC | 13:10 | |
| *** dwalsh has joined #openstack-kolla | 13:10 | |
| *** schwicht has joined #openstack-kolla | 13:12 | |
| *** eaguilar has quit IRC | 13:13 | |
| *** mliima__ has joined #openstack-kolla | 13:13 | |
| *** mgoddard__ has joined #openstack-kolla | 13:13 | |
| *** mliima_ has quit IRC | 13:16 | |
| *** mgoddard_ has quit IRC | 13:17 | |
| *** sdake has quit IRC | 13:17 | |
| bjolo | real 97m29.464s | 13:20 |
| bjolo | user 533m41.462s | 13:20 |
| bjolo | sys 29m9.598s | 13:20 |
| bjolo | Jeffrey4l, want me to generate a new ara log for you? | 13:20 |
| Jeffrey4l | bjolo, ya. it seems the result is the same. | 13:21 |
| Jeffrey4l | bjolo, are u sure u are using my PS? | 13:21 |
| bjolo | [root@eseldlx1042 tools]# cat kolla-ansible | grep -e reconfigure -e upgrade | 13:23 |
| bjolo | reconfigure Reconfigure OpenStack service | 13:23 |
| bjolo | upgrade Upgrades existing OpenStack Environment | 13:23 |
| bjolo | (upgrade) | 13:23 |
| bjolo | EXTRA_OPTS="$EXTRA_OPTS -e action=upgrade" | 13:23 |
| bjolo | (reconfigure) | 13:23 |
| bjolo | EXTRA_OPTS="$EXTRA_OPTS -e action=reconfigure" | 13:23 |
| bjolo | no serial | 13:23 |
| bjolo | [root@eseldlx1042 tools]# time ./kolla-ansible -i /etc/kolla/foglight reconfigure | 13:24 |
| Jeffrey4l | bjolo, could u show me the site.yml file? it is located in ansible/site.yml | 13:24 |
| *** dwalsh has quit IRC | 13:24 | |
| bjolo | http://paste.openstack.org/show/582244 | 13:24 |
| Jeffrey4l | ok. seems it is correct. | 13:25 |
| Jeffrey4l | bjolo, please send me the ara result. thanks. :)_ | 13:26 |
| bjolo | report running, will take a while | 13:26 |
| Jeffrey4l | OK> | 13:26 |
| bjolo | be advised, reconfigure had some fails in the end due to me shutting down horizon and keystone on 2 nodes. im troubleshooting tls issue | 13:27 |
| Jeffrey4l | bjolo, it expect to failed. when the container is down, we can not check whether the configuration is the latest. | 13:29 |
| bjolo | Jeffrey4l, is there a way i can add debug flag to a container? i.e. i want to get more output in the horizon logs | 13:29 |
| bjolo | tls works fine on command line, but not in horizon | 13:29 |
| Jeffrey4l | bjolo, what kind of debug log? | 13:29 |
| *** neilus has joined #openstack-kolla | 13:30 | |
| *** zhubingbing has joined #openstack-kolla | 13:30 | |
| berendt | is there a way to modify the delivered paste files without providing paste templates? | 13:30 |
| berendt | I would like to fix https://bugs.launchpad.net/kolla/+bug/1587747 | 13:31 |
| openstack | Launchpad bug 1587747 in kolla "Keystone: admin_token_auth middleware should be removed from paste ini file" [Low,Confirmed] | 13:31 |
| bjolo | Jeffrey4l, any type of debug from horizon | 13:31 |
| Jeffrey4l | you can get the horizon log in /var/lib/docker/volumes/kolla_log | 13:31 |
| Jeffrey4l | bjolo, ^ | 13:31 |
| bjolo | im looking there, but i dont get much helpful | 13:32 |
| bjolo | [Tue Sep 20 09:55:06.933287 2016] [:error] [pid 32] Unable to establish connection to https://foglight.mydoman.net:5000/v3/auth/tokens | 13:32 |
| bjolo | thats it | 13:32 |
| *** neilus has quit IRC | 13:34 | |
| bjolo | in the horizon container, i can ping and curl it. of course auth denied but | 13:34 |
| *** openstackstatus has joined #openstack-kolla | 13:36 | |
| *** ChanServ sets mode: +v openstackstatus | 13:36 | |
| Jeffrey4l | Unable to establish connection means can not reach. how can not curl it? | 13:37 |
| bjolo | proper curl -i -H.... authenticates fine when run inside the horizon container | 13:37 |
| bjolo | so why the heck isnt horizon finding it!? | 13:38 |
| Jeffrey4l | bjolo, i think you can run manage.py runserver in horizon container, and start the horizon in another port. the you can got the detailed logs. | 13:39 |
| bjolo | ok | 13:40 |
| bjolo | whats the whole string? | 13:40 |
| Jeffrey4l | runserver? | 13:41 |
| Jeffrey4l | hmm. it won't work. the https is forward by haproxy. | 13:41 |
| Jeffrey4l | bjolo, or try change this loglevel to info/debug here https://github.com/openstack/kolla/blob/master/ansible/roles/horizon/templates/horizon.conf.j2#L5 | 13:42 |
| *** salv-orl_ has joined #openstack-kolla | 13:42 | |
| *** gfidente has quit IRC | 13:42 | |
| -openstackstatus- NOTICE: OpenStack Infra now has a Twitter bot, follow it at https://twitter.com/openstackinfra | 13:43 | |
| *** salv-orlando has quit IRC | 13:45 | |
| *** jtriley has joined #openstack-kolla | 13:45 | |
| *** tonanhngo has joined #openstack-kolla | 13:48 | |
| *** sdake has joined #openstack-kolla | 13:49 | |
| *** dwalsh has joined #openstack-kolla | 13:51 | |
| *** tonanhngo has quit IRC | 13:53 | |
| *** mgoddard__ has quit IRC | 13:56 | |
| *** mgoddard has joined #openstack-kolla | 13:57 | |
| *** tonanhngo has joined #openstack-kolla | 14:01 | |
| *** huikang has joined #openstack-kolla | 14:01 | |
| bjolo | ok opened a bug on this | 14:01 |
| bjolo | https://bugs.launchpad.net/kolla/+bug/1625648 | 14:01 |
| openstack | Launchpad bug 1625648 in kolla "horizon login fails with TLS enabled" [Undecided,New] | 14:01 |
| *** bjolo is now known as bjolo_afk | 14:02 | |
| openstackgerrit | Md Nadeem proposed openstack/kolla: Add support to use pdb with tox https://review.openstack.org/373233 | 14:04 |
| openstackgerrit | Md Nadeem proposed openstack/kolla: Add support to use pdb with tox https://review.openstack.org/373233 | 14:05 |
| sdake | coolsvap_ thanks for bringing up benedikt's contributions | 14:06 |
| sdake | me was asleep at the wheel on that one | 14:06 |
| * sdake was.. | 14:06 | |
| *** tonanhngo has quit IRC | 14:06 | |
| *** bjolo_ has quit IRC | 14:07 | |
| *** lamt has joined #openstack-kolla | 14:07 | |
| *** daneyon has joined #openstack-kolla | 14:08 | |
| *** sdake_ has joined #openstack-kolla | 14:09 | |
| *** sdake has quit IRC | 14:11 | |
| *** inc0 has joined #openstack-kolla | 14:12 | |
| *** daneyon has quit IRC | 14:13 | |
| *** ayoung has joined #openstack-kolla | 14:15 | |
| *** tonanhngo has joined #openstack-kolla | 14:20 | |
| *** bdaca has joined #openstack-kolla | 14:24 | |
| *** fguillot has joined #openstack-kolla | 14:24 | |
| inc0 | good morning | 14:24 |
| *** gfidente has joined #openstack-kolla | 14:25 | |
| *** gfidente has joined #openstack-kolla | 14:25 | |
| berendt | have we already implemented credential encryption for keystone? | 14:31 |
| bdaca | inc0: do you have something against backporting https://bugs.launchpad.net/kolla/+bug/1623463 to mitaka? | 14:35 |
| openstack | Launchpad bug 1623463 in kolla "Kolla should support deploying Neutron in DVR mode" [Undecided,Fix released] - Assigned to Paul Bourke (pauldbourke) | 14:35 |
| inc0 | bdaca, well, I don't think it's bug for one thing | 14:36 |
| inc0 | also I'm not quite sure anyone with right mind would use DVR in Mitaka | 14:36 |
| bdaca | :D my boss asked me to try this on mitaka :D | 14:38 |
| openstackgerrit | Christian Berendt proposed openstack/kolla: Use keystone-paste.ini template for keystone https://review.openstack.org/373379 | 14:39 |
| bdaca | I will probably give it a try on my dev env, just wondered if any1 will have any use of it if I do it | 14:41 |
| openstackgerrit | Christian Berendt proposed openstack/kolla: Allow a custom haproxy configuration file https://review.openstack.org/368959 | 14:42 |
| berendt | bdaca if you tried DVR please write a report :) | 14:43 |
| openstackgerrit | Christian Berendt proposed openstack/kolla: Allow a custom keepalived configuration file https://review.openstack.org/368956 | 14:43 |
| bdaca | I am planning this at the begining of october, I will inform you about the results :) | 14:44 |
| *** senk has quit IRC | 14:47 | |
| *** kong has quit IRC | 14:51 | |
| inc0 | bdaca, what you can do is to try to install mitaka with kolla-newton | 14:53 |
| inc0 | I think I'm going to write some blogpost about it, we get lots of questions like that | 14:54 |
| inc0 | but I wouldn't bet my money of it working really well | 14:54 |
| bdaca | yeah, I have thought about this, in kolla-newton there are some features I'd like to test and use... | 14:56 |
| mliima__ | guys, running kolla-ansible deploy | 14:56 |
| mliima__ | i'm freezed here TASK [keystone : Creating admin project, user, role, service, and endpoint] **** | 14:57 |
| mliima__ | master branch | 14:57 |
| openstackgerrit | Christian Berendt proposed openstack/kolla: Add fwaas_driver.ini file to neutron-vpnaas-agent container https://review.openstack.org/373396 | 15:00 |
| *** eaguilar has joined #openstack-kolla | 15:01 | |
| *** mgoddard has quit IRC | 15:01 | |
| *** mgoddard has joined #openstack-kolla | 15:02 | |
| berendt | inc0 works well, i am doing this | 15:04 |
| *** lrensing has joined #openstack-kolla | 15:05 | |
| openstackgerrit | Merged openstack/kolla-kubernetes: Document more stuff https://review.openstack.org/371148 | 15:06 |
| *** egonzalez90 has quit IRC | 15:08 | |
| inc0 | berendt, building source containers with mitaka code? | 15:09 |
| berendt | inc0 yes | 15:09 |
| *** caowei has quit IRC | 15:10 | |
| inc0 | we keep getting this question, maybe do a little writeup how to do it with big red "not supported, you're on your own" | 15:10 |
| berendt | it is pretty easy, you only have to use a kolla-build.conf using mitaka sources... | 15:11 |
| berendt | the most important part is to set openstack-base to https://github.com/openstack/requirements/tarball/stable/mitaka | 15:11 |
| inc0 | yeah, it's the working part that I'm concerned about;) | 15:11 |
| *** zhurong has quit IRC | 15:11 | |
| berendt | but it is working pretty good | 15:11 |
| berendt | but i should mention that i only work with mature services, i think it is not a good idea to use e.g. stable/mitaka barbican or cloudkitty | 15:12 |
| *** Menthos has quit IRC | 15:13 | |
| inc0 | yeah, I don't have any idea what won't work, but it's not super tested, that's my point | 15:13 |
| inc0 | and if something breaks, we may not fix it | 15:14 |
| berendt | confirmed, we should not support it | 15:14 |
| inc0 | but it's a use-case people want, so I guess we should help | 15:14 |
| berendt | it is enough to support the stable branch with stable release | 15:14 |
| inc0 | berendt, might writing some doc part? | 15:14 |
| berendt | yes i can do this but no in this week | 15:15 |
| inc0 | with big red, "only for brave souls" | 15:15 |
| berendt | inc0 please open a bug report and assign it to me | 15:15 |
| inc0 | will do | 15:15 |
| inc0 | thanks | 15:15 |
| inc0 | berendt, https://bugs.launchpad.net/kolla/+bug/1625682 | 15:18 |
| openstack | Launchpad bug 1625682 in kolla "doc for deploying mitaka with newton kolla" [Wishlist,Confirmed] - Assigned to Christian Berendt (berendt) | 15:18 |
| *** neilus has joined #openstack-kolla | 15:18 | |
| *** pbourke has quit IRC | 15:18 | |
| *** pbourke has joined #openstack-kolla | 15:18 | |
| *** vhosakot has joined #openstack-kolla | 15:19 | |
| jmccarthy | Hi folks ! | 15:20 |
| jmccarthy | Ok got an ansible question relating to commit 423e3f3fdf07f40b46fed1125076880660d14c53 in master | 15:20 |
| jmccarthy | My question is how can the haproxy task that deals with it's template/configuration run correctly, when some of the facts it needs to populate ips for services may not be known ? | 15:21 |
| *** egonzalez90 has joined #openstack-kolla | 15:22 | |
| pbourke | Jeffrey4l: ^ | 15:22 |
| *** neilus has quit IRC | 15:22 | |
| pbourke | I think it was your commit :) | 15:22 |
| inc0 | jmccarthy, so I'm not 100% sure about this, but I think this doesn't affect facts | 15:22 |
| inc0 | just which nodes will be evaluated for task run | 15:22 |
| pbourke | haproxy.cfg needs the ips from all nodes | 15:22 |
| inc0 | yeah, but they already exists in tasks | 15:23 |
| *** tonanhngo has quit IRC | 15:23 | |
| inc0 | facts* | 15:23 |
| pbourke | facts are only discovered for hosts in that play | 15:23 |
| jmccarthy | inc0: Hmm ok I'm getting ansible throwing a wobbler when it tries to copy template over tho it seems | 15:23 |
| inc0 | yes, but there are tasks in common role that runs before | 15:23 |
| jmccarthy | For upgrade too ? | 15:23 |
| inc0 | so facts are getting populated | 15:23 |
| Jeffrey4l | pbourke, ok. checking | 15:24 |
| *** williamcaban has joined #openstack-kolla | 15:24 | |
| Jeffrey4l | jmccarthy, you need run haproxy role. | 15:24 |
| pbourke | Jeffrey4l: Im thinking without this entry https://github.com/openstack/kolla/blob/master/ansible/site.yml#L2-L7 | 15:25 |
| jmccarthy | Jeffrey41: Run it how ? It is being attempted when I do upgrade ? | 15:26 |
| pbourke | Jeffrey4l: we would need all hosts in the haproxy role, right? | 15:26 |
| Jeffrey4l | pbourke, no. | 15:26 |
| Jeffrey4l | when we want add ceilometer-api to haproxy, just `enable_ceilomter: yes` and run haproxy again, | 15:27 |
| *** msimonin has quit IRC | 15:27 | |
| Jeffrey4l | it will re-generate haproxy.cfg file with ceilometer-api related setting. | 15:27 |
| pbourke | Jeffrey4l: the question is more how does for example, does it know this fact https://github.com/openstack/kolla/blob/master/ansible/roles/haproxy/templates/haproxy.cfg.j2#L42-L44 | 15:28 |
| jmccarthy | Jeffrey41: Hmm ok yea I have enable_haproxy and some others set to yes, but that doesn't seem to be enuf ? | 15:28 |
| pbourke | Jeffrey4l: when the haproxy role is set to only run on the haproxy node | 15:28 |
| Jeffrey4l | pbourke, the groups is a magic group in ansible. it will all groups from inventory file. | 15:28 |
| openstackgerrit | Eduardo Gonzalez proposed openstack/kolla: Fix senlin api bind host https://review.openstack.org/373424 | 15:28 |
| Jeffrey4l | no matter what the - hosts is in site.yml file | 15:28 |
| pbourke | so its fetched facts from all nodes in the inventory regardless? | 15:28 |
| *** williamcaban has quit IRC | 15:29 | |
| Jeffrey4l | pbourke, yep. | 15:29 |
| Jeffrey4l | jmccarthy, what issue did you hit? | 15:29 |
| Jeffrey4l | pbourke, http://docs.ansible.com/ansible/playbooks_variables.html#magic-variables-and-how-to-access-information-about-other-hosts | 15:30 |
| Jeffrey4l | check this | 15:30 |
| *** kangh_ has joined #openstack-kolla | 15:35 | |
| openstackgerrit | Eduardo Gonzalez proposed openstack/kolla: Fix temtest missing spaces https://review.openstack.org/373431 | 15:36 |
| *** vhosakot has quit IRC | 15:36 | |
| *** berendt has quit IRC | 15:37 | |
| openstackgerrit | Eduardo Gonzalez proposed openstack/kolla: Fix tempest missing spaces https://review.openstack.org/373431 | 15:37 |
| *** huikang has quit IRC | 15:38 | |
| jmccarthy | I was getting error during config at 'Copying over haproxy.cfg' about cryptic "AnsibleUndefinedVariable: 'dict object' has no attribute u'ansible_bond0'" | 15:39 |
| inc0 | ahh this issue, hmm | 15:39 |
| Jeffrey4l | jmccarthy, are u sure you have bond0 interface in the nodes? | 15:40 |
| inc0 | ok, that makes sense, maybe it didnt propagate facts | 15:40 |
| jmccarthy | inc0: :) ! Yup | 15:40 |
| inc0 | jmccarthy, try to revert this commit and rerun | 15:40 |
| jmccarthy | Jeffrey41: 100% positive, the ip each node is ssh'd to is on a bond0 100% | 15:40 |
| inc0 | if it helps, we need to do this upstream | 15:40 |
| Jeffrey4l | jmccarthy, how about try ansible -m setup for that node. did u get the ansible_bond0? | 15:41 |
| inc0 | alternatively we can run this task on all nodes before we do anything http://docs.ansible.com/ansible/setup_module.html | 15:41 |
| jmccarthy | Jeffrey41: Yep, it's there alrighty | 15:41 |
| Jeffrey4l | hmm. | 15:42 |
| jmccarthy | inc0: Had heard me going on about this one before :P | 15:42 |
| jmccarthy | inc0: If I make sure all the hosts are back in to haproxy (like prior version) then it works altho pbourke may have hit it with does it know this fact https://github.com/openstack/kolla/blob/master/ansible/roles/haproxy/templates/haproxy.cfg.j2#L42-L44 | 15:44 |
| *** tonanhngo has joined #openstack-kolla | 15:44 | |
| inc0 | well, yeah | 15:45 |
| inc0 | I'd say better solution would be this: | 15:45 |
| jmccarthy | I'd not revert just yet :) I need to be a bit clearer on this | 15:45 |
| inc0 | https://github.com/openstack/kolla/blob/master/ansible/roles/baremetal/tasks/pre-install.yml#L7 | 15:45 |
| inc0 | on all nodes | 15:45 |
| *** matrohon has quit IRC | 15:46 | |
| inc0 | jmccarthy, so reason it might have passed testing is that most of us (devs) are running haproxy on same node as apis | 15:47 |
| *** eaguilar_ has joined #openstack-kolla | 15:52 | |
| *** eaguilar has quit IRC | 15:54 | |
| *** vhosakot has joined #openstack-kolla | 15:54 | |
| *** ntpttr_ has joined #openstack-kolla | 15:56 | |
| *** Marcellin__ has joined #openstack-kolla | 15:58 | |
| *** Serlex has quit IRC | 15:59 | |
| *** jheroux has joined #openstack-kolla | 16:02 | |
| bdaca | Has anyone any experience in using glusterFS as cinder backend in kolla? | 16:09 |
| *** senk has joined #openstack-kolla | 16:12 | |
| *** vhosakot has quit IRC | 16:13 | |
| *** dwalsh has quit IRC | 16:15 | |
| *** eaguilar_ has quit IRC | 16:19 | |
| *** Menthos has joined #openstack-kolla | 16:24 | |
| *** Menthos has quit IRC | 16:24 | |
| *** eaguilar has joined #openstack-kolla | 16:24 | |
| *** vhosakot has joined #openstack-kolla | 16:27 | |
| *** eaguilar has quit IRC | 16:29 | |
| *** eaguilar has joined #openstack-kolla | 16:29 | |
| openstackgerrit | Paul Bourke proposed openstack/kolla: Allow operators to provision multiple physical networks https://review.openstack.org/373455 | 16:35 |
| *** dwalsh has joined #openstack-kolla | 16:37 | |
| *** eaguilar has quit IRC | 16:40 | |
| *** unicell1 has quit IRC | 16:42 | |
| *** bjolo_ has joined #openstack-kolla | 16:43 | |
| *** vhosakot has quit IRC | 16:43 | |
| inc0 | pbourke, technically we should -2 this | 16:44 |
| *** vhosakot has joined #openstack-kolla | 16:44 | |
| inc0 | before we branch newton | 16:44 |
| inc0 | because it is kinda feature | 16:44 |
| inc0 | but you could publish a bug for it... | 16:44 |
| *** gfhellma has joined #openstack-kolla | 16:45 | |
| *** senk__ has joined #openstack-kolla | 16:45 | |
| *** senk has quit IRC | 16:45 | |
| *** unicell has joined #openstack-kolla | 16:50 | |
| bjolo_ | good evening | 16:51 |
| *** vhosakot has quit IRC | 16:51 | |
| jmccarthy | Jeffrey41: Ok just wondering, what is this do not merge ? https://review.openstack.org/#/c/373199/1/tools/kolla-ansible | 16:52 |
| jmccarthy | I see merged changes where serial is in site.yml, so then would that be global or per service ? | 16:53 |
| *** vhosakot has joined #openstack-kolla | 16:54 | |
| *** jtriley has quit IRC | 16:55 | |
| bdaca | good evening bjolo_ | 16:56 |
| *** mgoddard has quit IRC | 16:56 | |
| *** mgoddard_ has joined #openstack-kolla | 16:56 | |
| *** strigazi is now known as strigazi_AFK | 16:59 | |
| *** strigazi_AFK is now known as strigazi | 16:59 | |
| *** strigazi is now known as strigazi_AFK | 16:59 | |
| *** tonanhngo has quit IRC | 17:01 | |
| *** bjolof__ has joined #openstack-kolla | 17:01 | |
| *** bjolo_ has quit IRC | 17:01 | |
| *** inc0 has quit IRC | 17:02 | |
| *** inc0 has joined #openstack-kolla | 17:03 | |
| inc0 | sdake_, https://review.openstack.org/#/c/373455/1 | 17:05 |
| sdake_ | inc0 what would ou like me to do with that review | 17:06 |
| sdake_ | a link with no context is not helpful ;) | 17:06 |
| sdake_ | nm i got context | 17:07 |
| sdake_ | moment | 17:07 |
| *** jtriley has joined #openstack-kolla | 17:07 | |
| sdake_ | inc0 enjoy | 17:11 |
| *** unicell has quit IRC | 17:11 | |
| inc0 | yeah, tha'ts my thought too | 17:12 |
| inc0 | pbourke, ^ | 17:12 |
| inc0 | brb | 17:12 |
| *** eaguilar has joined #openstack-kolla | 17:13 | |
| *** inc0 has quit IRC | 17:15 | |
| *** Satya_ has joined #openstack-kolla | 17:17 | |
| Satya_ | Hi everyone | 17:17 |
| Satya_ | did anyone checked "https://bugs.launchpad.net/kolla/+bug/1625622" | 17:17 |
| openstack | Launchpad bug 1625622 in kolla "external ceph integration fails" [Undecided,Confirmed] - Assigned to Satya Sanjibani Routray (satroutr) | 17:17 |
| Satya_ | if we can decide the priority i will push the fix of it | 17:18 |
| *** williamcaban has joined #openstack-kolla | 17:22 | |
| sbezverk | akwasnie: ping | 17:26 |
| *** tonanhngo has joined #openstack-kolla | 17:28 | |
| *** dwalsh has quit IRC | 17:32 | |
| *** unicell has joined #openstack-kolla | 17:44 | |
| *** daneyon has joined #openstack-kolla | 17:45 | |
| sdake_ | Satya_ triaging now moment | 17:46 |
| Satya_ | sure | 17:46 |
| Satya_ | please let me know once triage is done | 17:46 |
| Satya_ | i will push the fix | 17:46 |
| sdake_ | Satya_ triage completed | 17:46 |
| *** tonanhngo has quit IRC | 17:47 | |
| Satya_ | Thanks pushing the fix for review in a moment | 17:47 |
| bjolof__ | pbourke, i really like the multiple external networks (change 373455) :) | 17:50 |
| *** daneyon has quit IRC | 17:50 | |
| *** zhubingbing has quit IRC | 17:51 | |
| *** zhubingbing has joined #openstack-kolla | 17:55 | |
| *** tonanhngo has joined #openstack-kolla | 18:00 | |
| sdake_ | good news everyone - my expensive amp was protected by a 59 cent fuse | 18:00 |
| sdake_ | thank god for fuses | 18:00 |
| bjolof__ | Satya_, about the ceph bug, it is high priority for me if that makes any difference :) | 18:00 |
| bjolof__ | sdake_, good :) | 18:00 |
| sdake_ | bjolof__ its marked critical at present | 18:00 |
| sdake_ | it affects the entire system | 18:00 |
| Satya_ | yes i am pushing the fix in sometime... | 18:01 |
| sdake_ | bjolof__ did you and Jeffrey4l come to a conclusion on the reconfigure problem? | 18:01 |
| bjolof__ | sdake_, not yet | 18:01 |
| bjolof__ | it is not serial | 18:01 |
| sdake_ | so the serial theory is shot down? | 18:01 |
| bjolof__ | i have done two reconfigures today, with and w/o serial. both took about the same time | 18:01 |
| bjolof__ | 96 and 102 min (or something like that) | 18:02 |
| sdake_ | 96 without serial? | 18:02 |
| bjolof__ | yes | 18:02 |
| sdake_ | i was really hopeful tht would fix theproblem | 18:02 |
| *** eaguilar_ has joined #openstack-kolla | 18:02 | |
| *** eaguilar has quit IRC | 18:04 | |
| bjolof__ | anywhere i can paste a 21MB tar file? | 18:05 |
| *** shardy has quit IRC | 18:05 | |
| *** williamcaban has quit IRC | 18:06 | |
| *** berendt has joined #openstack-kolla | 18:07 | |
| *** williamcaban has joined #openstack-kolla | 18:12 | |
| *** athomas has quit IRC | 18:14 | |
| *** senk__ has quit IRC | 18:14 | |
| *** Pavo has quit IRC | 18:16 | |
| *** vhosakot has quit IRC | 18:19 | |
| *** aswadr_ has quit IRC | 18:22 | |
| Satya_ | it will sdake :) | 18:24 |
| *** zhubingbing has quit IRC | 18:25 | |
| sdake_ | bjolof__ google drive or box comes to mind | 18:30 |
| *** srwilkers_ has quit IRC | 18:41 | |
| dave-mccowan | rhallisey hi ryan | 18:44 |
| rhallisey | dave-mccowan, hey dave | 18:44 |
| dave-mccowan | have you built kolla images with rhel lately? | 18:44 |
| rhallisey | not lately no | 18:45 |
| rhallisey | I saw there was a bug somewhere though | 18:45 |
| dave-mccowan | i patched common/config.py to add 'rhel' as an option in rh_base. not sure what tag to use. should it work with the default config, or do i need to configure cutom repos to build with the rhel? | 18:46 |
| *** williamcaban has quit IRC | 18:49 | |
| *** sdake_ is now known as sdake | 18:50 | |
| sdake | dave-mccowan there are instructions in the documenation for buildingwith rhel | 18:50 |
| sdake | last time I tested them they worked | 18:51 |
| rhallisey | dave-mccowan, I think it was rhos with base rhel | 18:51 |
| rhallisey | I don't think there was anything special needed | 18:51 |
| dave-mccowan | the instructions fail right away, 'rhel' is a valid choice for -b: https://github.com/openstack/kolla/blob/master/kolla/common/config.py#L21 | 18:54 |
| *** gfidente has quit IRC | 18:55 | |
| dave-mccowan | ah.. there was a commit yesterday. | 18:55 |
| dave-mccowan | but, i still get this error: | 18:57 |
| dave-mccowan | ERROR:kolla.image.build.base:Tag 7 not found in repository docker.io/library/rhel | 18:57 |
| *** eaguilar_ has quit IRC | 18:58 | |
| *** gfidente has joined #openstack-kolla | 19:00 | |
| mliima__ | sdake, how do you build your containers (exact command typed)? | 19:00 |
| mliima__ | i'm working on rc1 and the rabbitmq container still restarting | 19:01 |
| *** eaguilar has joined #openstack-kolla | 19:01 | |
| *** vhosakot has joined #openstack-kolla | 19:02 | |
| *** jpeeler has quit IRC | 19:03 | |
| *** mewald has joined #openstack-kolla | 19:06 | |
| sdake | ./build.py --registry 192.168.1.103:4000 --type source --push | 19:06 |
| sdake | i also run | 19:06 |
| sdake | ./build.py --registry 192.168.1.103:4000 --push (to run through centos binary) | 19:06 |
| sdake | mliima__ lets dig into your restarting problem more thouroughly - rather then guessing at the caue | 19:07 |
| sdake | because if its an actual bug its critical | 19:07 |
| mliima__ | sdake, https://bugs.launchpad.net/kolla/+bug/1624263 | 19:07 |
| openstack | Launchpad bug 1624263 in kolla "Adding a new RabbitMQ node to an existing cluster" [Medium,Confirmed] | 19:07 |
| sdake | ar you adding a new node? | 19:08 |
| sdake | I thought you started fresh | 19:08 |
| mliima__ | no no | 19:08 |
| mliima__ | sdake,are you working on which version? | 19:10 |
| sdake | mliima__ can you be a king and try the suggested workaround | 19:10 |
| sdake | masster latest | 19:10 |
| mliima__ | oh god | 19:10 |
| sdake | mliima__ sup? | 19:10 |
| mliima__ | master latest is working for you? | 19:11 |
| sdake | it did yesterday and the day before | 19:12 |
| sdake | i haven't tried today | 19:12 |
| sdake | no time yet | 19:12 |
| *** huikang has joined #openstack-kolla | 19:12 | |
| sdake | and have hairs to cut at 3pm so may not get to it until later | 19:12 |
| mliima__ | np sdake | 19:12 |
| mliima__ | i think that the problem can be my environment | 19:12 |
| mliima__ | :/ | 19:12 |
| mliima__ | i ran tools/diag | 19:13 |
| mliima__ | http://paste.openstack.org/show/582309/ | 19:13 |
| *** vhosakot has quit IRC | 19:14 | |
| dave-mccowan | sdake rhallisey could you try to build one container? tools/build.py -b rhel -i ./rhel-include keystone (if it works for you, then i know it's my environment, at least) :-) | 19:16 |
| *** eaguilar has quit IRC | 19:17 | |
| *** gfhellma has quit IRC | 19:25 | |
| sdake | mliima__ definatey environmental problem | 19:26 |
| sdake | kolla rc1 i an installed package | 19:26 |
| sdake | pip uninstalal kolla | 19:26 |
| sdake | i was pretty sure we have been through this ;) | 19:26 |
| sdake | deeveloeprs work from master git, operators work from pypi packaging | 19:26 |
| sdake | its all in th edocs | 19:26 |
| bjolof__ | how much logging do you guys see in /var/lib/docker/volumes/kolla_logs/_data/haproxy/haproxy.log? | 19:28 |
| *** eaguilar has joined #openstack-kolla | 19:29 | |
| bjolof__ | this is all i got in my haproxy log http://paste.openstack.org/show/582310/ | 19:30 |
| mliima__ | sdake, | 19:31 |
| mliima__ | works | 19:31 |
| mliima__ | haha | 19:31 |
| mliima__ | enviroment problem | 19:31 |
| sdake | mliima__ ya - points at a larger problem | 19:31 |
| sdake | and that is perhaps upgrades are broken | 19:31 |
| sdake | or perhaps they are not | 19:31 |
| mliima__ | well | 19:32 |
| mliima__ | it works | 19:32 |
| sdake | that theory needs to be validated | 19:32 |
| mliima__ | i'm happy | 19:32 |
| mliima__ | kolla is good | 19:32 |
| mliima__ | i love kolla | 19:32 |
| mliima__ | i love all | 19:32 |
| sdake | upgrade from 2.0.2 to 3.0.0 is what I mean | 19:32 |
| sdake | we like you too mliima__ :) | 19:32 |
| mliima__ | :) | 19:32 |
| *** tonanhngo has quit IRC | 19:42 | |
| *** salv-orlando has joined #openstack-kolla | 19:42 | |
| bjolof__ | anyone check their haproxy log if they actually see any traffic logging_ | 19:44 |
| bjolof__ | ? | 19:44 |
| *** salv-orl_ has quit IRC | 19:45 | |
| Satya_ | Hi sdake | 19:47 |
| sdake | Satya_ sup | 19:47 |
| sdake | bjolof__ traffic logging? | 19:47 |
| sdake | Ididn't know we had such a feature ;) | 19:47 |
| Satya_ | i getting a error while doing the git review | 19:47 |
| sdake | Satya_ which error | 19:47 |
| sdake | Satya_ i can help you through it | 19:48 |
| sdake | brb ,checking battery | 19:48 |
| *** tonanhngo has joined #openstack-kolla | 19:48 | |
| Satya_ | http://pastebin.com/dXTmDqFy | 19:48 |
| bjolof__ | i dont know haproxy logging that well, its kinda advanced ... but looking at the haproxy config and the documentation we should see a lot more compared to what i have in my logs | 19:49 |
| bjolof__ | im trying to troubleshoot why my horizon TLS logins just die silently. i.e. horizon logs that it can not connect to keystone. | 19:50 |
| bjolof__ | keystone logs shows nothing, they are blank | 19:50 |
| *** bdaca has quit IRC | 19:51 | |
| bjolof__ | and haproxy does not seem to log properly. i.e. from what i understand we should se more in the logfiles | 19:51 |
| sdake | bjolof__ - dave-mccowan expert on this stuff | 19:51 |
| bjolof__ | https://bugs.launchpad.net/kolla/+bug/1625648 | 19:51 |
| openstack | Launchpad bug 1625648 in kolla "horizon login fails with TLS enabled" [Undecided,New] | 19:51 |
| *** gfhellma has joined #openstack-kolla | 19:51 | |
| sdake | fwiw that worked for me last time i tried it | 19:52 |
| bjolof__ | the culprit is haproxy afai can tell right now | 19:53 |
| sdake | Satya_ your config is messed up | 19:53 |
| sdake | Satya_ i can help you correct it around 5pm pst | 19:53 |
| Satya_ | i guess without -s its going to work :P | 19:53 |
| sdake | Satya_ i have things happening now - so can't hae a live webex debug sesssion of your setup | 19:53 |
| Satya_ | NP | 19:53 |
| sdake | its not goingto work ou mean? | 19:53 |
| sdake | Satya_ is this your first commit to openstack | 19:54 |
| Satya_ | nope | 19:54 |
| Satya_ | usually i do commit to nova | 19:54 |
| Satya_ | first time to kolla | 19:54 |
| Satya_ | but its different VM which i am using now | 19:54 |
| berendt | is it not possible to use ADD/COPY when using the footer blocks with template override functionality? | 19:55 |
| Satya_ | To ssh://satya@review.openstack.org:29418/openstack/kolla.git ! [remote rejected] HEAD -> refs/publish/master (invalid committer) | 19:55 |
| Satya_ | :( | 19:55 |
| *** srwilkers has quit IRC | 19:59 | |
| *** huikang has quit IRC | 19:59 | |
| *** huikang has joined #openstack-kolla | 20:00 | |
| sdake | Satya_ does the different vm have your same ssh keys? | 20:00 |
| sdake | and same login ids? | 20:00 |
| sdake | and same git information? | 20:00 |
| Satya_ | nope | 20:00 |
| Satya_ | my previous VM gone corrupt | 20:00 |
| sdake | did you register the different ssh key with gerrit? | 20:00 |
| Satya_ | i added the sshkey tough | 20:01 |
| Satya_ | yes | 20:01 |
| sdake | sam elogin id? | 20:01 |
| Satya_ | yes | 20:01 |
| sdake | i'd recommend deleting your invalid key | 20:01 |
| sdake | (in gerrit) | 20:01 |
| sdake | what about permsissions on .ssh | 20:02 |
| sdake | they should be 700 | 20:02 |
| sdake | and permissions on all files in .ssh should be 600 | 20:02 |
| *** huikang has quit IRC | 20:05 | |
| Satya_ | sdake git the problem | 20:06 |
| Satya_ | [root@dhcp-blr-ptp-mercury-l4-10-177-161-116 kolla]# git config --global user.name "satya" [root@dhcp-blr-ptp-mercury-l4-10-177-161-116 kolla]# git config --global user.email "engg.sanj@gmail.com" [root@dhcp-blr-ptp-mercury-l4-10-177-161-116 kolla]# git review remote: Processing changes: refs: 1, done remote: remote: ERROR: In commit ae3122faa75c24a37f6bf6177e46069a5933929b remote: ERROR: committer email address root@dhcp-b | 20:07 |
| Satya_ | http://paste.openstack.org/show/582311/ | 20:07 |
| *** huikang has joined #openstack-kolla | 20:07 | |
| sdake | huikang can you do me a solid | 20:07 |
| sdake | huikang i have a bunch of personal errands to run today | 20:08 |
| sdake | huikang and not sure i can get to this tomorrow | 20:08 |
| sdake | https://review.openstack.org/#/c/366948/1 | 20:08 |
| *** eaguilar has quit IRC | 20:10 | |
| sdake | Satya_ don't run as user root | 20:10 |
| sdake | run as your rela user | 20:10 |
| sdake | whatever matches your gerrit id | 20:11 |
| sdake | and learn to use sudo when appropriate would be my recommendation :) | 20:11 |
| Satya_ | :) | 20:11 |
| Satya_ | sure | 20:11 |
| sdake | i know its hard | 20:11 |
| sdake | i went through this transition i thinking about 6-7 yearsago | 20:11 |
| sdake | and it was a real struggle or me | 20:11 |
| sdake | but the benefits outweigh the costs | 20:11 |
| sdake | huikang what i'd ask is if you could validate latest version of docker-py works with kolla | 20:14 |
| sdake | huikang if you can't or dont want to do so, let me know | 20:14 |
| *** huikang has quit IRC | 20:17 | |
| *** matrohon has joined #openstack-kolla | 20:17 | |
| *** huikang has joined #openstack-kolla | 20:17 | |
| *** eaguilar has joined #openstack-kolla | 20:18 | |
| *** lrensing has quit IRC | 20:21 | |
| *** huikang has quit IRC | 20:22 | |
| *** daneyon has joined #openstack-kolla | 20:28 | |
| *** mewald has quit IRC | 20:28 | |
| *** neilus has joined #openstack-kolla | 20:30 | |
| *** mewald has joined #openstack-kolla | 20:32 | |
| *** daneyon has quit IRC | 20:33 | |
| *** jtriley has quit IRC | 20:33 | |
| manjeets | o/ | 20:33 |
| *** Jeffrey4l has quit IRC | 20:35 | |
| *** mewald1 has joined #openstack-kolla | 20:39 | |
| *** mewald has quit IRC | 20:39 | |
| *** kangh_ has quit IRC | 20:43 | |
| *** huikang has joined #openstack-kolla | 20:44 | |
| *** berendt has quit IRC | 20:48 | |
| *** huikang has quit IRC | 20:49 | |
| openstackgerrit | satya proposed openstack/kolla: ceph-* the wild card is not work for json. added the support for the configurable keyring file in the all.yml, and called the variable under the json files. https://review.openstack.org/373525 | 20:50 |
| sdake | sup manjeets | 20:51 |
| manjeets | oh sorry I accidently typed in that message to check my irc because i was unable to post on other | 20:51 |
| Satya_ | https://review.openstack.org/373525 | 20:51 |
| manjeets | sorry if it disturbed any one | 20:51 |
| sdake | all good mn | 20:53 |
| sdake | manjeets | 20:53 |
| *** jamesdenton has quit IRC | 20:55 | |
| *** LamT_ has joined #openstack-kolla | 20:55 | |
| *** inc0 has joined #openstack-kolla | 20:55 | |
| *** ccesario has quit IRC | 20:57 | |
| *** mewald1 has quit IRC | 21:03 | |
| *** gfidente has quit IRC | 21:05 | |
| *** lamt has quit IRC | 21:05 | |
| *** inc0 has quit IRC | 21:08 | |
| *** matrohon has quit IRC | 21:15 | |
| Satya_ | hi | 21:19 |
| *** jheroux has quit IRC | 21:20 | |
| *** schwicht has quit IRC | 21:25 | |
| *** salv-orlando has quit IRC | 21:26 | |
| *** salv-orlando has joined #openstack-kolla | 21:26 | |
| *** jamesdenton has joined #openstack-kolla | 21:26 | |
| *** eaguilar has quit IRC | 21:28 | |
| *** jtriley has joined #openstack-kolla | 21:28 | |
| sdake | wirehead_ can I get an ack/nack if ou will be ready to go tomorrow with a dryrun | 21:29 |
| sdake | sbezverk has chimed in | 21:29 |
| *** vhosakot has joined #openstack-kolla | 21:33 | |
| wirehead_ | I thought I did. | 21:36 |
| wirehead_ | sdake: | 21:36 |
| sdake | wirehead_ oh there was an email thread perhaps i missed it | 21:36 |
| sdake | (it being your response) | 21:36 |
| sdake | your answer was | 21:36 |
| sdake | ? | 21:36 |
| wirehead_ | +1 slidedeck. | 21:37 |
| sdake | roger sounds like a go for launch | 21:37 |
| wirehead_ | I got cornered by a one-person angry OpenStack mob last Thursday in the SJ office. | 21:37 |
| *** huikang has joined #openstack-kolla | 21:40 | |
| huikang | ping sdake | 21:40 |
| *** gfhellma has quit IRC | 21:40 | |
| sdake | huikang - need to depart in t-2 minutes for haircut | 21:41 |
| huikang | I was away from my compute and missed all you meessage | 21:41 |
| sdake | huikang what can I do for you fine sir | 21:41 |
| huikang | just saw them right now | 21:41 |
| sdake | roger | 21:41 |
| *** kong has joined #openstack-kolla | 21:41 | |
| huikang | do you still want to do those? | 21:41 |
| sdake | the tessting? yes its required | 21:41 |
| sdake | if youcan't do it i'll get to it when i can | 21:42 |
| huikang | Ok, I will test later | 21:42 |
| sdake | deploy/reconfigure/upgrade operations with latest docker-py | 21:42 |
| sdake | i gave em a 2 day timer | 21:42 |
| huikang | sure | 21:42 |
| huikang | no problem | 21:42 |
| sdake | cool thanks for the assistance | 21:42 |
| huikang | np | 21:42 |
| huikang | enjoy your hair cut | 21:43 |
| *** schwicht has joined #openstack-kolla | 21:43 | |
| *** schwicht has quit IRC | 21:46 | |
| *** schwicht has joined #openstack-kolla | 21:46 | |
| *** williamcaban has joined #openstack-kolla | 21:50 | |
| *** wznoinsk has quit IRC | 21:50 | |
| *** williamcaban has quit IRC | 21:54 | |
| *** lamt has joined #openstack-kolla | 21:56 | |
| *** schwicht has quit IRC | 21:59 | |
| *** schwicht has joined #openstack-kolla | 22:00 | |
| *** wznoinsk has joined #openstack-kolla | 22:02 | |
| *** huikang has quit IRC | 22:05 | |
| *** huikang has joined #openstack-kolla | 22:05 | |
| *** huikang has quit IRC | 22:10 | |
| *** vhosakot has quit IRC | 22:17 | |
| *** daneyon has joined #openstack-kolla | 22:21 | |
| *** rstarmer has quit IRC | 22:22 | |
| *** jtriley has quit IRC | 22:24 | |
| *** neilus has quit IRC | 22:25 | |
| *** daneyon has quit IRC | 22:26 | |
| *** Satya_ has quit IRC | 22:26 | |
| *** schwicht has quit IRC | 22:28 | |
| *** tonanhngo has quit IRC | 22:28 | |
| *** gfhellma has joined #openstack-kolla | 22:34 | |
| *** gfhellma has quit IRC | 22:48 | |
| *** schwicht has joined #openstack-kolla | 22:55 | |
| *** schwicht has quit IRC | 22:56 | |
| *** rstarmer has joined #openstack-kolla | 22:57 | |
| *** neilus has joined #openstack-kolla | 22:57 | |
| *** schwicht has joined #openstack-kolla | 22:58 | |
| kfox1111 | is the ceph guide still accurite? | 22:59 |
| *** neilus has quit IRC | 23:01 | |
| *** rstarmer has quit IRC | 23:01 | |
| *** jtriley has joined #openstack-kolla | 23:05 | |
| kfox1111 | I'm trying to understand how kolla's ansible playbooks work. | 23:09 |
| kfox1111 | it looks like it fires up 3 mon containers in parallel with a bootstrap, each overwrites the delegate_host to its own hostname? | 23:10 |
| kfox1111 | then a subsequent task execs into that one host and pulls the keys and pushes them back to all the containers that need keys? | 23:10 |
| *** awiddersheim has joined #openstack-kolla | 23:13 | |
| openstackgerrit | Andrew Widdersheim proposed openstack/kolla: Revert "Add common role to Ansible site file" https://review.openstack.org/373557 | 23:19 |
| *** daneyon has joined #openstack-kolla | 23:22 | |
| *** huikang has joined #openstack-kolla | 23:23 | |
| *** daneyon has quit IRC | 23:27 | |
| openstackgerrit | Kevin Fox proposed openstack/kolla-kubernetes: WIP: basic ceph deployment support https://review.openstack.org/373256 | 23:29 |
| *** schwicht has quit IRC | 23:31 | |
| *** eaguilar has joined #openstack-kolla | 23:32 | |
| *** jtriley has quit IRC | 23:36 | |
| *** schwicht has joined #openstack-kolla | 23:38 | |
| *** williamcaban has joined #openstack-kolla | 23:38 | |
| *** jtriley has joined #openstack-kolla | 23:43 | |
| *** williamcaban has quit IRC | 23:43 | |
| *** tonanhngo has joined #openstack-kolla | 23:52 | |
| *** neilus has joined #openstack-kolla | 23:54 | |
| *** tonanhngo has quit IRC | 23:56 | |
| *** schwicht_at_work has joined #openstack-kolla | 23:59 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!