| *** jml has quit IRC | 00:12 | |
| *** jml has joined #tripleo | 00:16 | |
| *** penick has quit IRC | 00:18 | |
| *** morgan_remote_ has quit IRC | 00:20 | |
| *** andreaf has quit IRC | 00:26 | |
| *** andreaf has joined #tripleo | 00:26 | |
| openstackgerrit | Ben Nemec proposed a change to openstack/diskimage-builder: Enable dracut deploy ramdisks https://review.openstack.org/105275 | 00:34 |
|---|---|---|
| openstackgerrit | Ben Nemec proposed a change to openstack/diskimage-builder: Refactor ramdisk element to allow alternate implementations https://review.openstack.org/124235 | 00:34 |
| *** yamahata_ has joined #tripleo | 00:41 | |
| *** pradeep has quit IRC | 00:58 | |
| *** yamahata_ has quit IRC | 00:59 | |
| *** michchap has joined #tripleo | 01:02 | |
| *** killer_prince is now known as lazy_prince | 01:18 | |
| tchaypo | okay, so the good news is that it looks like the undercloud might not have been bitten by the horizon issue... it's just that mysql on the seed died at an inconvenient time and so the CREATE_COMPLETE message couldn't be logged | 01:22 |
| *** panda has quit IRC | 01:22 | |
| *** panda has joined #tripleo | 01:23 | |
| *** penick has joined #tripleo | 01:35 | |
| *** penick_ has joined #tripleo | 01:37 | |
| *** penick has quit IRC | 01:39 | |
| *** penick_ is now known as penick | 01:39 | |
| *** weshay has quit IRC | 01:40 | |
| *** nosnos has joined #tripleo | 01:45 | |
| *** andreaf has quit IRC | 01:52 | |
| *** rhallisey has quit IRC | 02:10 | |
| *** morgan_remote_ has joined #tripleo | 02:26 | |
| *** noslzzp has quit IRC | 02:27 | |
| *** sdake_ has joined #tripleo | 02:27 | |
| *** ramishra has joined #tripleo | 02:39 | |
| *** yamahata_ has joined #tripleo | 02:40 | |
| *** penick has quit IRC | 02:43 | |
| *** nosnos has quit IRC | 02:55 | |
| *** nosnos_ has joined #tripleo | 02:56 | |
| *** nosnos_ has quit IRC | 02:57 | |
| *** nosnos has joined #tripleo | 02:57 | |
| *** untriaged-bot has joined #tripleo | 03:00 | |
| untriaged-bot | Untriaged bugs so far: | 03:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1373063 | 03:00 |
| uvirtbot | Launchpad bug 1373063 in tripleo "port not added to bridge after a reboot" [Undecided,In progress] | 03:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1369256 | 03:00 |
| uvirtbot | Launchpad bug 1369256 in tripleo "ubuntu: hash sum mismatch when installing packages" [Undecided,New] | 03:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1366164 | 03:00 |
| uvirtbot | Launchpad bug 1366164 in tripleo "Add domain roles for admin user" [Undecided,New] | 03:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1367742 | 03:00 |
| uvirtbot | Launchpad bug 1367742 in tripleo "keepalived brings the VIP up in multiple places on a controller reboot" [Undecided,New] | 03:00 |
| *** untriaged-bot has quit IRC | 03:00 | |
| *** sdake_ has quit IRC | 03:00 | |
| *** nosnos has quit IRC | 03:01 | |
| *** ramishra has quit IRC | 03:01 | |
| *** sdake has quit IRC | 03:02 | |
| *** pradeep has joined #tripleo | 03:05 | |
| *** sdake_ has joined #tripleo | 03:05 | |
| *** pcrews has quit IRC | 03:13 | |
| *** Yanyanhu has joined #tripleo | 03:18 | |
| *** ramishra has joined #tripleo | 03:23 | |
| *** lazy_prince is now known as killer_prince | 03:28 | |
| lifeless | wa hey, looks like we're going to be broken by the fix to this bash thing | 03:33 |
| tchaypo | weeeee | 03:33 |
| tchaypo | we're relying on it over-parsing functions? | 03:33 |
| lifeless | pretty sure we export a function somewhere | 03:37 |
| *** dsneddon has quit IRC | 03:40 | |
| *** dsneddon has joined #tripleo | 03:41 | |
| tchaypo | Gah. I was assuming the fix would just be to stop it over-parsing; not to stop the ability to export functions entirely | 03:41 |
| lifeless | well | 03:42 |
| lifeless | folk are examining ramifications now | 03:42 |
| lifeless | there's a thread on debian about removing support entirely | 03:42 |
| *** pcrews has joined #tripleo | 03:43 | |
| tchaypo | that'll be fun. | 03:43 |
| tchaypo | oh well, more justification for a move to python :) | 03:43 |
| *** bvivek has joined #tripleo | 03:47 | |
| *** morazi has quit IRC | 03:51 | |
| *** morazi has joined #tripleo | 03:52 | |
| *** yuanying_ has joined #tripleo | 03:59 | |
| openstackgerrit | OpenStack Proposal Bot proposed a change to openstack/python-tuskarclient: Updated from global requirements https://review.openstack.org/116415 | 03:59 |
| openstackgerrit | OpenStack Proposal Bot proposed a change to openstack/tuskar: Updated from global requirements https://review.openstack.org/122674 | 04:00 |
| openstackgerrit | OpenStack Proposal Bot proposed a change to openstack/tuskar-ui: Updated from global requirements https://review.openstack.org/124262 | 04:00 |
| *** yuanyin__ has joined #tripleo | 04:02 | |
| *** yuanying_ has quit IRC | 04:02 | |
| *** yuanying has quit IRC | 04:03 | |
| *** ramishra_ has joined #tripleo | 04:07 | |
| *** ramishra has quit IRC | 04:08 | |
| *** yamahata_ has quit IRC | 04:08 | |
| *** yamahata_ has joined #tripleo | 04:08 | |
| *** yuanyin__ has quit IRC | 04:10 | |
| *** Azher has joined #tripleo | 04:13 | |
| *** sdake has joined #tripleo | 04:15 | |
| *** yuanying has joined #tripleo | 04:15 | |
| *** nikunj2512 has joined #tripleo | 04:22 | |
| *** killer_prince is now known as lazy_prince | 04:24 | |
| *** nosnos has joined #tripleo | 04:28 | |
| tchaypo | thanks bot | 04:28 |
| openstackgerrit | Motohiro 'Yuanying' Otsuka proposed a change to openstack/os-refresh-config: Coverage report was't created https://review.openstack.org/124264 | 04:31 |
| *** akuznetsov has quit IRC | 04:45 | |
| *** akuznetsov has joined #tripleo | 04:47 | |
| *** lazy_prince is now known as killer_prince | 04:48 | |
| *** ci-overcloud has joined #tripleo | 04:49 | |
| ci-overcloud | ************** ci-overcloud complete status=1 ************ | 04:49 |
| *** ci-overcloud has quit IRC | 04:49 | |
| openstackgerrit | Motohiro 'Yuanying' Otsuka proposed a change to openstack/os-refresh-config: Coverage report was't created https://review.openstack.org/124264 | 04:51 |
| *** cody-somerville has quit IRC | 04:54 | |
| openstackgerrit | Ian Wienand proposed a change to openstack/diskimage-builder: Standarise tracing for scripts https://review.openstack.org/119023 | 05:00 |
| *** morgan_remote_ has quit IRC | 05:00 | |
| *** yuanying_ has joined #tripleo | 05:01 | |
| *** killer_prince is now known as lazy_prince | 05:02 | |
| *** yuanying has quit IRC | 05:04 | |
| *** bvivek has quit IRC | 05:08 | |
| *** tzumainn has quit IRC | 05:11 | |
| *** pcrews has quit IRC | 05:23 | |
| *** sdake_ has quit IRC | 05:27 | |
| *** bvivek has joined #tripleo | 05:29 | |
| *** pcrews has joined #tripleo | 05:30 | |
| *** rdopieralski has joined #tripleo | 05:36 | |
| *** vdreamarkitex has quit IRC | 05:37 | |
| *** rushiagr_away is now known as rushiagr | 05:41 | |
| *** loki184 has joined #tripleo | 05:42 | |
| *** yamahata_ has quit IRC | 05:43 | |
| *** yamahata_ has joined #tripleo | 05:43 | |
| *** Yanyan has joined #tripleo | 05:44 | |
| *** Yanyanhu has quit IRC | 05:47 | |
| openstackgerrit | lokesh s proposed a change to openstack/tripleo-image-elements: adding local boot support for uboot arm arch https://review.openstack.org/106727 | 05:47 |
| *** jprovazn has joined #tripleo | 05:51 | |
| *** mrunge has joined #tripleo | 05:58 | |
| *** PIEgg has joined #tripleo | 06:08 | |
| *** PIEgg has quit IRC | 06:08 | |
| *** juice has quit IRC | 06:28 | |
| *** uvirtbot has quit IRC | 06:28 | |
| *** devananda has quit IRC | 06:29 | |
| *** grauwulf has quit IRC | 06:29 | |
| *** fabbione has quit IRC | 06:29 | |
| *** fabbione has joined #tripleo | 06:29 | |
| *** devananda has joined #tripleo | 06:31 | |
| *** grauwulf has joined #tripleo | 06:31 | |
| *** juice has joined #tripleo | 06:35 | |
| *** k4n0 has joined #tripleo | 06:37 | |
| *** pblaho has joined #tripleo | 06:43 | |
| *** uvirtbot has joined #tripleo | 06:43 | |
| *** pcrews has quit IRC | 06:44 | |
| *** yuanying_ has quit IRC | 06:52 | |
| *** yuanying has joined #tripleo | 06:53 | |
| *** yuanying has quit IRC | 06:58 | |
| *** ramishra_ has quit IRC | 07:00 | |
| *** yuanying has joined #tripleo | 07:00 | |
| *** pensu has joined #tripleo | 07:03 | |
| *** pensu has quit IRC | 07:09 | |
| *** pensu has joined #tripleo | 07:12 | |
| openstackgerrit | Motohiro 'Yuanying' Otsuka proposed a change to openstack/os-refresh-config: Coverage report was't created https://review.openstack.org/124264 | 07:14 |
| *** boris-42 has quit IRC | 07:15 | |
| *** grauwulf has quit IRC | 07:15 | |
| *** athomas has joined #tripleo | 07:17 | |
| *** grauwulf has joined #tripleo | 07:18 | |
| yuanying | StevenK: are you online? | 07:25 |
| *** pensu has quit IRC | 07:25 | |
| yuanying | StevenK: I fixed https://review.openstack.org/124264 to use testr. is it ok? | 07:26 |
| *** pensu has joined #tripleo | 07:26 | |
| *** jang1 has joined #tripleo | 07:27 | |
| *** jistr has joined #tripleo | 07:28 | |
| yuanying | StevenK: and other os-*-config projects uses "python setup.py test" instead of "testr". so we should change these? | 07:29 |
| StevenK | yuanying: Does it work with testr ? | 07:35 |
| yuanying | Yes, it works. | 07:35 |
| openstackgerrit | Nicholas Randon proposed a change to openstack/tripleo-image-elements: Fix RabbitMQ element clustering start and stop https://review.openstack.org/115524 | 07:37 |
| *** ramishra has joined #tripleo | 07:42 | |
| *** gfidente has joined #tripleo | 07:43 | |
| StevenK | yuanying: I'm trying to remember if we are trying to converge on testr, I shall talk to others on Monday and comment on the review. | 07:44 |
| *** yuanying has quit IRC | 07:45 | |
| *** yuanying has joined #tripleo | 07:46 | |
| yuanying | StevenK: Thank you. | 07:46 |
| *** bvivek has quit IRC | 07:54 | |
| *** regebro has joined #tripleo | 07:55 | |
| *** jcoufal has joined #tripleo | 07:59 | |
| *** jtomasek has joined #tripleo | 08:01 | |
| *** yuanying has quit IRC | 08:06 | |
| *** yuanying has joined #tripleo | 08:06 | |
| *** yuanying has quit IRC | 08:11 | |
| *** Azher has quit IRC | 08:16 | |
| *** bvivek has joined #tripleo | 08:20 | |
| *** akrivoka has joined #tripleo | 08:24 | |
| *** lucasagomes has joined #tripleo | 08:28 | |
| *** jp_at_hp has joined #tripleo | 08:28 | |
| *** jp_at_hp has quit IRC | 08:29 | |
| *** jp_at_hp has joined #tripleo | 08:29 | |
| *** derekh has joined #tripleo | 08:35 | |
| *** lazy_prince has quit IRC | 08:37 | |
| *** killer_prince has joined #tripleo | 08:46 | |
| *** killer_prince is now known as lazy_prince | 08:46 | |
| *** jang1 has quit IRC | 08:50 | |
| *** ramishra has quit IRC | 08:57 | |
| *** akrivoka has quit IRC | 08:59 | |
| *** untriaged-bot has joined #tripleo | 09:00 | |
| untriaged-bot | Untriaged bugs so far: | 09:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1373063 | 09:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1369256 | 09:00 |
| uvirtbot | Launchpad bug 1373063 in tripleo "port not added to bridge after a reboot" [Undecided,In progress] | 09:00 |
| untriaged-bot | https://bugs.launchpad.net/os-refresh-config/+bug/1374259 | 09:00 |
| uvirtbot | Launchpad bug 1369256 in tripleo "ubuntu: hash sum mismatch when installing packages" [Undecided,New] | 09:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1366164 | 09:00 |
| uvirtbot | Launchpad bug 1374259 in os-refresh-config "Coverage report was't created" [Undecided,In progress] | 09:00 |
| uvirtbot | Launchpad bug 1366164 in tripleo "Add domain roles for admin user" [Undecided,New] | 09:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1367742 | 09:00 |
| uvirtbot | Launchpad bug 1367742 in tripleo "keepalived brings the VIP up in multiple places on a controller reboot" [Undecided,New] | 09:00 |
| *** untriaged-bot has quit IRC | 09:00 | |
| *** pblaho has quit IRC | 09:04 | |
| *** lazy_prince has quit IRC | 09:06 | |
| *** lazy_prince has joined #tripleo | 09:06 | |
| *** Dafna has joined #tripleo | 09:08 | |
| openstackgerrit | Radomir Dopieralski proposed a change to openstack/tuskar-ui: Remove the selector from the top bar https://review.openstack.org/124339 | 09:12 |
| tchaypo | interesting that we got exactly 4 successes in this ci-overcloud run too | 09:13 |
| openstackgerrit | Radomir Dopieralski proposed a change to openstack/tuskar-ui: Remove the selector from the top bar https://review.openstack.org/124339 | 09:14 |
| tchaypo | after such a long run consistently getting 5... I'm now wondering if I imagined that, or if it was just chance, or if something changed | 09:14 |
| *** ramishra has joined #tripleo | 09:15 | |
| *** lsmola has joined #tripleo | 09:15 | |
| lazy_prince | exit | 09:18 |
| tchaypo | lazy_prince: ;) | 09:19 |
| shadower | pbelanyi_: ping | 09:20 |
| *** bvivek has quit IRC | 09:21 | |
| *** yamahata_ has quit IRC | 09:23 | |
| *** bvivek has joined #tripleo | 09:23 | |
| *** Yanyan has quit IRC | 09:24 | |
| *** gchamoul has quit IRC | 09:25 | |
| *** andreaf has joined #tripleo | 09:28 | |
| openstackgerrit | A change was merged to openstack/tuskar-ui: Updated from global requirements https://review.openstack.org/124262 | 09:28 |
| regebro | Uhm, OK. so we have static heat meatadata for the undercloud. And it's queryable via os-apply-config. | 09:28 |
| regebro | Is it queryable some other way? | 09:29 |
| *** gchamoul has joined #tripleo | 09:31 | |
| openstackgerrit | Derek Higgins proposed a change to openstack-infra/tripleo-ci: Revert "temprevert "Datepicker locale support"" https://review.openstack.org/124342 | 09:31 |
| regebro | Hm. I think we need to get the templates to then use it and stick it in a plan parameter. | 09:33 |
| tchaypo | okay, the good news is that on that run, the *only* error that seeme dto be hit is the nova scheduler race - so perhaps all the other errors were symptoms of keystone and the mysql tiny pool | 09:33 |
| tchaypo | lifeless: ^^^ | 09:33 |
| tchaypo | running again with max_retries=50, let's see how tis goes | 09:33 |
| openstackgerrit | Derek Higgins proposed a change to openstack-infra/tripleo-ci: Nothing to see here https://review.openstack.org/111011 | 09:37 |
| shadower | pbelanyi_: nevermind, just added a few comments to your patch. | 09:37 |
| openstackgerrit | Giulio Fidente proposed a change to openstack/tripleo-incubator: Improve documentation around cinder-volume-nfs usage https://review.openstack.org/124347 | 09:48 |
| openstackgerrit | Giulio Fidente proposed a change to openstack/tripleo-incubator: Improve documentation around usage of cinder-volume-nfs https://review.openstack.org/124347 | 09:59 |
| openstackgerrit | Tomas Sedovic proposed a change to openstack/tripleo-incubator: Pass resource registry file to Heat if it exists https://review.openstack.org/123761 | 10:01 |
| *** nosnos has quit IRC | 10:09 | |
| *** dkehn has quit IRC | 10:09 | |
| *** nosnos has joined #tripleo | 10:09 | |
| *** dkehn has joined #tripleo | 10:11 | |
| *** jprovazn has quit IRC | 10:14 | |
| openstackgerrit | Radomir Dopieralski proposed a change to openstack/tuskar-ui: Fix the live deployment icon https://review.openstack.org/124353 | 10:16 |
| openstackgerrit | Radomir Dopieralski proposed a change to openstack/tuskar-ui: Fix icon on live deployment overview https://review.openstack.org/124353 | 10:17 |
| openstackgerrit | Radomir Dopieralski proposed a change to openstack/tuskar-ui: Angularize the overview page https://review.openstack.org/124354 | 10:19 |
| *** athomas has quit IRC | 10:21 | |
| *** athomas has joined #tripleo | 10:30 | |
| *** loki184 has quit IRC | 10:46 | |
| *** pradeep1 has joined #tripleo | 10:55 | |
| *** pradeep has quit IRC | 10:55 | |
| *** ci-overcloud has joined #tripleo | 11:01 | |
| ci-overcloud | ************** ci-overcloud complete status=1 ************ | 11:01 |
| *** ci-overcloud has quit IRC | 11:01 | |
| *** pblaho has joined #tripleo | 11:06 | |
| *** pblaho has quit IRC | 11:07 | |
| *** pblaho has joined #tripleo | 11:07 | |
| *** jprovazn has joined #tripleo | 11:08 | |
| openstackgerrit | Lennart Regebro proposed a change to openstack/tuskar-ui: Hide the flavor selection if we don't do flavor matching https://review.openstack.org/124362 | 11:09 |
| *** jistr has quit IRC | 11:10 | |
| *** jistr has joined #tripleo | 11:14 | |
| *** lucasagomes is now known as lucas-hungry | 11:19 | |
| *** akrivoka has joined #tripleo | 11:22 | |
| openstackgerrit | A change was merged to openstack/os-refresh-config: Updated from global requirements https://review.openstack.org/120645 | 11:30 |
| *** bvivek has quit IRC | 11:33 | |
| *** bvivek has joined #tripleo | 11:33 | |
| *** akrivoka has quit IRC | 11:38 | |
| *** k4n0 has quit IRC | 11:46 | |
| openstackgerrit | A change was merged to openstack/python-tuskarclient: Updated from global requirements https://review.openstack.org/116415 | 11:49 |
| *** akrivoka has joined #tripleo | 11:49 | |
| openstackgerrit | Radomir Dopieralski proposed a change to openstack/tuskar-ui: Angularize the overview page https://review.openstack.org/124354 | 11:54 |
| *** akrivoka has quit IRC | 12:08 | |
| *** nikunj2512 has quit IRC | 12:09 | |
| *** vdreamarkitex has joined #tripleo | 12:17 | |
| *** radez_g0n3 is now known as radez | 12:17 | |
| *** slagle has quit IRC | 12:20 | |
| *** slagle has joined #tripleo | 12:20 | |
| openstackgerrit | A change was merged to openstack/tripleo-heat-templates: undercloud: use Nova compute Ironic driver https://review.openstack.org/122754 | 12:23 |
| *** nosnos has quit IRC | 12:24 | |
| *** nosnos has joined #tripleo | 12:24 | |
| regebro | slagle: I hope my last email clarified the issue? If not tell me. :-) | 12:24 |
| *** jang1 has joined #tripleo | 12:27 | |
| slagle | regebro: let me go read it :) | 12:29 |
| *** nosnos has quit IRC | 12:29 | |
| slagle | regebro: maybe we could write the value to the settings.py? | 12:32 |
| slagle | regebro: then you would have access to it from horizon no? | 12:32 |
| *** jdob has joined #tripleo | 12:33 | |
| regebro | slagle: That should work. It does feel like a horrid hack, but maybe that's OK. :-) | 12:33 |
| slagle | well, it is a setting ;) | 12:35 |
| slagle | you actually don't need a service to query via os-apply-config, but the problem we're going to run into is that you do have to be root | 12:36 |
| openstackgerrit | Nicholas Randon proposed a change to openstack/tripleo-image-elements: Fix RabbitMQ element clustering start and stop https://review.openstack.org/115524 | 12:37 |
| *** jml has quit IRC | 12:37 | |
| regebro | slagle: Root, and on the correct machine. | 12:39 |
| *** jml has joined #tripleo | 12:39 | |
| slagle | you're on the correct machine | 12:39 |
| slagle | lsmola: can you take a look at http://git.openstack.org/cgit/openstack/tripleo-image-elements/tree/elements/heat/os-apply-config/etc/heat/heat.conf and let me know if the configuration options to setup the trusts the way tuskar-ui needs are configurable? | 12:40 |
| regebro | slagle: When you set up devtest, horizon is not running on the undercloud. | 12:40 |
| slagle | lsmola: so that we can remove the admin password from settings.py | 12:40 |
| *** lazy_prince has quit IRC | 12:41 | |
| regebro | I don't know how common that will be in practice, but I don't think it's a good idea to break that. | 12:41 |
| slagle | regebro: eventually, it will be. i think we're solving for an actual deployment scenario, not a dev setup where you're running horizon locally | 12:41 |
| *** lucas-hungry is now known as lucasagomes | 12:41 | |
| slagle | for a dev setup, we can configure anything how we need to via a script/whatever | 12:42 |
| slagle | regebro: but i think the settings.py approach would work for a dev setup as well? | 12:42 |
| regebro | slagle: Yeah, we just have a default setting, and if it's the wrong one you can change it. | 12:43 |
| regebro | s/we/we can | 12:43 |
| regebro | slagle: OK, I'll look into adding this to os-apply-config | 12:44 |
| lsmola | slagle: doesn't look to be configurable | 12:45 |
| *** ramishra has quit IRC | 12:46 | |
| lsmola | slagle: but we should be able to set it through passtrough, right? | 12:46 |
| lsmola | slagle: might be worth to just wait till trusts is default in heat | 12:48 |
| slagle | regebro: it's already set in os-apply-config, we just need to query for the value and write it to the settings.py, which we can do in the tuskar-ui element directly | 12:49 |
| slagle | regebro: i can do that, if you'd like | 12:49 |
| slagle | lsmola: heat.conf does not appear to be passthrough enable. but, probably should be | 12:49 |
| lsmola | slagle: btw. the admin password is set in tuskar-ui element now | 12:49 |
| slagle | lsmola: right, and i want to get rid of that :) | 12:49 |
| regebro | slagle: Nah, I'll try, because I only understood half of what you said now, and I'd like to understand all of it. | 12:49 |
| lsmola | slagle: are yo utalking about that with regebro ? | 12:49 |
| slagle | lsmola: no, something else, but it reminded me of the password in there | 12:50 |
| lsmola | slagle: ok | 12:50 |
| *** rdopieralski has quit IRC | 12:51 | |
| *** ramishra has joined #tripleo | 12:56 | |
| openstackgerrit | James Slagle proposed a change to openstack/tripleo-image-elements: Passthrough enable heat.conf https://review.openstack.org/124393 | 12:56 |
| *** dprince has joined #tripleo | 12:56 | |
| *** rhallisey has joined #tripleo | 13:01 | |
| *** killer_prince has joined #tripleo | 13:01 | |
| *** killer_prince is now known as lazy_prince | 13:01 | |
| regebro | slagle: So when you say "tuskar-ui element" do you mean the _50_tuskar.py file, or inside the panel code, or? I'm not sure. :-) | 13:02 |
| slagle | regebro: it's in tripleo-image-elements repository. it's what's used to install tuskar-ui | 13:03 |
| regebro | slagle: Ah, forgot about that one. | 13:04 |
| slagle | regebro: there is a script there under os-refresh-config/post-configure.d. from that script, use the os-apply-config command to query for the value, then write it to the settings file | 13:04 |
| tchaypo | and now hp2 is back in a state where even with the scheduler retrying 50 times for each node it can never find a node with >0mb ram free so none of my nodes ever get built | 13:05 |
| slagle | lsmola: can you remind me of the config values in heat.conf? | 13:07 |
| lsmola | slagle: on hangout now | 13:08 |
| lsmola | slagle: deferred_auth_method=trusts | 13:09 |
| lsmola | slagle: trusts_delegated_roles=_member_ | 13:10 |
| slagle | thx | 13:10 |
| lsmola | slagle: but I tested it only with trusts_delegated_roles=admin | 13:10 |
| *** tzumainn has joined #tripleo | 13:11 | |
| *** dprince has quit IRC | 13:12 | |
| otoolee- | Is patch https://review.openstack.org/#/c/123999/ OK to merge? We need it to enable memcache on debian-based OSes, which in turn is used by the Swift proxy. | 13:13 |
| *** yamahata_ has joined #tripleo | 13:19 | |
| slagle | otoolee-: lgtm, approved it | 13:21 |
| otoolee- | slagle, Thanks. | 13:22 |
| radez | sdake: morning, I have the keystone container up and running but it looks like mariadb isn't running. the start command seems to reference some variables that look like they are supposted to be passed it | 13:22 |
| radez | *passed in, is there something that's supposed to be defined in the json passed to kubernetes? | 13:23 |
| *** Kiall has quit IRC | 13:24 | |
| openstackgerrit | A change was merged to openstack/tripleo-image-elements: Fix memcache listen port setting on debian-based OSes https://review.openstack.org/123999 | 13:26 |
| radez | sdake: nm I just found https://github.com/jlabocki/superhappyfunshow/blob/master/docker/keystone/keystone.json... I was writing my own :) | 13:27 |
| openstackgerrit | A change was merged to openstack/tripleo-image-elements: Add swift.proxy-memcache metadata to proxy-server.conf https://review.openstack.org/86312 | 13:27 |
| *** noslzzp has joined #tripleo | 13:29 | |
| *** noslzzp has quit IRC | 13:29 | |
| *** dprince has joined #tripleo | 13:39 | |
| *** mestery has quit IRC | 13:40 | |
| *** noslzzp has joined #tripleo | 13:40 | |
| sdake | radez yes you need to kubecfg -c service.json start service iirc | 13:42 |
| *** rushiagr is now known as rushiagr_away | 13:43 | |
| sdake | radez also check out my pending changeset | 13:43 |
| sdake | I registered a login for our images and uploaded a bunch there | 13:43 |
| sdake | kollaglue/fedora-rdo-mariadb for eg | 13:43 |
| radez | sdake: will do. still debugging mysql :) | 13:44 |
| sdake | you need to pod and service start those two | 13:44 |
| sdake | with the correct image | 13:44 |
| sdake | the image that is in the json is incorrect | 13:44 |
| openstackgerrit | James Slagle proposed a change to openstack/tripleo-image-elements: Add network-gateway element https://review.openstack.org/124399 | 13:44 |
| radez | sdake: yea I updated the json to point to the image that I built | 13:46 |
| radez | I pushed it into a private repo on my kube master | 13:46 |
| radez | sdake: is pod start different then pod create? | 13:47 |
| radez | brb | 13:47 |
| sdake | radez not sure | 13:49 |
| openstackgerrit | Tzu-Mainn Chen proposed a change to openstack/tuskar-ui: Adding plan parameter view on service configuration tab https://review.openstack.org/123529 | 13:49 |
| *** larsks has joined #tripleo | 13:49 | |
| radez | k... saga continues :) | 13:49 |
| larsks | radez: sdake said you had a question... | 13:49 |
| radez | larsks: yea, just got a walk up... gimme a sec | 13:50 |
| sdake | <radez> sdake: is pod start different then pod create? | 13:51 |
| radez | oh yea larsks ^ | 13:51 |
| larsks | radez: Well...sort of, I guess. "create" submits the pod to the scheduler, which will attempt to schedule it on a minion, at which point it will start. So "create" puts the pod in the queue in the "waiting" state, and it will start when it is successfully scheduled. | 13:52 |
| larsks | (and transitions from waiting -> running) | 13:52 |
| radez | larsks: right, but the /start.sh script will be executed in that process right? | 13:53 |
| radez | mysql wasn't starting in my container, just making sure I have my head wrapped around this to debug it properly | 13:53 |
| larsks | That's no longer a kubernetes question :). When a Docker container starts, it will do whatever it's ENTRYPOINT and CMD entries tell it to do. | 13:53 |
| sdake | I think I have the same problem | 13:53 |
| larsks | You can always test containers by hand by using "docker run" and providing the necessary environment variables with '-e var=value' command line options. I was giving sdake a few suggestions for debugging containers earlier. I should just write those up, I guess. | 13:54 |
| radez | larsks: ack, I think kube is doing it's thing properrly and it's a matter of figureing out the start.sh and container | 13:54 |
| radez | larsks: that's a good suggestion... | 13:54 |
| larsks | Let me put together a quick email. | 13:55 |
| sdake | mariadb has an entrypo8int.sh not a start.sh | 13:55 |
| sdake | larsks make suer to mail openstack-dev plz | 13:55 |
| larsks | Sure. | 13:55 |
| sdake | with [kolla] in the tags | 13:55 |
| *** pradeep1 has quit IRC | 13:57 | |
| radez | sdake: oh, ha... I didn't see that there's a mariadb container too | 14:04 |
| sdake | larsks running those containers manaully gets something that boots | 14:04 |
| sdake | by running manually, I mean "docker run" | 14:05 |
| larsks | sdake: Sure. I think the answers are going to be found by inspecting the containers that didn't seem to start correctly. | 14:05 |
| radez | sdake so you're running the maria db and the keystone containers? | 14:06 |
| sdake | my containers start in kub correctly they just dont provide keystone service | 14:06 |
| radez | er pods? | 14:06 |
| sdake | radez yup | 14:06 |
| sdake | pods and services | 14:06 |
| larsks | sdake: That sounds like "not starting correctly" to me :) | 14:06 |
| radez | that's what I'm missing i think... what do you mean pods and services? | 14:06 |
| sdake | maybe my test case is bust | 14:06 |
| sdake | there is stuff with -5000.service | 14:07 |
| sdake | run that with service start | 14:07 |
| larsks | radez: a "pod" is a group of linked containers (shared network namespace) that you create with "kubecfg create pods". | 14:07 |
| sdake | kubecfg -c file-500.json service start | 14:07 |
| larsks | radez: A "service" is a service discovery mechanism that sets up proxies and environment variables in containers so that your pods can discover the addresses to use for a given service. | 14:07 |
| radez | larsks: right, so do I do that once for keystone and once for maria db json files or do I need to tie them toether osmehow? | 14:07 |
| radez | so the services are somethign that happends as a result of pods create? | 14:08 |
| larsks | THe keystone pod discovers the mariadb endpoint using the environment variables provided due to the mariadb service. | 14:08 |
| larsks | No, the services are created with "kubecfg create services". | 14:08 |
| larsks | I highly recommend walking through the kube guestbook example: https://github.com/GoogleCloudPlatform/kubernetes/tree/master/examples/guestbook | 14:08 |
| larsks | ...before mucking about with anything else. | 14:09 |
| sdake | keystone-service-5000.json | 14:09 |
| sdake | see that file | 14:09 |
| sdake | that is the service you need to start | 14:09 |
| sdake | kubecfg -c keystone-service-5000.json service start | 14:09 |
| radez | larsks: I've looked though that... I guess Ineed to actually execute it | 14:09 |
| sdake | service start gets the docker containers rolling together | 14:09 |
| larsks | sdake: I know that there were some changes to the dockerfiles since the hackathon, and I have not tested any of the images since then. So it's entirely possible something is broken. | 14:10 |
| *** yamahata_ has quit IRC | 14:10 | |
| sdake | my keystone public address seems to work - eg:[sdake@minime ~(keystone_admin)]$ telnet 172.24.4.234 5000 | 14:12 |
| sdake | Trying 172.24.4.234... | 14:12 |
| sdake | Connected to 172.24.4.234. | 14:12 |
| sdake | Escape character is '^]'. | 14:12 |
| sdake | VERSION | 14:12 |
| sdake | <head> | 14:12 |
| sdake | <title>Error response</title> | 14:12 |
| larsks | I would use 'curl' for testing an http service myself. | 14:12 |
| larsks | But yeah, at least something is responding. | 14:12 |
| sdake | [sdake@minime ~(keystone_admin)]$ keystone endpoint-list | 14:14 |
| sdake | Authorization Failed: An unexpected error prevented the server from fulfilling your request. (HTTP 500) | 14:14 |
| larsks | Logs! | 14:14 |
| larsks | Environment variables! | 14:14 |
| sdake | how do we get the logs | 14:15 |
| larsks | In theory, 'docker logs' should have the stdout/stderr of the keystone process. Also, by nsenter-ing the container, setting environment variables per /proc/pid/1/environ, and running the start.sh script manually... | 14:16 |
| *** andreaf has quit IRC | 14:16 | |
| *** andreaf has joined #tripleo | 14:17 | |
| sdake | what is the container called, keystone-master? | 14:17 |
| larsks | No. You would need to run 'docker ps' to get the name (or id) of the container. | 14:17 |
| larsks | The container name is generated dynamically. | 14:17 |
| shadower | hey sdake | 14:18 |
| sdake | hey shadower | 14:18 |
| *** yamahata_ has joined #tripleo | 14:18 | |
| sdake | larsks that doesn't generate the keystone service logs | 14:18 |
| sdake | I'll have to chang ethe container config I think to log to stdout rather then file | 14:19 |
| larsks | sdake: So go for plan b. | 14:19 |
| larsks | sdake: Or just nsenter and look for the logs in /var/log. | 14:19 |
| *** vgnbkr has joined #tripleo | 14:20 | |
| vgnbkr | Good Morning. I'm working on Trove and have some questions about diskimage-builder. Is this the place? | 14:21 |
| *** jehb has joined #tripleo | 14:22 | |
| openstackgerrit | Peter Belanyi proposed a change to openstack/tripleo-heat-templates: Add converted version of block and object storage https://review.openstack.org/123713 | 14:22 |
| *** regebro has quit IRC | 14:22 | |
| radez | sdake: you can attach to the container on the node | 14:25 |
| *** dprince has quit IRC | 14:25 | |
| radez | sdake: use this script https://github.com/twiest/docker-vagrant-libvirt/blob/master/enter-container.sh | 14:25 |
| sdake | cool I'll give that a try | 14:25 |
| *** dprince has joined #tripleo | 14:25 | |
| radez | I jum on the node | 14:25 |
| radez | docker ps to get the id | 14:25 |
| radez | then pass that id ot the the script | 14:26 |
| radez | dropps you right in. | 14:26 |
| radez | I'm looking at the keystone logs on there | 14:26 |
| radez | sdake: i think we're at the smae place, but I've seen the logs <grin> | 14:26 |
| *** andreaf has quit IRC | 14:26 | |
| sdake | yes but I has slides!! | 14:26 |
| openstackgerrit | Ryan Brady proposed a change to openstack/diskimage-builder: Enable RHEL Registration https://review.openstack.org/123145 | 14:26 |
| *** andreaf has joined #tripleo | 14:27 | |
| radez | hehe | 14:28 |
| radez | sdake: compading the start script to the keystone config it looks like mine isn't getting the service information from mariadb | 14:28 |
| larsks | radez: Look at the environment variables assigned to pid 1. | 14:28 |
| larsks | See the email I just sent. | 14:29 |
| * radez checks it out | 14:29 | |
| *** ramishra has quit IRC | 14:31 | |
| jpeeler | hey kube folks, so what command is being used to register keystone with kubernetes? | 14:31 |
| sdake | jpeeler kubecfg | 14:32 |
| jpeeler | I haven't looked at the guestbook example yet, but if I'm only a few commands behind y'all then... | 14:32 |
| sdake | jpeeler on t he "master" node create4d with the kube template | 14:32 |
| larsks | kubecfg -c keystone.json create pods | 14:32 |
| larsks | jpeeler: ^^^ | 14:32 |
| sdake | you need to create mariadb first | 14:32 |
| sdake | after you create pods, you need to create services | 14:32 |
| sdake | or start them, not sure which ;-) | 14:32 |
| larsks | sdake: note that you can create services before pods. It's possible that the deployment will be "create all the services first" before starting any pods. Maybe. | 14:33 |
| radez | sdake: I think my env vars arn't matching up with the vars passed in becasue I tweaked my json files to create the pods and services | 14:34 |
| radez | so looks like the json files can impact what the env names are | 14:34 |
| larsks | radez: the environmetn are set based on the service names you provide. so if you change the service names, then yes, different env. var names. | 14:35 |
| radez | ah that ties it together... thx | 14:35 |
| jpeeler | perhaps silly question, but does the image line need updating in the json? | 14:36 |
| larsks | jpeeler: if it does not point at a valid image, then sure. | 14:36 |
| openstackgerrit | Ryan Brady proposed a change to openstack/diskimage-builder: Enable RHEL Registration https://review.openstack.org/123145 | 14:37 |
| openstackgerrit | Ryan Brady proposed a change to openstack/diskimage-builder: Enable RHEL Registration https://review.openstack.org/123145 | 14:39 |
| *** CaptTofu___ is now known as CaptTofu | 14:40 | |
| *** ramishra has joined #tripleo | 14:41 | |
| openstackgerrit | Jon-Paul Sullivan (jp_at_hp) proposed a change to openstack/tripleo-image-elements: CORE-1775 Ensure tempest writes a full manifest https://review.openstack.org/124425 | 14:46 |
| openstackgerrit | Tomas Sedovic proposed a change to openstack/tripleo-heat-templates: Compute and controller templates without merge.py https://review.openstack.org/123100 | 14:47 |
| openstackgerrit | Jon-Paul Sullivan (jp_at_hp) proposed a change to openstack/tripleo-image-elements: Ensure tempest writes a full manifest https://review.openstack.org/124425 | 14:47 |
| openstackgerrit | Tomas Sedovic proposed a change to openstack/tripleo-incubator: Pass resource registry file to Heat if it exists https://review.openstack.org/123761 | 14:52 |
| radez | sdake: I've got my env vars fixed up now... and I get this in my keystone log | 14:52 |
| radez | OperationalError: (OperationalError) (2013, "Lost connection to MySQL server at 'reading initial communication packet', system error: 0") | 14:52 |
| radez | which looks like it's because my mariadb instance is running on my 1st node and the service is connecting to the second node | 14:53 |
| *** mestery has joined #tripleo | 14:55 | |
| radez | larsks: if mariadb is running on the 1st node and I connect to 3306 on the second node should the proxies handle that properly>? | 14:55 |
| radez | that doesn't seem to be working... so either the proxies to take care of that or the service returned the wrogn ip? | 14:56 |
| larsks | radez: Yes. On each minion there should be a kube-proxy process running on the host that is bound to that port. | 14:56 |
| larsks | 'netstat -tln' on the host should answer the 'is kube-proxy listening' question. | 14:57 |
| radez | larsks: are there logs somewhere that I could figure out why proxy on minon2 doesn't send traffic to minion1 | 14:57 |
| radez | I see the proxy listening on the port | 14:57 |
| larsks | radez: Hmm, not sure. Take a look at what kube-proxy is logging in the journal? I am currently building and booting things locally, at which point I will see if I can reproduce what you're seeing... | 14:59 |
| *** untriaged-bot has joined #tripleo | 15:01 | |
| untriaged-bot | Untriaged bugs so far: | 15:01 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1373063 | 15:01 |
| uvirtbot | Launchpad bug 1373063 in tripleo "port not added to bridge after a reboot" [Undecided,In progress] | 15:01 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1369256 | 15:01 |
| uvirtbot | Launchpad bug 1369256 in tripleo "ubuntu: hash sum mismatch when installing packages" [Undecided,New] | 15:01 |
| untriaged-bot | https://bugs.launchpad.net/os-refresh-config/+bug/1374259 | 15:01 |
| uvirtbot | Launchpad bug 1374259 in os-refresh-config "Coverage report was't created" [Undecided,In progress] | 15:01 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1366164 | 15:01 |
| uvirtbot | Launchpad bug 1366164 in tripleo "Add domain roles for admin user" [Undecided,New] | 15:01 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1367742 | 15:01 |
| *** untriaged-bot has quit IRC | 15:01 | |
| uvirtbot | Launchpad bug 1367742 in tripleo "keepalived brings the VIP up in multiple places on a controller reboot" [Undecided,New] | 15:01 |
| *** marun has joined #tripleo | 15:01 | |
| radez | found the kube-proxy logs. | 15:05 |
| radez | Dial failed: dial tcp 10.251.3.10:3306: no route to host | 15:05 |
| radez | looks like the connection from one of the proxies to maria is busted | 15:05 |
| radez | gotta run out for a bit, ttyl | 15:05 |
| *** radez is now known as radez_g0n3 | 15:05 | |
| *** pcrews has joined #tripleo | 15:07 | |
| *** chuckC has joined #tripleo | 15:11 | |
| *** ChuckC_ has joined #tripleo | 15:12 | |
| *** jogo is now known as flashgordon | 15:17 | |
| *** pradeep1 has joined #tripleo | 15:17 | |
| slagle | lsmola: this is the change we need merged to set those configs: https://review.openstack.org/#/c/124393/ | 15:17 |
| slagle | lsmola: if you're in a +2 mood :) | 15:18 |
| sdake | radez jpeeler create an account on docker.io plz | 15:21 |
| openstackgerrit | A change was merged to openstack/tripleo-image-elements: Passthrough enable heat.conf https://review.openstack.org/124393 | 15:22 |
| *** anteaya has quit IRC | 15:23 | |
| *** pradeep1 has quit IRC | 15:25 | |
| *** jprovazn has quit IRC | 15:25 | |
| *** mrunge has quit IRC | 15:32 | |
| openstackgerrit | Peter Belanyi proposed a change to openstack/tripleo-heat-templates: Add converted version of block and object storage https://review.openstack.org/123713 | 15:36 |
| jpeeler | sdake: which host is the docker images supposed to be on? | 15:39 |
| jpeeler | my list pods output just reports "waiting" | 15:39 |
| sdake | jpeeler pull my happyfun repo | 15:39 |
| sdake | not jlebocki's | 15:39 |
| sdake | I have changes which use the globally registered dockerinc images | 15:40 |
| jpeeler | hrm, well which host would those images be downloaded on? it doesn't look like docker is running on the kube master | 15:41 |
| *** andreaf has quit IRC | 15:41 | |
| *** andreaf has joined #tripleo | 15:42 | |
| larsks | jpeeler: docker only runs on the minions. | 15:54 |
| larsks | jpeeler: the master schedules a pod on a minion, then the minion pulls and starts the image. | 15:54 |
| jpeeler | it would be neat if you could chain docker instances together, it looks like i just need to move the images to the minions then | 15:56 |
| jpeeler | if that doesn't work steve, i'll go the global route | 15:56 |
| *** bnemec is now known as beekneemech | 16:00 | |
| openstackgerrit | Ryan Brady proposed a change to openstack/diskimage-builder: Add svc-map element. https://review.openstack.org/96770 | 16:00 |
| larsks | jpeeler: what does "chain docker instances together" mean? | 16:03 |
| jpeeler | larsks: in my mind, to pull a docker image from another docker installation | 16:03 |
| larsks | Ah. Yeah, docker only knows how to pull from registries. You can run a registry locally and manually feed it images from docker hub, I guess -- that would get you faster start times on your minions for new images. | 16:05 |
| larsks | Note that kubernetes will *always* perform a 'pull' before starting a pod, even if the image exists locally. After building an image locally always make sure to 'docker push'. | 16:05 |
| sdake | https://review.openstack.org/124453 | 16:08 |
| sdake | once that goes through we should be g2g | 16:08 |
| *** mestery has quit IRC | 16:12 | |
| *** chuckC has quit IRC | 16:18 | |
| *** cody-somerville has joined #tripleo | 16:21 | |
| *** ChuckC has joined #tripleo | 16:24 | |
| *** ChuckC_ has quit IRC | 16:26 | |
| *** ChuckC has quit IRC | 16:30 | |
| *** anteaya has joined #tripleo | 16:34 | |
| openstackgerrit | A change was merged to openstack-infra/tripleo-ci: Update virsh ssh commands to support setting the locale https://review.openstack.org/124072 | 16:37 |
| derekh | lucasagomes: ^ I'll deploy the change on monday morning, first thing | 16:38 |
| lucasagomes | derekh, w00t! alright, thanks! | 16:39 |
| *** ramishra has quit IRC | 16:42 | |
| *** jistr is now known as jistr|afk | 16:45 | |
| *** mestery has joined #tripleo | 16:51 | |
| *** mestery has quit IRC | 16:51 | |
| *** mestery has joined #tripleo | 16:52 | |
| *** Dafna has quit IRC | 16:53 | |
| *** andreaf has quit IRC | 16:55 | |
| *** andreaf has joined #tripleo | 16:56 | |
| *** yamahata_ has quit IRC | 16:57 | |
| *** lucasagomes is now known as lucas-dinner | 17:08 | |
| *** lazy_prince has quit IRC | 17:08 | |
| *** saurabhs has joined #tripleo | 17:12 | |
| *** derekh has quit IRC | 17:14 | |
| *** ramishra has joined #tripleo | 17:14 | |
| *** athomas has quit IRC | 17:16 | |
| *** ChuckC has joined #tripleo | 17:24 | |
| *** chuckC_ has joined #tripleo | 17:26 | |
| *** ChuckC has quit IRC | 17:29 | |
| *** chuckC_ is now known as chuckC | 17:29 | |
| *** ChuckC_ has joined #tripleo | 17:30 | |
| *** ramishra has quit IRC | 17:34 | |
| *** pradeep1 has joined #tripleo | 17:36 | |
| *** pensu has quit IRC | 17:43 | |
| *** pradeep1 has quit IRC | 17:55 | |
| *** pensu has joined #tripleo | 17:59 | |
| *** sballe_ has joined #tripleo | 17:59 | |
| *** dprince has quit IRC | 18:01 | |
| *** bvivek has quit IRC | 18:06 | |
| *** morgan_remote_ has joined #tripleo | 18:07 | |
| *** killer_prince has joined #tripleo | 18:09 | |
| *** killer_prince is now known as lazy_prince | 18:09 | |
| *** jcoufal has quit IRC | 18:13 | |
| *** dprince has joined #tripleo | 18:13 | |
| *** jp_at_hp has quit IRC | 18:13 | |
| *** radez_g0n3 is now known as radez | 18:24 | |
| radez | jpeeler: I've also launched the containters in a private docker repo that run on my master | 18:25 |
| *** spzala has joined #tripleo | 18:26 | |
| radez | sdake: have you gotten the keystone containers working? I'm about to investigate further but I think there's something not right with the overlay network. the proxy on 1 node can connect to mysql on the other node | 18:26 |
| jpeeler | radez: that confuses me, i was just told that docker runs only on the minions | 18:27 |
| radez | jpeeler: I added a private-dcoer registry to the giddyup | 18:28 |
| radez | your inforamation is correct | 18:28 |
| radez | but another option is to build the containers and push them to a private repo, I was able to do this successfully | 18:28 |
| jpeeler | i clearly am not quite understanding something here | 18:29 |
| radez | it just so happened I put the docker repo on my master but it could have gone anywhere | 18:29 |
| radez | you can just ignore me... it's not vital to the setup | 18:29 |
| EmilienM | dprince: I wonder one thing stupid, but anyway. After having tripleo-puppet-elements on github/openstack, would it make sense to move puppet modules from github/stackforge to github/openstack ? | 18:29 |
| jpeeler | radez: i'm just trying to learn, my attempt at putting the images on the minions is not working | 18:29 |
| radez | jpeeler: have you looked at the logs on the minions? | 18:30 |
| radez | they will tell you where they are trying to pull from and if they're able to | 18:30 |
| radez | the commands on this post have been helpful to me: | 18:31 |
| radez | https://github.com/GoogleCloudPlatform/kubernetes/blob/master/docs/getting-started-guides/fedora/fedora_manual_config.md | 18:31 |
| jpeeler | radez: yes i see it failing: Error: image fedora-rdo-mariadb not found | 18:31 |
| radez | k, so if you're addressing the image by just "fedora-rdo-mariadb" in your json then it's looking in the public repos | 18:31 |
| radez | and i don't think these are in the public repos at all | 18:32 |
| jpeeler | i thought it would by default look in the private (local) image location | 18:32 |
| dprince | EmilienM: I kind of thought that was the point of stackforge was for community projects. | 18:32 |
| larsks | jpeeler: kubernetes always does a pull before starting a pod. If the pull fails, I think --> you lose. | 18:33 |
| radez | jpeeler: nope I think it will alawys try to pull according to what larsks said walier | 18:33 |
| openstackgerrit | Tzu-Mainn Chen proposed a change to openstack/tuskar-ui: Fix for node count form https://review.openstack.org/124486 | 18:33 |
| jpeeler | larsks: so does that mean public images have to be used? | 18:33 |
| radez | jpeeler: want me to walk you through putting a private docker repo on your masternode? | 18:33 |
| dprince | EmilienM: and on that front I sort of wonder if tripleo-puppet-elements actually belong on stackforge too since they are arguably more closely aligned w/ the puppet modules. | 18:33 |
| larsks | jpeeler: No, you can use a private repository. | 18:33 |
| larsks | s/repository/registry/ | 18:33 |
| larsks | But it has to be something that supports docker push/pull semantics. | 18:33 |
| radez | oh yea... me too s/repository/registry/g | 18:34 |
| dprince | EmilienM: I think you'd have a "tough sell" to argue all the stackforge/puppet modules belong under /openstack | 18:34 |
| radez | jpeeler: actually you could just use the one I setup, it's publically accessible. | 18:34 |
| jpeeler | radez: sure, unless using the kllaglue/fedora-rdo-* images makes it not really worth it | 18:34 |
| radez | maybe? I'm not sure | 18:34 |
| larsks | Our goal is ultimately to have the kolla project images built automatically in response to commits against the git repository. | 18:35 |
| larsks | So that ultimately they should be current and useful in most cases. | 18:35 |
| larsks | absolutely not the case right now :) | 18:35 |
| radez | jpeeler: try updating your image to "8.21.28.147:5000/fedora-rdo-mariadb" or to "kllaglue/fedora-rdo-mariadb" and see if it will launch | 18:35 |
| openstackgerrit | A change was merged to openstack/diskimage-builder: Add svc-map element. https://review.openstack.org/96770 | 18:36 |
| *** andreaf has quit IRC | 18:36 | |
| jpeeler | radez: do i need to delete all the pods/services or will it recognize the updated json as an update? | 18:36 |
| *** andreaf has joined #tripleo | 18:37 | |
| radez | jpeeler: delete and recreate afaik | 18:37 |
| larsks | jpeeler: I suspect you need to delete the existing pods first. You only need to delete/recreate the services if you have made changes to the the service json description. | 18:37 |
| *** jistr|afk is now known as jistr | 18:42 | |
| *** jistr has quit IRC | 18:42 | |
| radez | larsks: do you think there would be a problem with haveing a vxlan tunnel between the two minion nodes when the openstack inscances running the master and minions are running on an openstack cluster using gre tunnels? | 18:46 |
| larsks | radez: I don't think so. I don't even think the vxlan tunnels between minions are particularly relevant (because typical pod-to-service communication will be happening via the *host* addresses, via kube-proxy) | 18:47 |
| radez | larsks: hm, ok. my proxy on the minion that's not running the pod can't connect to the pod | 18:48 |
| larsks | radez: I would start by breaking out tcpdump and see to which address it is actually trying to connect. | 18:48 |
| larsks | Presumably, on the minion actually running the pod, you can contact the service on the host? | 18:49 |
| morgan_remote_ | lifeless tchaypo: index merged. If you use latest master of keystone I *think* the weird identity related failures should go away. | 18:50 |
| radez | larsks: yes I have a kbr0 on each minion. one is 10.251.3.0/24 and the other is 10.251.4.0/24 | 18:52 |
| radez | the mysql instance is running on 10.251.4.4 | 18:52 |
| radez | on the minon with the .3. address the kube-proxy logs show | 18:52 |
| radez | 10.251.4.4:3306: no route to host | 18:52 |
| radez | should the kbr0 interfaces all have the same subnet? or are they supposed to have different ones? | 18:53 |
| larsks | Hmmm. Are these systems to which I can connect? | 18:53 |
| radez | larsks: yup, gimme a pub key and I'll drop it in for you | 18:53 |
| larsks | http://chunk.io/f/71bc331491a847bf9ecd8b1800330910 | 18:53 |
| radez | larsks: I just have mariadb running now because that's what keystone was having trouble connecting to. | 18:53 |
| radez | sec | 18:53 |
| *** marun has quit IRC | 18:55 | |
| radez | larsks: fedora@8.21.28.147 <-master, fedora@8.21.28.148 <- node1, fedora@8.21.28.151 <- node2 | 18:56 |
| larsks | omg, someone who provides user@hostnames that I can just paste into my terminal. | 18:57 |
| larsks | A breath of fresh air. | 18:57 |
| larsks | logged in and looking around... | 18:58 |
| radez | ack | 18:58 |
| radez | larsks: so if you try and telnet to 8.21.28.148 on 3306 and then to 8.21.28.151 on one you will connect successfully and on the other you will get diconnected | 18:59 |
| radez | in the proxy logs one the one that disconnects the error occurs that it can't conntact the other's ?provider network? | 18:59 |
| radez | er overlay network... that's what I meant... | 19:00 |
| larsks | Oi, my connection is crappy. | 19:02 |
| radez | larsks: yea I had a good connection while I was in the office, when i got home it was really slow | 19:02 |
| larsks | radez: brb | 19:06 |
| radez | *nod* | 19:07 |
| *** rushiagr_away is now known as rushiagr | 19:11 | |
| * larsks is puzzled. There is no evidence that the system is trying to establish a vxlan connection to the other minion. Still poking. | 19:15 | |
| larsks | Oh, huh: /var/log/openvswitch/ovs-vswitchd.log: system@ovs-system: failed to add vxlan-0-4 as port: Address family not supported by protocol | 19:16 |
| larsks | Why is this kernel so old? | 19:16 |
| larsks | The minions still have 3.11.xx, but F20 is up to 3.16. | 19:16 |
| larsks | Huh. Please hold. | 19:18 |
| larsks | radez: ^^^ That was it. You were starting with an older F20 image than I have locally. | 19:19 |
| larsks | The original F20 kernel does not have vxlan support. | 19:19 |
| larsks | I have rebooted the systems. I expect them to work correctly when they reboot. | 19:19 |
| radez | larsks: ah ha! | 19:20 |
| larsks | So for example, on 10.0.03 this now works: ping 10.251.4.1 | 19:20 |
| radez | larsks: I was wondering why I lost connectivity to the node :) | 19:20 |
| *** andreaf has quit IRC | 19:21 | |
| larsks | Now, I'm curious if kubernetes will reschedule the pods all by itself. | 19:21 |
| larsks | Ooo, signs point to yes. | 19:21 |
| larsks | Totally works now. | 19:22 |
| *** andreaf has joined #tripleo | 19:22 | |
| radez | larsks: did you have to update the kernel or did they just need a reboot | 19:22 |
| radez | because doesn't the cloudinit script do a yum update> | 19:22 |
| radez | ? | 19:22 |
| larsks | They just needed a reboot. THe user-data script does a yum upgrade at first boot. | 19:23 |
| *** jp_at_hp has joined #tripleo | 19:23 | |
| radez | ack. thx for the debug larsks | 19:23 |
| larsks | Probably worth noting in the README. | 19:23 |
| openstackgerrit | A change was merged to openstack/diskimage-builder: Refactor ramdisk element to allow alternate implementations https://review.openstack.org/124235 | 19:23 |
| * radez fires off the keystone pods | 19:23 | |
| *** boris-42 has joined #tripleo | 19:27 | |
| larsks | well, you have a running keystone service that responds to requests on '/'... | 19:27 |
| *** rushiagr is now known as rushiagr_away | 19:29 | |
| openstackgerrit | patrick-crews proposed a change to openstack-infra/tripleo-ci: Test the upgrade codepath works as well. https://review.openstack.org/87758 | 19:30 |
| *** rook has joined #tripleo | 19:38 | |
| *** jtomasek has quit IRC | 19:38 | |
| radez | larsks: yup, and after setting up the pki cert I can do a token get | 19:39 |
| radez | sdake: ^ | 19:39 |
| larsks | Yay! | 19:40 |
| *** andreaf has quit IRC | 19:44 | |
| *** andreaf has joined #tripleo | 19:45 | |
| sdake | cool razez | 19:56 |
| radez | sdake: did you get it running to? | 19:56 |
| jpeeler | is it safe to say if 35357 is running that all is well? how are you testing keystone? | 20:05 |
| *** julim has quit IRC | 20:05 | |
| slagle | morganfainberg: hi, you know keystone right? ;) | 20:05 |
| morganfainberg | slagle, sometimes :) | 20:05 |
| slagle | morganfainberg: hehe, well i can go ask in #keystone, but figured since you were here, i'd start here first | 20:06 |
| morganfainberg | haha | 20:06 |
| morganfainberg | it's why i lurk around the openstack channels | 20:06 |
| slagle | morganfainberg: any idea why there's hardcoded id as the default value for member_role_id in common/config.py? | 20:06 |
| slagle | this is going to cause tripleo issues since we were explicitally creating the _member_ role, so it ends up with a auto generated id | 20:07 |
| slagle | then we create the first user (admin), keystone checks for the default id, which does not exist, so it tries to create the _member_ role again, and fails with a duplicate entry | 20:08 |
| slagle | this is a recent change in tripleo (os-cloud-config specifically) to precreate the _member_ role | 20:08 |
| slagle | i'm wondering if we should revert that | 20:08 |
| morganfainberg | slagle, we should be auto populating that role *if* needed | 20:09 |
| morganfainberg | slagle, iirc | 20:09 |
| slagle | https://review.openstack.org/#/c/117017/ | 20:09 |
| morganfainberg | but.. that is the first i've heard about a conflict going on | 20:09 |
| slagle | morganfainberg: right, so it does auto populate, it just queries by the hardcoded id vs. the name | 20:09 |
| morganfainberg | ah, right ok so it's because ID is supposed to be immutable | 20:10 |
| morganfainberg | names *in theory* should be mutable | 20:10 |
| morganfainberg | honestly though roles, not sure if that is an accurate statement without checking | 20:10 |
| slagle | ok, so basically, don't ever create the _member_ role yourself | 20:10 |
| slagle | let keystone do it | 20:10 |
| morganfainberg | slagle, well you *can* create it, just specify the role_id in the config if you do ;) | 20:11 |
| slagle | hmm, yea, ok. that makes sense | 20:11 |
| morganfainberg | slagle, i'm not a fan of the auto-create stuff like that, but that pre-dates my days in keystone | 20:11 |
| morganfainberg | slagle, it *might* be work setting an id like _member_ (not sure if that works) that mirrors the name and then you don't need to worry about keystone creating something you don't create | 20:13 |
| morganfainberg | s/work/worth | 20:13 |
| morganfainberg | though... that may not be possible with the autogen ids. | 20:13 |
| slagle | then i would also have to set that value in keystone.conf though | 20:14 |
| morganfainberg | slagle, but it would be static in that case. | 20:14 |
| slagle | right | 20:14 |
| morganfainberg | slagle, known on both sides. if you can specify the id. i'm not sure you can without looking | 20:14 |
| slagle | i think i'm just going to go the route of letting keystone create the role | 20:16 |
| slagle | i don't know why we need to do it in os-cloud-config tbh | 20:17 |
| lifeless | tchaypo: see morgan_remote_'s comment at your 0450 | 20:19 |
| openstackgerrit | James Slagle proposed a change to openstack/os-cloud-config: Do not create the _member_ role https://review.openstack.org/124508 | 20:23 |
| openstackgerrit | James Slagle proposed a change to openstack/os-cloud-config: Do not create the _member_ role https://review.openstack.org/124508 | 20:24 |
| *** dprince has quit IRC | 20:26 | |
| *** jehb has quit IRC | 20:29 | |
| morganfainberg | slagle, if it works without you creating it, i like less logic that needs to be maintained | 20:29 |
| *** rwsu has quit IRC | 20:31 | |
| slagle | me too | 20:32 |
| *** jtomasek has joined #tripleo | 20:33 | |
| *** morganfainberg is now known as morgan | 20:36 | |
| *** morgan is now known as morganfainberg | 20:37 | |
| *** jml has quit IRC | 20:44 | |
| *** jml has joined #tripleo | 20:45 | |
| *** radez is now known as radez_g0n3 | 20:50 | |
| *** jdob has quit IRC | 20:51 | |
| *** jtomasek has quit IRC | 20:53 | |
| *** untriaged-bot has joined #tripleo | 21:00 | |
| untriaged-bot | Untriaged bugs so far: | 21:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1373063 | 21:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1369256 | 21:00 |
| uvirtbot | Launchpad bug 1373063 in tripleo "port not added to bridge after a reboot" [Undecided,In progress] | 21:00 |
| untriaged-bot | https://bugs.launchpad.net/os-refresh-config/+bug/1374259 | 21:00 |
| uvirtbot | Launchpad bug 1369256 in tripleo "ubuntu: hash sum mismatch when installing packages" [Undecided,New] | 21:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1366164 | 21:00 |
| uvirtbot | Launchpad bug 1374259 in os-refresh-config "Coverage report was't created" [Undecided,In progress] | 21:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1367742 | 21:00 |
| uvirtbot | Launchpad bug 1366164 in tripleo "Add domain roles for admin user" [Undecided,New] | 21:00 |
| uvirtbot | Launchpad bug 1367742 in tripleo "keepalived brings the VIP up in multiple places on a controller reboot" [Undecided,New] | 21:00 |
| *** untriaged-bot has quit IRC | 21:00 | |
| *** mestery has quit IRC | 21:14 | |
| *** mestery has joined #tripleo | 21:15 | |
| SpamapS | https://bugs.launchpad.net/tripleo/+bug/1374626 | 21:15 |
| uvirtbot | Launchpad bug 1374626 in tripleo "UIDs of data-owning users might change between deployed images" [Critical,Triaged] | 21:15 |
| SpamapS | Check it out.. pretty interesting problem we just found. | 21:15 |
| slagle | sounds like that was fun to debug | 21:18 |
| slagle | :) | 21:18 |
| SpamapS | state is hard | 21:19 |
| *** morgan_remote_ has quit IRC | 21:20 | |
| lifeless | SpamapS: time to assert ownership? | 21:23 |
| SpamapS | lifeless: not sure what you mean. | 21:24 |
| lifeless | SpamapS: in /mnt/state | 21:24 |
| SpamapS | We do, I think, have to have a single place to map UID <-> name for images. | 21:24 |
| SpamapS | lifeless: Oh right thats what I'm saying. | 21:25 |
| SpamapS | simplest thing I think would be to backup /etc/passwd to /mnt/state so the state record keeps a copy of its own symbolic mapping... when booting into a new image and seeing that there, we need to freak out if any users are deleted or changed. | 21:26 |
| slagle | does ubuntu offer static id's for users? | 21:26 |
| slagle | i think most of the openstack users are already assigned static id's on fedora | 21:26 |
| slagle | (just wondering if there already is a map) | 21:26 |
| *** rhallisey has quit IRC | 21:27 | |
| SpamapS | yes | 21:29 |
| SpamapS | hm no | 21:29 |
| SpamapS | slagle: no it looks like it is unique per system | 21:29 |
| SpamapS | slagle: http://bazaar.launchpad.net/~ubuntu-branches/ubuntu/utopic/cinder/utopic/view/head:/debian/cinder-common.postinst | 21:31 |
| mordred | SpamapS: ldap ftw? (/me hides) | 21:31 |
| *** andreaf has quit IRC | 21:31 | |
| SpamapS | mordred: I choose _YOU_ to set that up. | 21:31 |
| *** andreaf has joined #tripleo | 21:32 | |
| slagle | SpamapS: yea, actually even on fedora the static id is only used if you install via the openstack packages | 21:32 |
| SpamapS | https://www.debian.org/doc/debian-policy/ch-opersys.html | 21:33 |
| slagle | still, maybe it's worth using that same id when installing from source | 21:33 |
| SpamapS | policy is a bit vague on whether or not debian/ubuntu should have static uids for them | 21:33 |
| mordred | SpamapS: morganfainberg and ayoung tell me that freeipa is the easy way to do that ... | 21:33 |
| SpamapS | but Debian only allocates 99 static UID's so it wouldn't scale to have any package have those uids | 21:34 |
| SpamapS | so the answer if doing packages is to pre-create the user | 21:34 |
| lifeless | SpamapS: so error is better than nothing | 21:34 |
| lifeless | SpamapS: but perhaps not that great operationally | 21:35 |
| SpamapS | slagle: I don't think we need the UID's to be static across all distros | 21:35 |
| SpamapS | lifeless: we can do a merge, I'm just... hesitant. ;) | 21:35 |
| SpamapS | like if the old image has cinder == 1001, and the new one has cinder == 1002, it's not a super expensive operation to do chmod -R | 21:35 |
| SpamapS | find rather | 21:35 |
| SpamapS | find and chmod | 21:35 |
| SpamapS | but ... | 21:35 |
| SpamapS | complexity ... :-P | 21:35 |
| lifeless | SpamapS: well, I was thinking perhaps an assertion | 21:36 |
| lifeless | SpamapS: not a merge | 21:36 |
| lifeless | e.g. | 21:36 |
| lifeless | /mnt/state//var/log/ shall be root:root and only root:root | 21:36 |
| SpamapS | Ah yeah thats what I was trying to avoid. :-P | 21:36 |
| SpamapS | Much better operationally, I agree. | 21:36 |
| SpamapS | and actually, that's easier to maintain than a static list | 21:37 |
| SpamapS | mmk I'm convinced | 21:38 |
| SpamapS | lifeless: +1 .. realistically it would only be in the tens of assertions | 21:38 |
| SpamapS | Goes along nicely with register-state-path too.. can have a register-state-owner or something | 21:38 |
| *** akuznetsov has quit IRC | 21:39 | |
| *** jml has quit IRC | 21:39 | |
| *** akuznetsov has joined #tripleo | 21:41 | |
| *** jml has joined #tripleo | 21:42 | |
| *** marun has joined #tripleo | 21:45 | |
| morganfainberg | SpamapS, mordred, i saw my name and FreeIPA | 21:47 |
| mordred | morganfainberg: freeipa all the things | 21:48 |
| morganfainberg | mordred, if only it worked on ubuntu :( | 21:48 |
| SpamapS | ow | 21:49 |
| morganfainberg | I greatly dislike that debian/ubuntu doesn't let us have (easily) specified / reserved uids | 21:49 |
| morganfainberg | it was an issue metacloud "fixed" by using a CMS to ensure uids were al ways the same and pre-created the users before isntalling packages and "dealing" with the fallout | 21:50 |
| morganfainberg | but it does suck that you get different IDs for different installs sometimes. | 21:51 |
| SpamapS | it only sucks when you care about UID's | 21:51 |
| morganfainberg | as a former sysadmin ... i want my environments to be consistent! | 21:51 |
| morganfainberg | ;) | 21:51 |
| SpamapS | the moral of the story is: don't care about UID's | 21:51 |
| morganfainberg | but... | 21:52 |
| SpamapS | don't | 21:52 |
| SpamapS | no | 21:52 |
| morganfainberg | haha | 21:52 |
| SpamapS | DON'T | 21:52 |
| SpamapS | :) | 21:52 |
| SpamapS | I recall fixing a 10-million file FTP server (when 10 million files was a lot of files) and having to fsck fully one time, and then having half the files come out with the wrong uids | 21:53 |
| SpamapS | tar to the rescue | 21:53 |
| morganfainberg | i also liked having a singl samhain profile for my entire system (and truely diskless os systems) | 21:53 |
| morganfainberg | haha, yeah tar would help there. | 21:53 |
| SpamapS | tar was our backup solution. :) zomg did it take forever to untar those files though | 21:53 |
| SpamapS | Luckily our contract for that data was 3 nines. :-P | 21:54 |
| morganfainberg | SpamapS, lol | 21:54 |
| SpamapS | "you'll get it when we godamn well feel like it" | 21:54 |
| morganfainberg | "9.99%" uptime right? | 21:54 |
| SpamapS | right | 21:54 |
| *** enterprisedc has quit IRC | 21:57 | |
| *** enterprisedc has joined #tripleo | 21:59 | |
| *** tserong has quit IRC | 22:05 | |
| *** tserong has joined #tripleo | 22:07 | |
| *** andreaf has quit IRC | 22:19 | |
| *** andreaf has joined #tripleo | 22:19 | |
| gfidente | well I've deployed freeipa though, there are nice things with it like the sudoers on ldap | 22:20 |
| gfidente | or the opportunity to kerberize everything | 22:20 |
| gfidente | it's not just the UIDs | 22:20 |
| gfidente | I know you don't *need* freeipa for that, was just sparing my 2cents | 22:21 |
| *** pensu has quit IRC | 22:26 | |
| openstackgerrit | Giulio Fidente proposed a change to openstack/tripleo-incubator: Improve documentation around usage of cinder-volume-nfs https://review.openstack.org/124347 | 22:36 |
| sdake | radez did you put your pki chane somewhere I can download it? | 22:43 |
| openstackgerrit | Ben Nemec proposed a change to openstack/diskimage-builder: Allow custom rootfs labels https://review.openstack.org/121944 | 22:44 |
| sdake | larsks what was rade'z fix? do you have it in a repo? | 22:46 |
| *** rwsu has joined #tripleo | 22:47 | |
| *** pcrews has quit IRC | 22:54 | |
| larsks | He was booting an original, un-updated Fedora 20 image. The kernel on that image is too old to support vxlan. The user-data script does a yum upgrade at boot, but you need to reboot to activate the new kernel. | 22:56 |
| larsks | I will update the README with a note re: kernel versions after kids are asleep. | 22:57 |
| sdake | running f20 on his host you mean? | 22:58 |
| sdake | what about the stuff about pki? | 22:58 |
| *** dkehn has quit IRC | 23:00 | |
| *** jml has quit IRC | 23:02 | |
| *** dkehn has joined #tripleo | 23:02 | |
| *** jml has joined #tripleo | 23:03 | |
| *** yamahata_ has joined #tripleo | 23:09 | |
| *** jang1 has quit IRC | 23:15 | |
| *** rwsu has quit IRC | 23:30 | |
| *** gfidente has quit IRC | 23:39 | |
| *** pcrews has joined #tripleo | 23:43 | |
| *** chuckC has quit IRC | 23:48 | |
| *** marun has quit IRC | 23:50 | |
| larsks | radez is booting up kube using my Heat templates. The instances were using the standard F20 cloud image, which is missing vxlan support because the kernel is too old. | 23:52 |
| larsks | I think he may have had to run the pki setup script. | 23:52 |
| *** dkehn has quit IRC | 23:53 | |
| *** dkehn has joined #tripleo | 23:55 | |
| *** jeblair is now known as corvus | 23:58 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!