| openstackgerrit | James Kirsch proposed openstack/kolla-ansible master: Add support for encrypting backend HAProxy traffic https://review.opendev.org/664516 | 00:00 |
|---|---|---|
| *** gfidente|afk has quit IRC | 00:51 | |
| *** zhanglong has joined #openstack-kolla | 01:15 | |
| *** dciabrin_ has joined #openstack-kolla | 01:21 | |
| *** dciabrin has quit IRC | 01:22 | |
| *** chrizl has quit IRC | 01:57 | |
| openstackgerrit | James Kirsch proposed openstack/kolla-ansible master: Add support for encrypting backend HAProxy traffic https://review.opendev.org/664516 | 02:13 |
| openstackgerrit | James Kirsch proposed openstack/kolla-ansible master: Add support for encrypting backend HAProxy traffic https://review.opendev.org/664516 | 02:22 |
| *** k_mouza has joined #openstack-kolla | 02:31 | |
| *** negronjl has quit IRC | 02:34 | |
| *** k_mouza has quit IRC | 02:35 | |
| *** negronjl has joined #openstack-kolla | 02:37 | |
| *** pbing19 has joined #openstack-kolla | 03:42 | |
| *** hongbin has joined #openstack-kolla | 03:48 | |
| *** dking_desktop has quit IRC | 04:23 | |
| *** pbing19 has quit IRC | 04:35 | |
| *** hongbin has quit IRC | 04:41 | |
| *** skramaja has joined #openstack-kolla | 04:45 | |
| *** goldyfruit has quit IRC | 04:49 | |
| *** pbing19 has joined #openstack-kolla | 05:07 | |
| *** Tony31 has joined #openstack-kolla | 05:14 | |
| Tony31 | good morning | 05:18 |
| *** vesper has quit IRC | 05:18 | |
| *** vesper11 has joined #openstack-kolla | 05:23 | |
| *** iniazi_ is now known as iniazi | 05:24 | |
| *** dcapone2004 has quit IRC | 05:30 | |
| *** evrardjp has quit IRC | 05:34 | |
| *** evrardjp has joined #openstack-kolla | 05:34 | |
| *** igordc has joined #openstack-kolla | 05:51 | |
| *** igordc has quit IRC | 06:00 | |
| openstackgerrit | Yongjun Bai proposed openstack/kolla-ansible master: WIP:Add support api service TLS. https://review.opendev.org/701704 | 06:09 |
| openstackgerrit | Yongjun Bai proposed openstack/kolla-ansible master: WIP:Add support api service TLS. https://review.opendev.org/701704 | 06:15 |
| *** rgogunskiy has joined #openstack-kolla | 06:24 | |
| mnasiadka | morning | 07:17 |
| *** cah_link has joined #openstack-kolla | 07:26 | |
| Tony31 | Hi good morning | 07:27 |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: WIP: CI: Add CentOS 8 ceph-ansible job https://review.opendev.org/706886 | 07:30 |
| Tony31 | has anyone modified horizons local_settings through kolla ? | 07:31 |
| Tony31 | Seems the config is not merged. I applied 2 x settings and it replaced local_settings and now the horizon is not reachable (http error 400) | 07:32 |
| *** rgogunskiy_ has joined #openstack-kolla | 07:32 | |
| Tony31 | Do I need to use the entire local_settings config ? | 07:32 |
| Tony31 | or is there possibly a different error on my part? | 07:32 |
| Tony31 | my other config was merged OK (cinder.conf / keystone.conf) | 07:33 |
| *** rgogunsk_ has joined #openstack-kolla | 07:33 | |
| *** rgogunskiy has quit IRC | 07:34 | |
| yoctozepto | Tony31: morning here indeed | 07:35 |
| yoctozepto | Tony31: good evening there | 07:36 |
| Tony31 | Evening :) | 07:36 |
| yoctozepto | Tony31: is this Train? | 07:36 |
| Tony31 | Indeed Train | 07:36 |
| yoctozepto | ok, looking | 07:36 |
| Tony31 | I've removed my kolla local_settings file, and doing a full reconfigure (hoping the default file comes back) | 07:36 |
| yoctozepto | Tony31: the file "to merge" should be named custom_local_settings | 07:37 |
| Tony31 | bah - I missed that. | 07:37 |
| yoctozepto | local_settings will indeed overwrite everything | 07:37 |
| Tony31 | thanks | 07:37 |
| *** rgogunskiy_ has quit IRC | 07:37 | |
| yoctozepto | Tony31: you might generally like https://docs.openstack.org/kolla-ansible/train/reference/index.html | 07:38 |
| yoctozepto | if kayobe somehow missed to link to that | 07:38 |
| Tony31 | oo thanks | 07:38 |
| Tony31 | yoctozepto Thanks for the link. Thats what I needed. I only had https://docs.openstack.org/kayobe/latest/configuration/kolla-ansible.html#service-configuration and https://docs.openstack.org/kolla-ansible/latest/admin/advanced-configuration.html#openstack-service-configuration-in-kolla | 07:43 |
| yoctozepto | yw | 07:44 |
| yoctozepto | worth reporting any docs shortcomings | 07:44 |
| Tony31 | I will, but I need to check if I was just blind-reading or something. | 07:45 |
| Tony31 | 😎 | 07:45 |
| yoctozepto | Tony31: looks like the paragraph just above the one mentioned by you has the right link | 07:46 |
| yoctozepto | well, it's a bit conflated | 07:46 |
| yoctozepto | the hardest part about software is docs | 07:47 |
| yoctozepto | not changing my mind | 07:47 |
| yoctozepto | anyways, we won't feel bad if you drop us reports about docs low quality on https://bugs.launchpad.net/kolla-ansible or https://bugs.launchpad.net/kolla-ansible | 07:49 |
| yoctozepto | always something to improve | 07:49 |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: WIP: CI: Add CentOS 8 ceph-ansible job https://review.opendev.org/706886 | 07:54 |
| Tony31 | I would like to help with it. I dont yet fully understand some things so I am planning to get it working; then I should understand it better | 07:56 |
| Tony31 | I may destroy what I have done and set it up again. It's not been difficult. I can then go through the guide again and make notes. | 07:57 |
| Tony31 | Some context or "summary" sections would be helpful for n00bs like myself | 07:57 |
| Tony31 | internally I always put a "summary" and give a high-level what the doc or JIRA ticket is about | 07:58 |
| Tony31 | It also makes it a bit easier to flick through pages to find the right one which you need (in my opinion) | 07:58 |
| *** tonythomas has joined #openstack-kolla | 07:58 | |
| openstackgerrit | Merged openstack/kolla-ansible master: Docs: show other versions https://review.opendev.org/706595 | 07:58 |
| yoctozepto | Tony31: indeed it does | 08:00 |
| openstackgerrit | Merged openstack/kolla-ansible master: Ironic: fix documentation https://review.opendev.org/706822 | 08:02 |
| Tony31 | I'm unable to get the `custom_local_settings` to get applied | 08:05 |
| Tony31 | default file is back, which is a good start :) | 08:06 |
| Tony31 | the custom local settings file literally has one line: `OPENSTACK_KEYSTONE_MULTIDOMAIN_SUPPORT = True` | 08:06 |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: Fix hardcoded gnocchi keyring https://review.opendev.org/707073 | 08:06 |
| Tony31 | there's no `[DEFAULT]` section in `local_settings` | 08:08 |
| mnasiadka | Tony31: just set horizon_keystone_multidomain: "yes" in kolla's globals.yml | 08:08 |
| Tony31 | ok i will try | 08:08 |
| Tony31 | I just noticed the custom local settings file is copied to `/etc/kolla/horizon` on the controller | 08:08 |
| Tony31 | thank you mnasiadka | 08:09 |
| *** bengates has joined #openstack-kolla | 08:10 | |
| Tony31 | mnasiadka - domain support is enabled now, confirmed with grep on the config file.. Just waiting for services to come up. | 08:14 |
| *** pbing19 has quit IRC | 08:16 | |
| yoctozepto | Tony31: this is exactly my override and it works ;-) | 08:16 |
| openstackgerrit | Chason Chan proposed openstack/kolla master: Docs: show other versions https://review.opendev.org/707074 | 08:17 |
| Tony31 | :D | 08:17 |
| *** xinliang has quit IRC | 08:19 | |
| mnasiadka | yoctozepto: https://zuul.opendev.org/t/openstack/build/cd09bc63a3534def93cd71b27131af3f/log/primary/logs/kolla/all-CRITICAL.txt#2 | 08:20 |
| mnasiadka | yoctozepto: maybe it's time for oslo.db to handle such message? :D | 08:20 |
| Tony31 | Last time I configured the "domain" for LDAP, I then had to run some openstack commands to set it up. Is this still the same today or is it done in kolla/kayobe with a config file? | 08:20 |
| Tony31 | `openstack domain create mycompany` | 08:22 |
| openstackgerrit | Radosław Piliszek proposed openstack/kolla stable/rocky: Change fetch_fernet_tokens.py behaviour to check for minimum number of tokens https://review.opendev.org/705960 | 08:22 |
| openstackgerrit | Radosław Piliszek proposed openstack/kolla stable/rocky: nova-libvirt: add UEFI packages to support UEFI instances https://review.opendev.org/706065 | 08:22 |
| openstackgerrit | Radosław Piliszek proposed openstack/kolla stable/rocky: [rocky] Bump versions https://review.opendev.org/706661 | 08:23 |
| yoctozepto | Tony31: same | 08:23 |
| yoctozepto | keystone is not that good at ldap domains | 08:23 |
| Tony31 | it makes things slow, right? | 08:24 |
| yoctozepto | mnasiadka: lol, and it's single-node | 08:25 |
| yoctozepto | no errors in mariadb logs gah | 08:25 |
| yoctozepto | Tony31: nah, the performance is fine here because the directory is small | 08:25 |
| yoctozepto | Tony31: but the problem is that the code is fugly and you have to do these things manually | 08:26 |
| mnasiadka | yoctozepto: maybe we need to set some variable on single node to ignore some quirks... | 08:26 |
| yoctozepto | after creating domain you have to restart keystone as well | 08:26 |
| Tony31 | ok thanks | 08:26 |
| yoctozepto | to let it pick up the ldap config | 08:26 |
| yoctozepto | etc. | 08:26 |
| Tony31 | understood - | 08:26 |
| mnasiadka | yoctozepto: you don't need to - you can set the domain auth settings using API :) | 08:26 |
| yoctozepto | mnasiadka: well, neutron-server got CRIT | 08:27 |
| Tony31 | I am just trying to run some openstack commands now and i get an error that it can't connect to the internal API url | 08:27 |
| mnasiadka | unless you are very persistent on using config from files :) | 08:27 |
| yoctozepto | mnasiadka: I must've missed that | 08:27 |
| yoctozepto | I love docs | 08:27 |
| Tony31 | running from my local system, sourced the RC file which points to the public url. I try "nova list" and get an error about the internal API | 08:27 |
| mnasiadka | yoctozepto: yeah well, regarding mariadb and neutron - we either add 1047 to oslo.db filter, or we fix mariadb - chose one :) | 08:27 |
| yoctozepto | Tony31: doesn't the rc tell it to use internal APIs? | 08:28 |
| *** carli has joined #openstack-kolla | 08:28 | |
| Tony31 | RC file points to the public url | 08:28 |
| mnasiadka | yoctozepto: https://docs.openstack.org/api-ref/identity/v3/?expanded=create-domain-configuration-detail#create-domain-configuration :) | 08:28 |
| *** priteau has joined #openstack-kolla | 08:28 | |
| mnasiadka | yoctozepto: of course openstackclient does not support that, but curl does ;) | 08:28 |
| yoctozepto | mnasiadka: no idea, maybe we should just use upstream mariadb as on ubuntu as this one has some weirdness | 08:29 |
| yoctozepto | mnasiadka: must be the reason I did not use it! :D | 08:29 |
| mnasiadka | yoctozepto: I had hundreds of keystone domains each using different ldap server :) | 08:29 |
| hrw | morning | 08:29 |
| yoctozepto | mnasiadka: I am really puzzled as to how a single-node mariadb managed to produce that error message | 08:30 |
| Tony31 | ok, so. I get an error about being unable to connect to the internal api if I use `nova list` but the command works if I use `openstack server list`. | 08:31 |
| mnasiadka | yoctozepto: well, is there a big version difference between centos and ubuntu? | 08:31 |
| Tony31 | I guess this is probably normal - | 08:31 |
| yoctozepto | mnasiadka: same minor, different patch, newer galera in this - though always the issue is that centos downstream does its own patches | 08:32 |
| yoctozepto | so hard to compare | 08:32 |
| mnasiadka | ugh, I love these ugly approaches | 08:32 |
| yoctozepto | Tony31: old commands use different variables | 08:33 |
| yoctozepto | I believe rc provides proper only for new osc | 08:33 |
| Tony31 | ah ok - i need to get used to it :) | 08:33 |
| *** pbing19 has joined #openstack-kolla | 08:40 | |
| hrw | folks: infra type images work for all-in-one local install. I now need to get that working on CI. any help? | 08:42 |
| hrw | https://review.opendev.org/#/c/698753/ https://review.opendev.org/#/c/706982/ implement kolla and kolla-ansible parts | 08:42 |
| patchbot | patch 698753 - kolla - WIP: Create 'infra' type of images - 12 patch sets | 08:42 |
| patchbot | patch 706982 - kolla-ansible - WIP: Use 'infra' type of images - 1 patch set | 08:42 |
| *** jopdorp has joined #openstack-kolla | 08:43 | |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: Fix fernet bootstrap and key distribution - follow up https://review.opendev.org/707080 | 08:44 |
| *** riuzen has joined #openstack-kolla | 08:45 | |
| openstackgerrit | Marcin Juszkiewicz proposed openstack/kolla-ansible master: WIP: Use 'infra' type of images https://review.opendev.org/706982 | 08:45 |
| hrw | this may help | 08:45 |
| openstackgerrit | Marcin Juszkiewicz proposed openstack/kolla-ansible master: WIP: Use 'infra' type of images https://review.opendev.org/706982 | 08:47 |
| hrw | and CI jobs slashed | 08:47 |
| *** dking_desktop has joined #openstack-kolla | 08:47 | |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: WIP: CI: Add CentOS 8 ceph-ansible job https://review.opendev.org/706886 | 08:52 |
| *** rpittau|afk is now known as rpittau | 08:55 | |
| *** xinliang has joined #openstack-kolla | 08:55 | |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: External Ceph: Fix hardcoded gnocchi keyring https://review.opendev.org/707073 | 08:59 |
| *** dking_desktop has quit IRC | 09:02 | |
| *** dougsz has joined #openstack-kolla | 09:14 | |
| openstackgerrit | Merged openstack/kolla stable/rocky: Fixes fluentd for rocky https://review.opendev.org/706713 | 09:19 |
| *** Tony31 has quit IRC | 09:20 | |
| *** k_mouza has joined #openstack-kolla | 09:21 | |
| *** gbatir_ has joined #openstack-kolla | 09:26 | |
| mgoddard | morning all | 09:33 |
| gbatir_ | hello all | 09:33 |
| gbatir_ | i've been testing ovn using this patch https://review.opendev.org/#/c/696841/89 for the last few weeks | 09:34 |
| patchbot | patch 696841 - kolla-ansible - OVN Support - 89 patch sets | 09:34 |
| gbatir_ | on a 16 node cluster | 09:34 |
| gbatir_ | and it seems to work great | 09:35 |
| gbatir_ | but going over 150-200 vms i get hit by these issues | 09:35 |
| gbatir_ | https://www.openvswitch.org/support/ovscon2019/day1/1436-OVSCON-Nouman.pdf | 09:36 |
| gbatir_ | is it possible to deploy by default ovn with those intervals increased from the default of 5s | 09:37 |
| gbatir_ | ? | 09:37 |
| riuzen | hi, i want to try magnum service on kolla-ansible, any recommendation which openstack version or ansible-kolla should i use? | 09:37 |
| openstackgerrit | Marcin Juszkiewicz proposed openstack/kolla-ansible master: WIP: Use 'infra' type of images https://review.opendev.org/706982 | 09:38 |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: WIP: CI: Add CentOS 8 ceph-ansible job https://review.opendev.org/706886 | 09:38 |
| mnasiadka | gbatir_: Hi there, currently not - but I can add these to the deployment - so a user can fine tune them - thanks for the reference | 09:40 |
| mnasiadka | I'll sync the defaults with tripleo | 09:43 |
| *** ivve has joined #openstack-kolla | 09:53 | |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: OVN Support https://review.opendev.org/696841 | 10:01 |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible stable/train: Ironic: fix documentation https://review.opendev.org/707090 | 10:04 |
| mgoddard | riuzen: train is the latest openstack release, although Stein has had more exposure | 10:07 |
| *** tony31 has joined #openstack-kolla | 10:07 | |
| tony31 | whats the best way to restart keystone? | 10:08 |
| tony31 | kill the container or ? | 10:09 |
| tony31 | I done `docker restart keystone` | 10:10 |
| openstackgerrit | Mark Goddard proposed openstack/kayobe master: [ussuri][goal] Drop python 2.7 support and testing https://review.opendev.org/695617 | 10:12 |
| mgoddard | tony31: that's correct | 10:13 |
| tony31 | thank you | 10:13 |
| jopdorp | Hi all, I've been working on a multinode deployment of kolla | 10:16 |
| jopdorp | with ceph | 10:16 |
| tony31 | had to use red hat docs to figure out how to make the default `admin` openstack user, be able to set domain context of the domain, so I could add a project under the domain | 10:16 |
| jopdorp | and I found that there weere plenty of challenges along the way | 10:16 |
| tony31 | ref: https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/16.0/html/integrate_with_identity_service/sec-active-directory | 10:16 |
| jopdorp | Right now I'm still working on getting cloudkitty to run | 10:17 |
| jopdorp | If anyone is interested I could extend the documentation | 10:17 |
| jopdorp | write something like a production example, those were useful in the openstack-ansible docs | 10:18 |
| tony31 | sounds great :) | 10:19 |
| jopdorp | tony31, what kind of deployment are you working on? | 10:20 |
| tony31 | jopdorp I am using kayobe to learn about it and deploy it for a single org. | 10:20 |
| tony31 | ...to replace an old Pike/TripleO deployment that cannot be updated or maintained | 10:20 |
| jopdorp | kayobe is a deployment on kubernetes right? | 10:20 |
| tony31 | from what I understand about it, kayobe deploys kolla-ansible | 10:21 |
| jopdorp | mgoddard, any idea about the cloudkitty images? they seem to be unavailable | 10:21 |
| tony31 | someone could correct me if I am wrong | 10:21 |
| jopdorp | right | 10:22 |
| jopdorp | maybe i should give it a try too | 10:22 |
| jopdorp | I'm trying out different openstack deployment approaches as a research for out new public cloud :) | 10:22 |
| jopdorp | https://pastebin.com/C5HWhTbD | 10:23 |
| jopdorp | cloudkitty image seems to be unavailable | 10:23 |
| tony31 | well having experience of tripleo, I don't recommend that unless you have a team of people that know it very well. | 10:24 |
| tony31 | I am a lone person that is doing this for my org, and kayobe so far is fantastic in comparisson | 10:24 |
| jopdorp | nice | 10:25 |
| jopdorp | I did openstack-ansible | 10:25 |
| jopdorp | and kolla ansible | 10:25 |
| openstackgerrit | Merged openstack/kolla master: Docs: show other versions https://review.opendev.org/707074 | 10:25 |
| tony31 | I just checked the ansible file for kayobe and `cloudkitty` is there as an option to enable | 10:25 |
| jopdorp | with some scripts to automate provisioning the bare metal with maas | 10:25 |
| mnasiadka | jopdorp: as per https://docs.openstack.org/kolla/latest/support_matrix.html - cloudkitty does not build for ubuntu/binary | 10:25 |
| jopdorp | mnasiadka great link! | 10:26 |
| jopdorp | maybe I'll migrate the entire cluster to centos then | 10:26 |
| jopdorp | seems to have slightly better support | 10:27 |
| *** priteau has quit IRC | 10:28 | |
| *** riuzen has quit IRC | 10:31 | |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible master: Ironic: fix documentation again https://review.opendev.org/707093 | 10:32 |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible master: WIP: Symlink deploy-guide to docs https://review.opendev.org/707096 | 10:36 |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible master: DNM: change docs https://review.opendev.org/707097 | 10:36 |
| mgoddard | jopdorp: cloudkitty images are unavailable for ubuntu/binary: https://docs.openstack.org/kolla/latest/support_matrix.html | 10:39 |
| mgoddard | oops, mnasiadka got there first | 10:39 |
| jopdorp | haha | 10:40 |
| jopdorp | thanks! | 10:40 |
| jopdorp | I've destroyed the cluster, now trying it with centos | 10:40 |
| mgoddard | good plan :p | 10:40 |
| jopdorp | hehe | 10:40 |
| jopdorp | is binary release even recommended? | 10:40 |
| jopdorp | I'm using the binaries because I think it might be faster to deploy and may give less problems, as the compilation can't fail anymore | 10:41 |
| tony31 | general question if that's ok? My devops guys are deploying into kubernetes. At the moment we use google GKE as well as internally > kubernetes on VMs on Openstack. I have looked at OKD Which is the open source Openshift. Is there an openstack project which gives the same? Would it be `Zun` ? | 10:41 |
| jopdorp | is there any better docs to get octavia to run than this? https://www.lijiawang.org/posts/kolla-octavia.html | 10:42 |
| jopdorp | because that one is kind of chinese :p | 10:42 |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: Remove kolla-ceph https://review.opendev.org/704309 | 10:43 |
| mnasiadka | jopdorp: binary vs source - https://docs.openstack.org/kolla/latest/admin/image-building.html#build-openstack-from-source | 10:43 |
| jopdorp | thanks mnasiadka | 10:44 |
| jopdorp | is it correct to think that i need to install octavia to get loadbalancing to work? | 10:45 |
| *** pbing19 has quit IRC | 10:45 | |
| jopdorp | because when i pressed the loadbalancer item in horizon i got all these errors saying that the load balancer endpoint couldn't be found | 10:45 |
| mnasiadka | yes, octavia is the lbaas implementation | 10:47 |
| *** tony31 has quit IRC | 10:52 | |
| jopdorp | mnasiadka, any better docs about how to configure the certificates than this https://www.lijiawang.org/posts/kolla-octavia.html | 10:53 |
| jopdorp | ? | 10:53 |
| mnasiadka | jopdorp: the official octavia one should be good? https://docs.openstack.org/octavia/latest/admin/guides/certificates.html | 10:54 |
| jopdorp | ah, great, thanks! | 10:54 |
| jopdorp | I'm now deploying without octavia | 10:55 |
| *** tony31 has joined #openstack-kolla | 10:55 | |
| mgoddard | jopdorp: if you are using train, there was a bug about us configuring certificates incorrectly | 10:56 |
| jopdorp | mnasiadka if I want to add the service separately, is it correct that I enable it in globals.yml and then do "kolla-ansible -i ./multinode deploy --tags octavia" or are there more steps? or do i need to use "reconfigure" ? | 10:56 |
| mgoddard | fix was proposed here: https://review.opendev.org/706203 | 10:56 |
| patchbot | patch 706203 - kolla-ansible - Adapt to Octavia Certificate Configuration Guide. - 3 patch sets | 10:56 |
| jopdorp | maybe I will wait for the fix | 10:57 |
| jopdorp | anyway it's good to know how to add services to a running cluster, without interrupting my clinet's vms | 10:57 |
| jopdorp | clients | 10:57 |
| jopdorp | just deploy with the --tags? | 10:57 |
| jopdorp | i read that for adding compute nodes i need to use reconfigure | 10:58 |
| jopdorp | but that's not the same as adding services | 10:58 |
| tony31 | I need to check cinder api log but the /var/log/cinder is empty in the container. Could anyone tell me where do the logs go? | 11:00 |
| jopdorp | they should be on the controller node | 11:01 |
| tony31 | think i found it | 11:01 |
| tony31 | `/var/log/kolla/cinder` | 11:01 |
| jopdorp | yeah | 11:01 |
| tony31 | thanks :) | 11:01 |
| jopdorp | :p no problem, I'm also still a total noob, but hope to be giving other people some more support soon ^_^ | 11:02 |
| jopdorp | mnasiadka and mgoddard, are you employed by a kolla foundation or something? | 11:02 |
| *** priteau has joined #openstack-kolla | 11:03 | |
| mgoddard | jopdorp: http://stackhpc.com/ | 11:03 |
| mgoddard | consultancy available :) | 11:03 |
| *** pbing19 has joined #openstack-kolla | 11:04 | |
| tony31 | what is a "weighted backend" my cinder api error was that "no weighted backend found for volume type" | 11:05 |
| tony31 | during volume create | 11:05 |
| priteau | tony31: Cinder has a scheduler which goes through filters and weighers to select where to host your volume. This probably means that your request didn't find any backend that can be used. | 11:12 |
| *** zhanglong has quit IRC | 11:12 | |
| *** zhanglong has joined #openstack-kolla | 11:15 | |
| tony31 | Hi Priteau, I think you are right. I must have mis-config somewhere. I'll work on it tomorrow, as now I am in dire need of food. Thanks for responding. :) | 11:19 |
| jopdorp | mgoddard http://stackhpc.com/ looks interesting! | 11:21 |
| *** rpittau is now known as rpittau|bbl | 11:21 | |
| mgoddard | jopdorp: it is :) | 11:22 |
| jopdorp | so you guys developed kayobe? | 11:24 |
| jopdorp | haha | 11:24 |
| jopdorp | okay, you basically made it by yourself :p | 11:24 |
| jopdorp | 80% of all commits xD | 11:25 |
| openstackgerrit | Chason Chan proposed openstack/kayobe master: Docs: show other versions https://review.opendev.org/707113 | 11:25 |
| jopdorp | maybe I'll give kayobe a try too | 11:26 |
| jopdorp | I've been writing a ton of scripts to be able to deploy kolla to unprovisioned machines in one go | 11:27 |
| jopdorp | looks like you did the same, but are much much further, mgoddard | 11:28 |
| mgoddard | jopdorp: I can recommend kayobe. Completely unbiased :) | 11:28 |
| jopdorp | lol | 11:32 |
| jopdorp | I completely believe you | 11:32 |
| *** sluna has joined #openstack-kolla | 11:34 | |
| *** tony31 has quit IRC | 11:34 | |
| mgoddard | seriously though, sounds like it might fit your needs | 11:38 |
| hrw | https://marcin.juszkiewicz.com.pl/2020/02/11/my-whole-carrier-is-built-on-foss/ | 11:40 |
| *** pbing19 has quit IRC | 11:46 | |
| yoctozepto | hrw: The page isn’t redirecting properly | 11:47 |
| hrw | reload | 11:47 |
| yoctozepto | better | 11:47 |
| jopdorp | mgoddard, I think it might! | 11:47 |
| jopdorp | see you later! gotto go :) | 11:48 |
| *** jopdorp has quit IRC | 11:48 | |
| cosmicsound | good day | 11:49 |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: WIP: CI: Add CentOS 8 ceph-ansible job https://review.opendev.org/706886 | 11:53 |
| *** vmixor has joined #openstack-kolla | 11:55 | |
| hrw | 2020-02-11T10:58:31.160997132Z ++ ln -s /etc/openstack-dashboard/custom_local_settings /openstack_dashboard/local/custom_local_settings.py | 11:57 |
| hrw | 2020-02-11T10:58:31.164582162Z ln: failed to create symbolic link '/openstack_dashboard/local/custom_local_settings.py': No such file or directory | 11:57 |
| hrw | again horizon ;( | 11:57 |
| hrw | to fix this and infra should pass all-in-one on CI | 11:57 |
| *** Wasaac has quit IRC | 12:00 | |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: CI: Add CentOS 8 ceph-ansible job https://review.opendev.org/706886 | 12:00 |
| *** Wasaac has joined #openstack-kolla | 12:00 | |
| hrw | hm... extend_start.sh for horizon is wrong... | 12:09 |
| hrw | 2020-02-11T10:58:31.160569937Z ++ CUSTOM_SETTINGS_FILE=/openstack_dashboard/local/custom_local_settings.py | 12:09 |
| hrw | ${SITE_PACKAGES} became empty | 12:09 |
| *** ricolin has quit IRC | 12:10 | |
| *** Wasaac has quit IRC | 12:11 | |
| *** Wasaac has joined #openstack-kolla | 12:12 | |
| openstackgerrit | Marcin Juszkiewicz proposed openstack/kolla master: WIP: Create 'infra' type of images https://review.opendev.org/698753 | 12:13 |
| hrw | added small hack to horizon | 12:14 |
| *** lklimin has joined #openstack-kolla | 12:15 | |
| *** kplant has joined #openstack-kolla | 12:19 | |
| *** gfidente has joined #openstack-kolla | 12:21 | |
| *** bengates has quit IRC | 12:38 | |
| openstackgerrit | Yongjun Bai proposed openstack/kolla-ansible master: Add support for encrypting nova api https://review.opendev.org/707131 | 12:39 |
| * hrw off | 12:40 | |
| *** bengates has joined #openstack-kolla | 12:54 | |
| openstackgerrit | Merged openstack/kayobe master: Docs: show other versions https://review.opendev.org/707113 | 12:54 |
| *** vmixor has quit IRC | 12:55 | |
| *** vmixor has joined #openstack-kolla | 12:58 | |
| *** carli has quit IRC | 13:01 | |
| *** tony31 has joined #openstack-kolla | 13:03 | |
| *** mixor has joined #openstack-kolla | 13:03 | |
| *** vmixor has quit IRC | 13:04 | |
| openstackgerrit | Mark Goddard proposed openstack/kayobe master: Stop gzipping logs in get-logs.sh https://review.opendev.org/707137 | 13:05 |
| cosmicsound | Guess the undercloud is not recommended to be my working pc | 13:13 |
| *** goldyfruit has joined #openstack-kolla | 13:18 | |
| tony31 | Unable to get this cinder backend to work. Config matches what I have running in prod and as far as I Can see the doc is still the same https://docs.openstack.org/cinder/train/configuration/block-storage/drivers/nimble-volume-driver.html. The error in the scheduler log is that the backend is not found. There is no log in the api log. | 13:18 |
| tony31 | I've restarted the containers cinder api and cinder scheduler | 13:19 |
| tony31 | also ran reconfigure. | 13:19 |
| tony31 | Is there possibly anything else I've missed ? | 13:19 |
| *** rpittau|bbl is now known as rpittau | 13:22 | |
| tony31 | kolla.yml has both `kolla_enable_cinder: yes` and `kolla_enable_cinder_backend_iscsi: yes` | 13:22 |
| *** vmixor has joined #openstack-kolla | 13:25 | |
| *** mixor has quit IRC | 13:26 | |
| *** ne0h|nihilist has joined #openstack-kolla | 13:28 | |
| *** zhanglong has quit IRC | 13:33 | |
| *** zhanglong has joined #openstack-kolla | 13:34 | |
| mnasiadka | tony31: I think we are not installing nimble drivers in cinder-volume image | 13:38 |
| tony31 | I done a search and the nimble.py is there | 13:38 |
| tony31 | but i am not sure if that is correct / all that is required | 13:39 |
| tony31 | could you explain some more? maybe I could check / confirm? | 13:39 |
| mnasiadka | or maybe it's in tree | 13:39 |
| tony31 | is there meant to be a cinder volume container? | 13:40 |
| mnasiadka | they removed it in Ussuri from in-tree, so should work | 13:40 |
| tony31 | i have only cinder-api and cinder-scheduler | 13:40 |
| tony31 | Sorry I dont understand "tree" | 13:41 |
| mnasiadka | in-tree means it should be included in cinder code | 13:43 |
| mnasiadka | tony31: but I'm pretty sure you need cinder-volume | 13:43 |
| tony31 | I am wondering if there should be a cinder-volume container ? | 13:43 |
| tony31 | `# docker ps | grep cind | 13:44 |
| tony31 | cinder_api | 13:44 |
| yoctozepto | tony31: sleepless night? | 13:45 |
| mnasiadka | tony31: there should be, but you need a node in [storage] group | 13:45 |
| tony31 | yes this cinder problem causing it :) | 13:45 |
| tony31 | bah | 13:45 |
| tony31 | thanks - I dont have any storage nodes because I have external storage - I thought | 13:46 |
| tony31 | can I add the controller node as a storage node? | 13:47 |
| mnasiadka | should work, it's external storage - I assume Nova will just take iscsi urls from Cinder | 13:48 |
| mnasiadka | tony31: but I would rethink twice using Nimble with Cinder - in Ussuri support for it has been removed | 13:49 |
| mnasiadka | tony31: https://opendev.org/openstack/cinder/commit/83267c72ae6f66eb3d368296676cc1cdd9837e88 | 13:49 |
| mnasiadka | and in Train it was marked as unsupported | 13:49 |
| *** reph has joined #openstack-kolla | 13:50 | |
| cosmicsound | How much experience do you people have with SSDs running OSes and failing if not on RAID | 13:50 |
| tony31 | mnasiadka thats no problem as far as I understand this, it's just because nimble storage / hpe is not maintaining the driver. I'll have to deploy the driver myself during the ussuri upgrade (somehow) | 13:50 |
| tony31 | cosmicsound - I have smasung enterprise grade 500gb ssd's in a Lenovo running raid for almost 3 years only. | 13:51 |
| tony31 | no issues yet | 13:51 |
| tony31 | Do I need to manually add something like `[storage]` in the kayobe inventory file ? | 13:52 |
| tony31 | ok - there is a storage group in the groups file, so I copied that to the hosts inventory and added the controller node under it | 13:55 |
| tony31 | will try a host configure / reconfigure deploy | 13:55 |
| *** kplant has quit IRC | 14:03 | |
| *** pbing19 has joined #openstack-kolla | 14:05 | |
| *** kplant has joined #openstack-kolla | 14:07 | |
| *** zhanglong has quit IRC | 14:28 | |
| *** TrevorV has joined #openstack-kolla | 14:36 | |
| *** goldyfruit has quit IRC | 14:38 | |
| *** goldyfruit has joined #openstack-kolla | 14:38 | |
| *** dking_desktop has joined #openstack-kolla | 14:40 | |
| openstackgerrit | Viktor Michalek proposed openstack/kolla-ansible master: Support for custom API paste files https://review.opendev.org/695054 | 14:43 |
| *** reph has quit IRC | 14:47 | |
| *** reph has joined #openstack-kolla | 14:48 | |
| openstackgerrit | Viktor Michalek proposed openstack/kolla-ansible master: Added support for custom logging.conf https://review.opendev.org/679812 | 14:48 |
| openstackgerrit | Viktor Michalek proposed openstack/kolla-ansible master: Support for custom API paste files https://review.opendev.org/695054 | 14:49 |
| openstackgerrit | Viktor Michalek proposed openstack/kolla-ansible master: Adds support of CADF notifying for other APIs next to keystone. https://review.opendev.org/674579 | 14:49 |
| dking_desktop | yoctozepto: Sorry, it seems that I managed to be logged out this morning. Was there any update? | 14:50 |
| dking_desktop | I'm wondering what the common scenario is for Ironic with k-a with regard to VLANs. | 14:53 |
| *** skramaja has quit IRC | 14:56 | |
| *** reph has quit IRC | 14:58 | |
| *** reph has joined #openstack-kolla | 14:58 | |
| tony31 | `2020-02-11 22:23:58.264 43 WARNING cinder.utils [-] Volume driver (NimbleISCSIDriver 4.0.1) is currently unsupported and may be removed in the next release of OpenStack. Use at your own risk. | 15:11 |
| tony31 | the driver section in cinder.conf` | 15:11 |
| tony31 | 😬 | 15:11 |
| *** rgogunsk_ has quit IRC | 15:18 | |
| openstackgerrit | Julia Kreger proposed openstack/kayobe master: [docs] Change CoreOS URLs to Centos URLs fir IPA https://review.opendev.org/707171 | 15:26 |
| *** gfidente has quit IRC | 15:27 | |
| *** jopdorp has joined #openstack-kolla | 15:28 | |
| jopdorp | any idea about trove-dashboard crashing horizon? | 15:28 |
| jopdorp | i get this when accessing horizon dashboard: | 15:28 |
| openstackgerrit | Mark Goddard proposed openstack/kayobe master: [docs] Change CoreOS URLs to Centos URLs fir IPA https://review.opendev.org/707171 | 15:28 |
| jopdorp | https://pastebin.com/7W6eNNAy | 15:28 |
| *** gfidente has joined #openstack-kolla | 15:32 | |
| *** chrizl has joined #openstack-kolla | 15:39 | |
| *** gbatir_ has quit IRC | 15:52 | |
| *** KeithMnemonic has quit IRC | 15:56 | |
| *** Trevor_V has joined #openstack-kolla | 16:03 | |
| openstackgerrit | Mark Goddard proposed openstack/kayobe master: [docs] Change CoreOS URLs to Centos URLs fir IPA https://review.opendev.org/707171 | 16:06 |
| *** TrevorV has quit IRC | 16:07 | |
| *** dcapone2004 has joined #openstack-kolla | 16:13 | |
| *** ivve has quit IRC | 16:13 | |
| *** Wasaac has quit IRC | 16:48 | |
| *** reph has quit IRC | 16:48 | |
| *** Wasaac has joined #openstack-kolla | 16:49 | |
| jopdorp | I can't get my vm's to connect to the internet :( | 16:51 |
| jopdorp | then I found this: https://docs.openstack.org/install-guide/launch-instance-networks-provider.html | 16:51 |
| jopdorp | but running "openstack network create --share --external --provider-physical-network provider --provider-network-type flat provider" | 16:51 |
| jopdorp | gives me an error | 16:51 |
| jopdorp | Error while executing command: BadRequestException: 400, Invalid input for operation: physical_network 'provider' unknown for flat provider network. | 16:51 |
| jopdorp | I thought i didn't need to enable provider networks in globals to get internet connection for a vm. I've successfully install openstack with kolla ansible now | 16:52 |
| *** kplant has quit IRC | 16:52 | |
| jopdorp | and i ran the init script kollavenv/share/kolla-ansible/init-runonce | 16:53 |
| *** kplant has joined #openstack-kolla | 16:53 | |
| jopdorp | but the demo vm doesn't have an internet connection | 16:53 |
| *** kplant has quit IRC | 16:57 | |
| *** kplant has joined #openstack-kolla | 16:58 | |
| *** pbing19 has quit IRC | 16:59 | |
| *** rpittau is now known as rpittau|afk | 17:00 | |
| *** priteau has quit IRC | 17:00 | |
| *** bengates has quit IRC | 17:01 | |
| *** bengates has joined #openstack-kolla | 17:01 | |
| yoctozepto | dking_desktop: not aware of any updates | 17:04 |
| dking_desktop | Okay. Thanks for the info. I'm just setting ironic_inspector_internal_fqdn to the external entry in globals.yml. That's probably not the best solution, but it works for me for now. | 17:06 |
| openstackgerrit | Merged openstack/kayobe master: [docs] Change CoreOS URLs to Centos URLs fir IPA https://review.opendev.org/707171 | 17:07 |
| *** dave-mccowan has joined #openstack-kolla | 17:09 | |
| mgoddard | dking_desktop: ironic networking can be fiddly, especially if you want multi-tenancy | 17:11 |
| mgoddard | the nodes need to be able to access the internal API network from the cleaning and provisioning networks (if different) | 17:12 |
| mgoddard | also cleaning and provisioning share the same TFTP server, so routing may need to be provided for this | 17:12 |
| mgoddard | we configure the TFTP server on teh internal API network, and assume routing exists from cleaning and provisioning networks to get to it | 17:13 |
| jopdorp | mgoddard, no tips for getting networking to work for the vms? | 17:15 |
| dking_desktop | In my current setup, we're using VLANs to isolate networks. However, our baremetal nodes' PXE functionality doesn't handle VLANs. Already, I'm going to be using the native VLAN on the switch port to direct untagged traffic to the VLAN I'm using for my dnsmasq interface. I'm not certain how such nodes could also access the separate internal API network before they're provisioned. | 17:16 |
| mgoddard | jopdorp: the external network needs to have a gateway configured that can access the internet | 17:16 |
| *** ivve has joined #openstack-kolla | 17:16 | |
| mgoddard | dking_desktop: the ironic_dnsmasq container is just for inspection. Neutron provides DHCP for provisioning and cleaning | 17:17 |
| jopdorp | mgoddard, does that mean I need to have provider networks enabled in globals? | 17:17 |
| mgoddard | jopdorp: not necessarily | 17:17 |
| mgoddard | dking_desktop: have you looked at networking-generic-switch or networking-ansible to manage VLANs on your switches? | 17:18 |
| *** pbing19 has joined #openstack-kolla | 17:18 | |
| mgoddard | or other vendor-specific drivers if supported by your switches | 17:18 |
| jopdorp | mgoddard, when i try to create a network from horizon with a subnet that is the same as my physical subnet which contains the gateway, it just fails | 17:18 |
| mgoddard | you can only have one flat network per physnet | 17:19 |
| jopdorp | does this make sense? | 17:19 |
| jopdorp | public1-subnet10.0.2.0/24IPv410.10.8.1149 | 17:19 |
| jopdorp | the gateway on a different cidr than the subnet itself | 17:20 |
| dking_desktop | No, I haven't looked at that yet. Once things are running, we'll be looking at building tools to automate changes on the switch. | 17:20 |
| dking_desktop | I'm not certain that I understand what you are suggesting, though. Are you suggesting that the nodes first be introspected, and then change the switch ports to have untagged packets routed differently later? I'm sorry if I'm missing the intent. | 17:22 |
| mgoddard | jopdorp: nope. gateway must be on the subnet otherwise it could not be reached | 17:23 |
| *** igordc has joined #openstack-kolla | 17:24 | |
| jopdorp | mgoddard, does that mean I need to create an ip on the compute host within the cidr 10.0.2.0/24 and create a gateway in that cidr too? | 17:26 |
| jopdorp | I'm not sure where the 10.0.2.0/24 idea comes from, why didn't it use a cidr that was configured in one of my network interfaces | 17:26 |
| mgoddard | dking_desktop: if you aren't changing VLANs I'd suggest using flat networking for now. cleaning == provisioning == tenant network | 17:28 |
| mgoddard | jopdorp: init-runonce is just for testing. In production you need to create a network using a subnet that has internet access | 17:28 |
| mgoddard | in your case 10.10.8.0 | 17:29 |
| jopdorp | thanks | 17:29 |
| jopdorp | I'm doing that now | 17:29 |
| jopdorp | :p hope it's gonna work! launching an isntance now :) | 17:31 |
| dking_desktop | Thanks for the tips. I might look into that more in just a bit. | 17:31 |
| jopdorp | omg yes! | 17:32 |
| jopdorp | mgoddard, you are the bomb! | 17:32 |
| mgoddard | jopdorp: boom | 17:32 |
| *** evrardjp has quit IRC | 17:34 | |
| *** evrardjp has joined #openstack-kolla | 17:34 | |
| openstackgerrit | James Kirsch proposed openstack/kolla-ansible master: Construct service configuration urls using kolla_internal_fqdn https://review.opendev.org/706627 | 17:42 |
| *** k_mouza has quit IRC | 17:42 | |
| *** dougsz has quit IRC | 17:56 | |
| *** igordc has quit IRC | 18:02 | |
| *** igordc has joined #openstack-kolla | 18:06 | |
| *** jopdorp has quit IRC | 18:09 | |
| *** tonythomas has quit IRC | 18:13 | |
| *** k_mouza has joined #openstack-kolla | 18:13 | |
| *** k_mouza has quit IRC | 18:18 | |
| mnasiadka | mgoddard, yoctozepto, osmanlicilegi, hrw: workaround for bindep failures (caused by new virtualenv) on centos7 merged, you can recheck your changes if they failed on RETRY_LIMIT | 18:21 |
| *** hjensas has quit IRC | 18:21 | |
| *** umbSublime has quit IRC | 18:23 | |
| *** bersace has quit IRC | 18:24 | |
| *** bersace has joined #openstack-kolla | 18:29 | |
| yoctozepto | mnasiadka: ack, thanks - btw what's the workaround? | 18:30 |
| mnasiadka | yoctozepto: https://review.opendev.org/#/c/707200/ - tl;dr - running chmod in base-jobs :) | 18:30 |
| patchbot | patch 707200 - opendev/base-jobs - Workaround root virtualenv break with v20 (MERGED) - 4 patch sets | 18:30 |
| mnasiadka | yoctozepto: proper fix is merged but all images need to be rebuilt | 18:31 |
| yoctozepto | mnasiadka: rotfl | 18:32 |
| mnasiadka | yoctozepto: well, it is what it is :D | 18:33 |
| mnasiadka | yoctozepto: virtualenv 20.0 made my day in a lot open source projects :) | 18:33 |
| yoctozepto | mnasiadka: glad I had other assignments :-) | 18:33 |
| hrw | mnasiadka: yay... | 18:34 |
| hrw | queued anyway | 18:34 |
| *** hjensas has joined #openstack-kolla | 18:44 | |
| *** imcsk8 has joined #openstack-kolla | 18:44 | |
| openstackgerrit | James Kirsch proposed openstack/kolla-ansible master: Construct service REST API urls using configured protocol https://review.opendev.org/707233 | 18:52 |
| *** gfidente is now known as gfidente|afk | 18:56 | |
| mnasiadka | hrw: linaro london mirror in some weird state? https://zuul.opendev.org/t/openstack/build/7f40c50c9572418ba55742bfed01d941/log/job-output.txt#989 | 19:01 |
| *** pbing19 has quit IRC | 19:23 | |
| openstackgerrit | Merged openstack/kayobe master: [ussuri][goal] Drop python 2.7 support and testing https://review.opendev.org/695617 | 19:56 |
| *** tony31 has quit IRC | 20:02 | |
| openstackgerrit | Merged openstack/kolla stable/stein: [stein] Bump versions https://review.opendev.org/706660 | 20:03 |
| *** Wasaac has quit IRC | 20:15 | |
| *** Wasaac_ has joined #openstack-kolla | 20:15 | |
| *** Wasaac_ is now known as Wasaac | 20:16 | |
| *** rgogunskiy has joined #openstack-kolla | 20:33 | |
| *** rgogunskiy has quit IRC | 20:37 | |
| openstackgerrit | Gaëtan Trellu proposed openstack/kolla-ansible master: Use internal API for masakari-monitor https://review.opendev.org/697712 | 20:38 |
| *** ne0h|nihilist has quit IRC | 20:57 | |
| *** kplant has quit IRC | 21:02 | |
| openstackgerrit | Merged openstack/kolla stable/rocky: Change fetch_fernet_tokens.py behaviour to check for minimum number of tokens https://review.opendev.org/705960 | 21:09 |
| *** Wasaac has quit IRC | 21:11 | |
| *** Wasaac has joined #openstack-kolla | 21:12 | |
| *** noxoid has quit IRC | 21:29 | |
| *** Trevor_V has quit IRC | 21:37 | |
| dking_desktop | What would be the correct channel for openstack cli questions? | 21:52 |
| *** vmixor has quit IRC | 21:59 | |
| goldyfruit | dking_desktop, #openstack-sdks | 22:07 |
| dking_desktop | goldyfruit: Thank you very much. | 22:07 |
| openstackgerrit | Merged openstack/kolla stable/rocky: [rocky] Bump versions https://review.opendev.org/706661 | 22:07 |
| *** umbSublime has joined #openstack-kolla | 22:11 | |
| *** cah_link has quit IRC | 22:32 | |
| sorin-mihai | given that https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/8.0_release_notes/rhel-8_0_0_release#deprecated_functionality at 5.5.15. Security mentions that "libselinux-python is available only through its module", should this be mentioned in the quickstart? | 23:17 |
| sorin-mihai | whithout the fix everyone will get a hard "Error: Unable to find a match: python-devel libselinux-python" | 23:21 |
| sorin-mihai | or, since libselinux-python3 is already installed by default, add a mention to install python3-devel or platform-python-devel instead of python-devel, for centos 8? | 23:28 |
| sorin-mihai | er, platform-python-devel doesn't seem enough, so scratch that, but python3-devel still seems a better option | 23:30 |
| *** reph has joined #openstack-kolla | 23:33 | |
| *** igordc has quit IRC | 23:46 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!