Sunday, 2020-03-08

« Saturday, 2020-03-07 Index Monday, 2020-03-09 »
*** factor has joined #openstack-infra00:01
*** ociuhandu has joined #openstack-infra00:03
*** ociuhandu has quit IRC00:08
*** ahosam has quit IRC00:28
*** jamesmcarthur has quit IRC00:30
*** ijw has quit IRC00:33
*** rcernin has joined #openstack-infra00:38
*** factor has quit IRC00:43
*** eharney has quit IRC00:44
*** dulek has quit IRC00:54
*** jrist has quit IRC00:54
*** raukadah has quit IRC00:54
*** yamamoto has joined #openstack-infra00:54
*** chandankumar has joined #openstack-infra00:56
*** jrist has joined #openstack-infra01:03
*** dulek has joined #openstack-infra01:06
*** rkukura_ has joined #openstack-infra01:06
*** rkukura has quit IRC01:08
*** rkukura_ is now known as rkukura01:08
*** yamamoto has quit IRC01:26
*** Liang__ has joined #openstack-infra01:29
*** Liang__ has quit IRC01:39
*** Liang__ has joined #openstack-infra01:44
*** rkukura has quit IRC01:44
*** rkukura has joined #openstack-infra01:47
*** dannins has joined #openstack-infra02:03
*** matt_kosut has joined #openstack-infra02:40
*** matt_kosut has quit IRC02:45
*** rkukura has quit IRC02:46
*** rkukura has joined #openstack-infra02:51
openstackgerritGhanshyam Mann proposed openstack/project-config master: Add devstack-plugin-nfs project under QA  https://review.opendev.org/71183402:53
openstackgerritGhanshyam Mann proposed openstack/project-config master: Add devstack-plugin-nfs project under QA  https://review.opendev.org/71183402:58
*** apetrich has quit IRC04:00
*** ociuhandu has joined #openstack-infra04:06
*** ociuhandu has quit IRC04:10
*** HenryG has joined #openstack-infra04:31
*** evrardjp has quit IRC05:35
*** evrardjp has joined #openstack-infra05:35
*** Liang__ has quit IRC06:01
*** larainema has joined #openstack-infra06:19
*** yamamoto has joined #openstack-infra06:35
AJaegerinfra-root, is that really the right way- importing instead of rename ^06:44
*** Lucas_Gray has joined #openstack-infra07:20
*** yamamoto has quit IRC07:56
*** yamamoto has joined #openstack-infra07:57
*** Liang__ has joined #openstack-infra08:30
*** Lucas_Gray has quit IRC08:36
*** ahosam has joined #openstack-infra08:57
*** Lucas_Gray has joined #openstack-infra09:05
*** matt_kosut has joined #openstack-infra09:11
*** yamamoto has quit IRC09:24
*** xek_ has joined #openstack-infra09:36
*** ahosam has quit IRC09:38
*** yamamoto has joined #openstack-infra09:54
*** Lucas_Gray has quit IRC10:07
*** Lucas_Gray has joined #openstack-infra10:10
*** yamamoto has quit IRC10:43
*** ociuhandu has joined #openstack-infra11:04
*** ociuhandu has quit IRC11:09
*** yamamoto has joined #openstack-infra11:14
*** yamamoto has quit IRC11:49
*** yamamoto has joined #openstack-infra11:51
*** Lucas_Gray has quit IRC11:55
*** yamamoto has quit IRC12:23
*** yamamoto has joined #openstack-infra12:32
*** ociuhandu has joined #openstack-infra12:33
*** ociuhandu has quit IRC12:43
*** yamamoto has quit IRC12:44
*** ociuhandu has joined #openstack-infra12:45
*** ociuhandu has quit IRC12:50
*** Lucas_Gray has joined #openstack-infra13:07
*** yamamoto has joined #openstack-infra13:15
*** Lucas_Gray has quit IRC13:17
donnydso i have been search for what extra specs I need to put into the multi-numa flavor and have been coming up short - does anyone remember what they were supposed to be13:22
*** xek_ has quit IRC13:26
donnydfungi: I need to reboot the mirror - I forgot to reduce the MTU to 1480 to accommodate for the ipv6 tunnel and I know we had that issue last time with the retransmits13:31
fungidonnyd: that's fine, nothing's moving it yet13:41
donnydwould it be possible for you to login and check the mtu setting was properly changed in the instance?13:41
donnydits rebooting now13:41
donnydshould be set to 148013:47
*** n0on1 has joined #openstack-infra13:50
*** Lucas_Gray has joined #openstack-infra13:53
*** slaweq has joined #openstack-infra13:59
*** tosky has joined #openstack-infra14:05
*** ahosam has joined #openstack-infra14:11
fungi2: ens3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1480 qdisc fq_codel state UP mode DEFAULT group default qlen 100014:13
fungidonnyd: ^ lgtm, thanks!14:13
*** slaweq has quit IRC14:13
donnydawesome14:13
donnydI don't want to hit those retrans issues this time around14:13
fungiindeed14:14
donnydthe only thing I have left to do is setup the multi-numa flavor the same as it was before and I failed to capture it before I shut down the old controlller14:15
donnydI looked through the old logs  - but I can't seem to find where we talked about it14:15
donnydI will have to wait for sean-k-mooney to be around - I think he will remember what it's supposed to be14:16
donnydI also don't know if anyone was actually using the nested virt feature14:17
donnydits not hard to turn back on - just wanted to make sure it was in use14:17
mnasercould someone re-encrypt https://opendev.org/opendev/system-config/src/branch/master/.zuul.yaml#L7-L22 into opendev/lodgeit (or opendev/project-config maybe?)14:19
*** yamamoto has quit IRC14:19
mnaseri'll need that to be able to publish docker images14:19
donnydmnaser: are you doing any of the numa stuff ? do you know if anyone is using nested virt on vexxhost?14:20
*** slaweq has joined #openstack-infra14:20
mnaserdonnyd: we don't do numa because live migration is a pita but nested virt is working ok afaik14:20
donnydthanks :)14:21
donnydI was going to enable it by default - but I wasn't sure if anyone was actually using the feature14:21
mnaseri believe octavia and manila do14:23
donnydthanks mnaser14:24
cgoncalvesoctavia uses nested virt14:24
*** slaweq has quit IRC14:25
*** n0on1 has quit IRC14:30
*** Lucas_Gray has quit IRC14:44
*** ykarel is now known as ykarel|away14:52
*** Lucas_Gray has joined #openstack-infra14:53
donnydfungi: while the instances itself brought up the correct mtu - lower layers were very unhappy when i tried to migrate the instance.15:01
donnydit would seem a full shutdown and start fixed the issue thoguh15:02
fungidonnyd: okay, cool, thanks for the heads up15:06
fungimnaser: i'll give it a go in a minute15:06
mnaserfungi: thanks!15:07
fungihrm looks like https://review.opendev.org/711325 removed the zuul v3 migration guide from the infra manual, and didn't move that content anywhere. openstack may want to copy the old content into the contributor guide since there's an upcoming cycle goal to convert all remaining legacy jobs15:52
fungiunfortunately that was also where we were documenting opendev's policy around administrative handling of zuul secrets. i'll make a note to resurrect those bits and put them somewhere15:54
fungias well as our examples for how to encrypt secrets in opendev15:54
*** ociuhandu has joined #openstack-infra15:58
*** jhesketh has quit IRC16:01
*** jhesketh has joined #openstack-infra16:02
fungimnaser: http://paste.openstack.org/show/79046216:03
*** ociuhandu has quit IRC16:03
mnaserfungi: great, i'll try to follow the same pattern and get some images up16:06
*** ociuhandu has joined #openstack-infra16:06
*** ociuhandu has quit IRC16:12
*** xek_ has joined #openstack-infra16:14
openstackgerritMohammed Naser proposed opendev/lodgeit master: Upload container images  https://review.opendev.org/71185416:16
mnasermordred: ^ fyi, still needs some revision to add something to pick up env variables for sql and whatnot16:18
openstackgerritMerged openstack/project-config master: Start minimally using Open Edge environment  https://review.opendev.org/71183116:18
fungidonnyd: ^ i'll try to make sure nodes are booting and find some build results for jobs which ran there in a little while16:19
fungiand then you can decide what you want to scale max-servers up to16:20
mnaserdang16:21
mnaserlodgeit is running with manage.py runserver16:21
*** slaweq has joined #openstack-infra16:22
mnasermordred: let me know how you feel about having uwsgi inside lodgeit requirements so the docker image runs lodgeit via uwsgi instead of through the built in (i assume) developement server16:22
mnaser..or any other approach, i guess16:23
fungiftr, that sounds fine (even preferable) to me if it works16:32
*** ahosam has quit IRC16:40
*** Lucas_Gray has quit IRC16:51
*** yamamoto has joined #openstack-infra16:53
*** yamamoto has quit IRC16:58
*** yolanda has quit IRC17:06
*** yolanda has joined #openstack-infra17:19
*** prometheanfire has joined #openstack-infra17:23
donnydfungi: it would seem that there is some sort of networking issue17:28
donnydit looks like glean isn't bringing up the networking interface again17:30
fungidonnyd: hrm, i'm about to throw some okonomiyaki on the griddle, but can take a closer look in an hour or so17:31
fungiif nobody beats me to it17:31
donnydyea no worries  - I am working on some test instances - its reproducible17:31
fungiahh, okay. thanks for looking into it!17:32
*** jamesmcarthur has joined #openstack-infra17:33
*** jamesmcarthur has quit IRC17:34
*** jamesmcarthur has joined #openstack-infra17:34
donnydinteresting - we didn't have any issues with the Openstack CI projects ipv6 connectivity - but surely are for the zuul project17:35
donnydI think I know what is wrong17:35
*** evrardjp has quit IRC17:35
*** evrardjp has joined #openstack-infra17:35
*** jamesmcarthur has quit IRC17:44
*** ricolin has joined #openstack-infra17:49
*** armax has quit IRC17:50
*** jamesmcarthur has joined #openstack-infra17:51
*** slaweq has quit IRC18:03
*** ociuhandu has joined #openstack-infra18:20
*** ociuhandu has quit IRC18:26
*** jamesmcarthur has quit IRC18:29
*** jamesmcarthur has joined #openstack-infra18:31
*** jamesmcarthur has quit IRC18:36
*** jamesmcarthur has joined #openstack-infra18:41
*** jamesmcarthur has quit IRC18:53
*** ociuhandu has joined #openstack-infra18:54
*** armax has joined #openstack-infra18:57
*** ociuhandu has quit IRC18:59
*** armax has quit IRC19:05
*** ralonsoh has joined #openstack-infra19:07
*** dave-mccowan has joined #openstack-infra19:16
*** armax has joined #openstack-infra19:20
*** dave-mccowan has quit IRC19:31
*** aedc_ has quit IRC19:40
*** jamesmcarthur has joined #openstack-infra19:45
donnydok - fixed the networking issue19:47
fungiawesome19:47
donnydand this is as random as it gets - but my switches require vlans to be configured for trunk ports (grrr dell) - but the one vlan neutron picked for the zuul network for some reason did not get setup properly19:48
donnydall should be good now19:48
donnydI will watch a few more test nodes go by though before I call it good19:49
donnydhttps://usercontent.irccloud-cdn.com/file/35mZbnCI/image.png19:49
donnydbut here is the test node and its reachable from outside my network19:50
*** aedc has joined #openstack-infra19:53
*** jamesmcarthur has quit IRC20:05
*** jamesmcarthur has joined #openstack-infra20:05
fungivlans configured so you can set what will be the native vlan, i guess?20:07
fungier, default vlan for untagged frames on the trunk i mean20:07
*** jamesmcarthur has quit IRC20:09
*** jamesmcarthur has joined #openstack-infra20:10
*** jamesmcarthur has quit IRC20:15
openstackgerritMohammed Naser proposed opendev/lodgeit master: Upload container images  https://review.opendev.org/71185420:28
mordredmnaser: ^^ one gotcha in that patch20:37
mordredmnaser: also, I agree with fungi - uwsgi sounds great20:37
mnasermordred: urgh just changed laptops and my vim setup is still not cleaned up20:37
mordredmnaser: I feel you :)20:38
mnasermordred: ok ill iterate more and make it so docker run will get it running inside uwsgi20:38
mordredcool20:38
mordredfwiw - we've been doing our container images just expecting to bind-mount in a config file somewhere ... largely because we don't have a "good" way to pass secrets in env vars sicne we're not in k8s ... I'm guessing the env vars would be better for you, so when you work on that it would be great if we could still bind-mount a config file20:39
*** jamesmcarthur has joined #openstack-infra20:40
mnasermordred: hmm ok i havent thought about that, but i'll try to think on how we can do that, im not sure if lodgeit even uses a config file20:41
mordredmnaser: oh jeez. it seems like we're currently templating out a manage.py *WHAT*20:41
mnaseryep.20:41
mnaserlol20:41
mordredhttps://opendev.org/opendev/puppet-lodgeit/src/branch/master/templates/manage.py.erb20:41
mordredso - clearly we'll need a new $something too :)20:42
mnaseryeah i mean i def did something to read env variables but i think a config file might add a bit more complexity20:42
mnaserand with uwsgi in place it kinda ignores manage.py20:43
openstackgerritMohammed Naser proposed opendev/lodgeit master: Upload container images  https://review.opendev.org/71185420:43
mnaserthis is what im thinking so far.. i'm still needing to write the actual uwsgi file20:43
mordredwell - I think we can extend what you've got there to also read a file20:44
mordredwhich just happens to be easier for us20:45
mordredbut don't worry about it too much - if the thing you've got there works for env vars - it's an easy followup for me to do whatever we need there too20:45
*** jamesmcarthur has quit IRC20:47
openstackgerritMohammed Naser proposed opendev/lodgeit master: Upload container images  https://review.opendev.org/71185420:48
mnaserok i need to figure out of a clean way to get docker to run on a mac to test that locally20:48
mordredmnaser: docker-for-mac works great20:49
mordredor docker desktop or whatever it's called20:49
mnasermordred: im considering dogfooding https://docs.docker.com/machine/drivers/openstack/ :)20:49
mordredwow20:50
mnaserlets seee how well it works20:50
*** dave-mccowan has joined #openstack-infra21:03
*** ralonsoh has quit IRC21:09
mnasersigh, it's trying to contact v2.0 api for some reason and getting a 404 obviously21:10
mnaserhttps://github.com/docker/machine/blob/master/drivers/openstack/openstack.go#L653-L67221:13
mnaserwelp21:13
*** ociuhandu has joined #openstack-infra21:14
mnaserthe only place tenant id is used is when listing floating ips...21:15
fungii bet that worked great once upon a time21:15
mnaserwhich means someone has been naughty and tried to run this as an admin21:15
fungioh, yep21:15
mnaserand was like "oh wait i should find a way to filter my own floating ips only"21:15
mnaserpretty sure tenant id is provided when authenticating..21:16
mnaserlet me try to put the tenant id in there and see if the rest works21:16
*** dave-mccowan has quit IRC21:18
mnaserlooks like this is the combo that worked (in combination of OS_TENANT_ID instead of OS_TENANT_NAME): `docker-machine -D create --driver openstack --openstack-flavor-name v2-highcpu-8 --openstack-image-name "Debian GNU/Linux 10 (buster) [2019-12-02]" --openstack-net-name public --openstack-ssh-user debian vm`21:21
mnaserit's installing docker now :>21:21
mnaserit's working, wewt21:21
*** jamesmcarthur has joined #openstack-infra21:23
mnaseroh man, docker builds are so nice and quick because they happen remotely21:23
*** matt_kosut has quit IRC21:25
mnaserpoop.  it (obviously) didn't do boot from volume so ran out of disk space :-)21:25
*** jamesmcarthur has quit IRC21:27
*** rcernin has quit IRC21:30
fungihow'd you get past it assuming keystone v2 api?21:32
mnaserfungi: by using OS_TENANT_ID inside the envvars instead of OS_TENANT_NAME so it doesn't try and do the lookup21:33
fungiahh, okay21:34
*** xek_ has quit IRC21:36
mnasermordred: do you happen to know if there's a way to use a specific package inside build time only with the container builds inside opendev using bindep.txt ?21:40
mnaseri added gcc to get uwsgi to build, but it's being installed in the final image too which is useless21:40
mnasernvm i think it already does it21:41
fungiyeah, i think you have to set the profile for it to "build"?21:43
*** ociuhandu has quit IRC21:44
mnaserhttp://199.204.45.163:3276921:44
mnaserthat's using `docker run -it -e LODGEIT_DBURI=sqlite:////tmp/lodgeit.db -e LODGEIT_SECRET_KEY=foo --rm -P opendev/lodgeit` built locally :)21:45
fungislick!21:46
*** rkukura has quit IRC21:46
mnaseri'll clean this up a little bit more (uwsgi complanining that it's running as root) and some other warnings and the image should be ready21:46
*** jamesmcarthur has joined #openstack-infra21:56
openstackgerritMohammed Naser proposed openstack/project-config master: Add vexxhost/lodgeit-helm  https://review.opendev.org/71186122:00
*** jamesmcarthur has quit IRC22:01
openstackgerritMohammed Naser proposed opendev/lodgeit master: Upload container images  https://review.opendev.org/71185422:02
clarkbmordred: we actually can set env vars. its how weuse the mariadb image22:02
mnaservoila.  that's a working image.  all you need is: `docker run -it -P opendev/lodgeit` :)22:03
clarkbfungi: sorry about the v3 migration change. I can look at it tomorrow to pull our the still relevant bits22:03
mnaser(you can specify envvars if you need, but it defaults to sqlite3)22:03
fungiclarkb: i can take care of it, just trying to decide where the best place is to put it22:03
fungileaning toward adding a section in doc/source/testing.rst22:04
fungifor the bits related to managing zuul secrets anyway22:04
*** dannins has quit IRC22:06
*** imacdonn has quit IRC22:07
*** imacdonn has joined #openstack-infra22:07
mnaserah gr22:08
mnaseri think i might need to add PyMySQL to requirements.txxt22:09
mnaserbut is sqlalchemy from 2014 compatible with it, who knows :)22:10
mnaserit seems like lodgeit python deps are installing via system packages, wonder what version of sqlalchemy is installed there22:11
mordredclarkb: we can set env vars - but can we set secrets in env vars safely?22:12
clarkbmordred: we do for maroadb22:13
mordredgreat22:13
clarkbits how giteaworks22:13
mordrednod. cool then22:13
mordredmnaser: 1.0.11+ds1-1ubuntu222:13
mordredmnaser: is the sqlalchemy on paste.o.o22:14
mnasermordred: ok so the pin on 0.9.3 in requirements.txt means that we're probably ok to jump to 1.x series22:14
mordredbut - maybe it'll work with normal sqlalchey and pymysql22:14
mnaseri'll just uncap and we'll see what happens :p22:14
mordred:)22:14
mordredthere's not MUCH databasing going on22:14
mnaseryeah i cant imagine it's using such complex tooling22:15
openstackgerritMohammed Naser proposed opendev/lodgeit master: Upload container images  https://review.opendev.org/71185422:15
mnaserlets see how that works22:15
* mnaser goes to get food22:15
*** tkajinam has joined #openstack-infra22:50
*** yamamoto has joined #openstack-infra22:56
*** yamamoto has quit IRC23:00
*** jamesmcarthur has joined #openstack-infra23:05
*** jamesmcarthur has quit IRC23:07
*** jamesmcarthur_ has joined #openstack-infra23:07
*** dchen has joined #openstack-infra23:15
*** Goneri has joined #openstack-infra23:21
fungiin fact, lodgeit's schema is about as simple as it gets23:21
*** jamesmcarthur_ has quit IRC23:27
*** rcernin has joined #openstack-infra23:27
*** jamesmcarthur has joined #openstack-infra23:29
*** yamamoto has joined #openstack-infra23:36
*** tosky has quit IRC23:40
*** Goneri has quit IRC23:48
gmannAJaeger: I was in impression that moving x namespace repo back to openstack via import and no retirement or rename etc.  - https://review.opendev.org/#/c/711834/223:53
gmannrename process does not applies here i think which is for openstack projects rename23:54
« Saturday, 2020-03-07 Index Monday, 2020-03-09 »

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!