| *** tideline has joined #openstack-ansible | 00:00 | |
| *** tideline has quit IRC | 00:01 | |
| *** hamzy_ has joined #openstack-ansible | 00:02 | |
| *** hamzy has joined #openstack-ansible | 00:08 | |
| cloudnull | osnaya: the container ips are static and stored in inventory for eth1 + | 00:09 |
|---|---|---|
| cloudnull | the inventory generation script takes care of all of that. | 00:09 |
| cloudnull | for eth0 the ip is from dhcp, which is provided by dnsmasq | 00:10 |
| *** hamzy_ has quit IRC | 00:10 | |
| cloudnull | d34dh0r53: if its in requirements it should be built | 00:11 |
| cloudnull | and the version will be listed in the absolute requirement file | 00:12 |
| d34dh0r53 | cloudnull: yeah, it's not and I can't figure out why, it's in requirements, but not absolute_requirements | 00:12 |
| d34dh0r53 | cryptography btw | 00:12 |
| cloudnull | there's no wheel for it at all ? | 00:13 |
| cloudnull | also mind having a look at https://review.openstack.org/#/c/560749? | 00:13 |
| *** AR2018 has quit IRC | 00:13 | |
| *** vnogin has joined #openstack-ansible | 00:17 | |
| *** vnogin has quit IRC | 00:21 | |
| osnaya | I was hitting issue yesterday on Install pip packages.... After @jra suggested, did Uprev from OSA 16.0.8 to 16.0.10 and that problem is resolved. But then I hit the issue that @jra was hitting after upRev which is infra1_nova_api_placement and few other containers not starting... @http://paste.openstack.org/show/719112/ -----Any pointers? | 00:24 |
| cloudnull | osnaya: does /var/log/lxc/lxc-infra1_nova_api_placement_container-11e9ce02.log have anything interesting in it ? | 00:26 |
| cloudnull | from the output its hard to know why the container wont start | 00:26 |
| cloudnull | going to eat, but will be back later. | 00:27 |
| osnaya | @cloudnull will this be on the target host? | 00:27 |
| osnaya | @going thru the log on target... will let u know findings | 00:30 |
| d34dh0r53 | cloudnull: no wheel at all | 00:31 |
| d34dh0r53 | it's the SHA bump review that I'm working on | 00:31 |
| *** hybridpollo has joined #openstack-ansible | 00:32 | |
| d34dh0r53 | https://review.openstack.org/#/c/558082/ btw | 00:33 |
| d34dh0r53 | I like those ELK roles, I think I'm going to hijack them :) | 00:33 |
| osnaya | @cloudnull @odyssey4m4 /var/log/lxc/infra1_nova_api_placement_container..... shows up ERRORs on execution script /var/lib/lxc/....container.../autodev, config section lxc - Script exited with status 127 (failed to run autodev hooks-> failed to setup container"... | 00:36 |
| *** hybridpollo has quit IRC | 00:45 | |
| *** hvvclint has joined #openstack-ansible | 00:45 | |
| *** AR2018 has joined #openstack-ansible | 00:47 | |
| *** osnaya has quit IRC | 00:50 | |
| *** osnaya has joined #openstack-ansible | 00:57 | |
| osnaya | @cloudnull got to go.... will be back online in couple of hours... | 00:59 |
| openstackgerrit | Merged openstack/openstack-ansible stable/queens: Add container/service clean up for nova/cinder/heat https://review.openstack.org/559721 | 00:59 |
| osnaya | @cloudnull thank you for the pointer... I will look thru /var/log/lxc msg on other container failures as well and check if the issue is same on those failures | 00:59 |
| *** osnaya has quit IRC | 01:00 | |
| *** ianychoi_ is now known as ianychoi | 01:01 | |
| *** radeks has quit IRC | 01:05 | |
| openstackgerrit | Merged openstack/openstack-ansible-ops master: Update README, beat deployment, and configs https://review.openstack.org/560749 | 01:13 |
| openstackgerrit | Merged openstack/openstack-ansible-ops master: Add filebeat support https://review.openstack.org/560944 | 01:13 |
| cloudnull | d34dh0r53: thanks! | 01:15 |
| cloudnull | I adapted the filebeat things from your original role . | 01:16 |
| cloudnull | just updated for the 6.x syntax | 01:16 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Update docs and add inventory example https://review.openstack.org/561092 | 01:18 |
| logan- | now just make it install only on teh host and use all the journal linking :D | 01:21 |
| d34dh0r53 | cloudnull: yeah, I think I can just drop in the logstash stuff and be good | 01:22 |
| cloudnull | logan-: ++ | 01:33 |
| cloudnull | Im testing the journalbeat. | 01:33 |
| cloudnull | if that works then it'd be marvelous | 01:33 |
| cloudnull | also APM monitoring | 01:33 |
| cloudnull | i did packetbeat that way though. | 01:34 |
| cloudnull | its all installed on the host but if an app container is present on the host packetbeat will adjust the config as needed. | 01:35 |
| cloudnull | same with auditbeat for that matter too | 01:36 |
| cloudnull | metricbeat is everywhere but that was kinda unavoidable given i wanted to consume metrics from the containers | 01:37 |
| cloudnull | i have kibana with some nice dashboards, but im considering having grafana for better graphs and graphing that wont break on major version changes. | 01:40 |
| *** pylot has quit IRC | 01:53 | |
| *** pylot has joined #openstack-ansible | 01:59 | |
| logan- | ahh neat cloudnull. is the container name/hostname embedded in the journals? i was looking at the docs on how to query them and it seems a little tough having to use machine ids | 02:00 |
| cloudnull | it is | 02:00 |
| cloudnull | you can filter based on the hostname | 02:00 |
| logan- | ahh cool | 02:01 |
| cloudnull | with https://review.openstack.org/#/c/555982/ enabled you can read all the journals from the whole cloud on one host | 02:01 |
| logan- | awesome | 02:02 |
| *** tomtom001 has quit IRC | 02:03 | |
| *** tomtom001 has joined #openstack-ansible | 02:03 | |
| logan- | so reading the review if i specify log_hosts it shouldnt do syslog shipping, only jouranls | 02:03 |
| logan- | journals* | 02:03 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible master: Convert rsyslog to an include_task with group_vars https://review.openstack.org/556002 | 02:04 |
| cloudnull | it will do both right now | 02:04 |
| logan- | ahh i see ^ now lol | 02:05 |
| cloudnull | but with that pr, which i just rebased, syslog shipping will be optional | 02:05 |
| logan- | great | 02:05 |
| cloudnull | which will further lower IO | 02:05 |
| *** gaoyan has joined #openstack-ansible | 02:13 | |
| *** armaan has quit IRC | 02:25 | |
| *** armaan has joined #openstack-ansible | 02:26 | |
| *** AR2018 has quit IRC | 02:27 | |
| *** nicolasbock has quit IRC | 02:47 | |
| *** dave-mccowan has quit IRC | 03:05 | |
| *** hvvclint has quit IRC | 03:34 | |
| *** phre4kad has joined #openstack-ansible | 03:41 | |
| *** phre4kad has quit IRC | 03:43 | |
| *** gaoyan has quit IRC | 03:48 | |
| *** gaoyan has joined #openstack-ansible | 03:51 | |
| *** fbcraysiii has joined #openstack-ansible | 03:57 | |
| *** fbcraysiii has quit IRC | 03:59 | |
| *** udesale has joined #openstack-ansible | 04:01 | |
| *** poopcat has quit IRC | 04:07 | |
| *** jonher has joined #openstack-ansible | 04:19 | |
| *** jonher_ has quit IRC | 04:21 | |
| *** lbragstad has joined #openstack-ansible | 04:26 | |
| *** lbragstad has quit IRC | 04:28 | |
| *** gaoyan has quit IRC | 04:41 | |
| *** yifei has quit IRC | 05:17 | |
| *** vnogin has joined #openstack-ansible | 05:17 | |
| *** yifei has joined #openstack-ansible | 05:17 | |
| *** vnogin has quit IRC | 05:22 | |
| *** cjloader has joined #openstack-ansible | 05:27 | |
| *** cjloader has quit IRC | 05:32 | |
| *** gaoyan has joined #openstack-ansible | 05:39 | |
| odyssey4me | d34dh0r53 prometheanfire cryptography is deliberately built from source every time, see https://github.com/openstack/openstack-ansible/blob/d057e9522f37fbbe446aa79cbff5f6c23150d944/inventory/group_vars/repo_all.yml#L46-L60 | 05:41 |
| odyssey4me | jrosser does https://review.openstack.org/559750 apply to stable/queens? if so, could you backport that patch? | 05:43 |
| openstackgerrit | Taseer Ahmed proposed openstack/openstack-ansible master: Improve Congress scenario testing https://review.openstack.org/560839 | 05:43 |
| odyssey4me | evrardjp hwoarang could we get some eyes on https://review.openstack.org/#/q/I846f78c80e4b5eb8421dbdebf3fc943be0bc84df please as they're quite important to get merged asap, given the current builds are breaking things | 05:44 |
| *** yifei has quit IRC | 05:45 | |
| *** yifei has joined #openstack-ansible | 05:46 | |
| *** radeks has joined #openstack-ansible | 05:49 | |
| *** fdsa_ has joined #openstack-ansible | 05:51 | |
| d34dh0r53 | odyssey4me: any idea why it wouldn't be building? | 05:52 |
| odyssey4me | d34dh0r53 it does build, but it does so from source every time on purpose | 05:52 |
| odyssey4me | The explanation as to why is included in the link I posted | 05:53 |
| d34dh0r53 | so where would it be located? | 05:53 |
| d34dh0r53 | yeah, I got that but the deploy is failing because it can't find cryptography | 05:53 |
| *** fdsa has quit IRC | 05:54 | |
| odyssey4me | hmm, IIRC it should still build on the repo host and disseminate from there | 05:54 |
| d34dh0r53 | yeah, that's what I figured | 05:54 |
| odyssey4me | have you got some logs or some sort of analysis to look at? | 05:54 |
| odyssey4me | etherpad or whatever? | 05:54 |
| prometheanfire | odyssey4me: hi? why was I pinged for that? | 05:54 |
| odyssey4me | prometheanfire you were discussing it at the same time | 05:55 |
| * prometheanfire must have forgotten | 05:55 | |
| d34dh0r53 | odyssey4me: root@162.209.96.241, your key is on there | 05:55 |
| d34dh0r53 | it's had gate-check-commit.sh aio_metal deploy run on it | 05:56 |
| odyssey4me | as to the purpose of each of the requirements/constraints files - take a look at the repo build role | 05:56 |
| d34dh0r53 | yeah, I read the comments, makes sense | 05:56 |
| odyssey4me | d34dh0r53 is there a tmux session or something with the error in it? | 05:56 |
| d34dh0r53 | I just can't figure out why cryptography isn't building :) | 05:56 |
| d34dh0r53 | I just started tmux, I'll recreate the error real quick | 05:57 |
| d34dh0r53 | it's metal so no containers | 05:57 |
| odyssey4me | ok, watching it now | 05:57 |
| odyssey4me | hmm, so this is specifically an on-metal build - alright, good to know | 05:58 |
| odyssey4me | in which playbook does it fail | 05:58 |
| odyssey4me | ah, during the venv build | 05:58 |
| *** mpjetta has quit IRC | 05:59 | |
| *** mpjetta has joined #openstack-ansible | 06:00 | |
| d34dh0r53 | odyssey4me: btw, this is cherrypicked https://review.openstack.org/#/c/558082/ on that host | 06:01 |
| *** radeks has quit IRC | 06:02 | |
| d34dh0r53 | that horizon error bothers me as well, but I figured I'd tackle the cryptography error first | 06:03 |
| odyssey4me | the horizon error looks like an incomplete git repo | 06:04 |
| d34dh0r53 | yeah, it looks like a good sha | 06:05 |
| odyssey4me | and this issue you're seeing is not really unique to the sha bump or the onmetal, I've seen similar issues with https://review.openstack.org/549012 and have been wanting to figure it out | 06:06 |
| d34dh0r53 | ahh | 06:08 |
| odyssey4me | hmm, same symptom - it downloads the package, but doesn't build a wheel | 06:08 |
| odyssey4me | same for python-libvirt | 06:08 |
| odyssey4me | or libvirt-python rather | 06:08 |
| d34dh0r53 | Is there a log of the actual build? | 06:08 |
| odyssey4me | d34dh0r53 aren't you looking at the same tmux session? | 06:09 |
| d34dh0r53 | yeah, but I mean the gcc or whatever output | 06:09 |
| odyssey4me | this is the output from pip, that's about as good as we get | 06:09 |
| d34dh0r53 | hmm, ok | 06:10 |
| odyssey4me | so it would appear that the no-binary is preventing a wheel being built, which is quite odd - although, lemme validate some more things first | 06:11 |
| *** armaan has quit IRC | 06:14 | |
| *** armaan has joined #openstack-ansible | 06:14 | |
| *** evin has joined #openstack-ansible | 06:18 | |
| odyssey4me | hmm, interesting - configobj also isn't built into a wheel | 06:18 |
| odyssey4me | gonna extract the exact command and re-run via cli and try to get extra verbosity | 06:22 |
| d34dh0r53 | cool | 06:22 |
| odyssey4me | bother, not enough buffer history | 06:22 |
| odyssey4me | lemme kill this tmux session and create a new one with more buffer | 06:23 |
| d34dh0r53 | cool | 06:23 |
| odyssey4me | actually, we can get it from a gate job | 06:24 |
| odyssey4me | ara should give it to us, I think | 06:24 |
| d34dh0r53 | oh, yeah | 06:24 |
| odyssey4me | there we go: http://logs.openstack.org/82/558082/2/check/openstack-ansible-deploy-aio_metal-ubuntu-xenial/f0fa7a6/logs/ara-report/result/8b144e42-a85c-49ce-981e-0cbc3d98a946/ | 06:25 |
| *** maybebuggy has joined #openstack-ansible | 06:27 | |
| d34dh0r53 | :/ that's not what I was expecting | 06:28 |
| odyssey4me | that's infuriating | 06:28 |
| odyssey4me | good news is that the log output is the same, so that's not an indication of failure | 06:29 |
| odyssey4me | it seems more likely that the move of the stuff from the temp folder might be at fault here | 06:29 |
| odyssey4me | lemme get more coffee, brb | 06:30 |
| *** mma has joined #openstack-ansible | 06:30 | |
| d34dh0r53 | no worries | 06:31 |
| odyssey4me | for master, this will all go away soon - but for queens this is important | 06:32 |
| odyssey4me | weird - built via cli and it works | 06:34 |
| odyssey4me | done via requirements file by ansible and it doesn't | 06:35 |
| d34dh0r53 | this is bizarre man | 06:36 |
| odyssey4me | I wonder if the wheel build somehow needs to be told it's running headless | 06:36 |
| odyssey4me | maybe something in the script requires a session, or needs to be told it doesn't have one | 06:37 |
| d34dh0r53 | hmm, possibly | 06:37 |
| evrardjp | hey | 06:37 |
| d34dh0r53 | nothing in the docs for pip wheel | 06:38 |
| d34dh0r53 | hey evrardjp | 06:38 |
| evrardjp | could you do a summary of this conversation? | 06:38 |
| evrardjp | Because I wanted to release today, and I guess you're talking about the sha bump issues | 06:38 |
| d34dh0r53 | yeah so the repo_pip_no_binary wheels aren't building for some reason | 06:39 |
| odyssey4me | evrardjp this is only a master-related thing, as far as I know | 06:39 |
| odyssey4me | d34dh0r53 not just those - also configobj | 06:39 |
| *** shardy has joined #openstack-ansible | 06:39 | |
| d34dh0r53 | right | 06:39 |
| d34dh0r53 | sorry, late for me :) | 06:39 |
| odyssey4me | but yes, for some reason when building cryptography, configobj and libvirt-python using pip wheel via ansible, it just doesn't build the wheel - whereas with the CLI it does | 06:40 |
| odyssey4me | that's as far as we have it right now | 06:40 |
| evrardjp | where did you observe this? | 06:41 |
| evrardjp | I ran a build of master without the sha bump and it worked fine yesterday | 06:41 |
| evrardjp | so it's really the sha bump itself | 06:41 |
| d34dh0r53 | odyssey4me: dash, not underscore in libvirt-python | 06:41 |
| odyssey4me | d34dh0r53 pip sees them as the same | 06:41 |
| d34dh0r53 | oh, didn't know that | 06:41 |
| d34dh0r53 | pip is black magic to me | 06:42 |
| odyssey4me | d34dh0r53 learn through trial-by-fire :) | 06:42 |
| d34dh0r53 | hah | 06:42 |
| evrardjp | packages names can't contain - iirc | 06:42 |
| evrardjp | in the install process, not the package name itself | 06:42 |
| odyssey4me | evrardjp ok, can look to see if there's a difference that matters there | 06:42 |
| odyssey4me | perhaps build another host to compare with | 06:43 |
| odyssey4me | lemme check that sha bump again | 06:43 |
| evrardjp | odyssey4me: before leaving yesterday, I did a deploy with no sha bump | 06:43 |
| evrardjp | then added the sha bump on top | 06:43 |
| odyssey4me | do you have that handy for me to inspect? | 06:43 |
| evrardjp | and rebuild repo and the rest | 06:43 |
| evrardjp | and it worked. | 06:43 |
| odyssey4me | it seems that it should be bed time for d34dh0r53 now | 06:43 |
| evrardjp | odyssey4me: yes I have | 06:43 |
| *** evin has quit IRC | 06:43 | |
| evrardjp | I can give you access to this "upgraded" thing | 06:43 |
| d34dh0r53 | odyssey4me: getting close, I got a nap on the couch a little earlier | 06:44 |
| odyssey4me | hmm, I see the sha bump includes pip and setuptools changes | 06:44 |
| evrardjp | odyssey4me: I tried without the pip and setuptools | 06:45 |
| evrardjp | no difference | 06:45 |
| evrardjp | but the bump tool didn't bump to a b1 version | 06:45 |
| evrardjp | it wasn't showing latest package, so I might have to fix my bump tool | 06:45 |
| evrardjp | or what's in master for the same matter | 06:46 |
| evrardjp | (maybe the pypi interface change with the latest website etc.) | 06:46 |
| odyssey4me | I suspect we may have a red herring here - the build doesn't do cryptography because it fails to build horizon: http://logs.openstack.org/82/558082/2/check/openstack-ansible-deploy-aio_metal-ubuntu-xenial/f0fa7a6/logs/host/repo/wheel_build.log | 06:46 |
| d34dh0r53 | ahh, that could make sense because it looks like there is absolutely nothing wrong with the build | 06:47 |
| odyssey4me | yeah, so it does all the downloads first - then processes the builds that need to be done from source (ie it doesn't have a wheel for it) | 06:47 |
| odyssey4me | and it's failing on the horizon one, so it never gets to cryptography | 06:48 |
| odyssey4me | curious, there's no horizon in the openstackgit folder | 06:49 |
| evrardjp | odyssey4me: could we stop cloning from local to local? | 06:50 |
| odyssey4me | oh of course - does the on metal have horizon configured on it | 06:50 |
| evrardjp | replacing git+file:/// with / | 06:50 |
| evrardjp | odyssey4me: nope | 06:50 |
| odyssey4me | well then, so there's the reason there isn't one | 06:51 |
| evrardjp | yup | 06:51 |
| odyssey4me | ok - so something else must be requiring horizon, but it's not there | 06:51 |
| odyssey4me | ok, this makes *much* more sense | 06:51 |
| evrardjp | yes indeed | 06:51 |
| d34dh0r53 | yeah | 06:51 |
| evrardjp | I wasn't aware of a problem with horizon | 06:51 |
| odyssey4me | heat-dashboard, possibly | 06:51 |
| evrardjp | I just saw glance and config obj | 06:51 |
| evrardjp | odyssey4me: that could make sense, but heat-dashboard is only listed in horizon | 06:52 |
| evrardjp | doesn't it | 06:52 |
| evrardjp | ? | 06:52 |
| evrardjp | yup | 06:52 |
| evrardjp | and the repo_build test | 06:52 |
| *** evin has joined #openstack-ansible | 06:53 | |
| evrardjp | we need to dig into the requirements themselves | 06:53 |
| odyssey4me | hmm, it's not in https://github.com/openstack/openstack-ansible/blob/master/playbooks/defaults/repo_packages/openstack_services.yml | 06:53 |
| *** DanyC_ has quit IRC | 06:53 | |
| evrardjp | http://codesearch.openstack.org/?q=heat-dashboard&i=nope&files=requirements.txt&repos= | 06:53 |
| evrardjp | that's a good point | 06:54 |
| evrardjp | that's the issue | 06:54 |
| evrardjp | but still | 06:54 |
| evrardjp | how the hell is that supposed to happen? | 06:54 |
| evrardjp | and what about cryptography? | 06:54 |
| odyssey4me | yeah, it's only in the horizon role | 06:54 |
| evrardjp | odyssey4me: yeah, and no other packages need it | 06:55 |
| odyssey4me | but the reason it's being pulled in is because it has no group to be scoped to | 06:55 |
| evrardjp | haha | 06:55 |
| evrardjp | I see | 06:55 |
| odyssey4me | right, lemme try out a fix for that | 06:55 |
| evrardjp | I can do it in the sha bump | 06:55 |
| evrardjp | adding it | 06:55 |
| evrardjp | see how it goes | 06:55 |
| evrardjp | and you can work on the repo in the meantime | 06:55 |
| evrardjp | ok? | 06:55 |
| evrardjp | or did you mean something else | 06:56 |
| evrardjp | when was the heat dashboard introduced? A long time ago I think | 06:56 |
| evrardjp | let me check | 06:56 |
| evrardjp | maybe a few weeks | 06:56 |
| odyssey4me | gimme a chance to verify that this solves the proble | 06:57 |
| evrardjp | jan 26 | 06:57 |
| evrardjp | so i thought of something | 06:58 |
| evrardjp | odyssey4me: can't we already remove the defaults/... thing ? | 06:58 |
| evrardjp | it's only used by the lookup for which we can say the location is in /etc/roles/ | 06:59 |
| evrardjp | /etc/ansible/roles/ | 06:59 |
| evrardjp | (it's the morning for me) | 06:59 |
| odyssey4me | no we can't, because then role tests wouldn't work | 06:59 |
| d34dh0r53 | night fellas | 06:59 |
| odyssey4me | also, we're moving all those settings into the roles and out of the integrated build | 07:00 |
| evrardjp | oh we already do | 07:00 |
| evrardjp | d34dh0r53: gn | 07:00 |
| evrardjp | odyssey4me: that's my point | 07:00 |
| evrardjp | give me a sec | 07:00 |
| odyssey4me | the lookup uses the defaults, then uses the repo_packages on top of that | 07:00 |
| *** armaan has quit IRC | 07:01 | |
| *** armaan has joined #openstack-ansible | 07:02 | |
| evrardjp | odyssey4me: the roles have the same variables | 07:02 |
| evrardjp | example: horizon has the horizon_dashboard variable | 07:03 |
| evrardjp | you see what I mean? | 07:03 |
| *** tomtom001 has quit IRC | 07:03 | |
| odyssey4me | evrardjp no, I'm not sure I'm understanding your point | 07:04 |
| *** tomtom001 has joined #openstack-ansible | 07:04 | |
| odyssey4me | regardless of the existing structure, which will be changed - we still have to work with it for stable branches | 07:04 |
| odyssey4me | for now, though, until it changes, we work with what's there | 07:04 |
| evrardjp | yeah, so here I propose we simply add the heat dashboard into defaults/ for master and queens | 07:04 |
| evrardjp | i will do that in a separate commit | 07:05 |
| odyssey4me | yes, although that horizon patch was not ported to queens | 07:05 |
| odyssey4me | if you do, make sure the integrated build patch lands first | 07:05 |
| evrardjp | queens is deploying fine | 07:06 |
| *** mwe11 has quit IRC | 07:06 | |
| evrardjp | so step 1; introduce the heat dashboard into sha bump | 07:06 |
| evrardjp | step 2: talk english | 07:06 |
| odyssey4me | yes, because the horizon patch to add the heat dashboard is not in queens | 07:06 |
| evrardjp | step 3: explain we can get rid of defaults | 07:06 |
| evrardjp | as of today | 07:07 |
| evrardjp | no fancy things involved | 07:07 |
| *** mwe1 has joined #openstack-ansible | 07:07 | |
| odyssey4me | oh actually, it seems that it is: https://github.com/openstack/openstack-ansible-os_horizon/commit/53113aa25de768c6f5be36b9be0c6b60187a04b1 | 07:07 |
| evrardjp | I said the 26 jan | 07:07 |
| evrardjp | :p | 07:07 |
| evrardjp | do the math :) | 07:08 |
| evrardjp | but still | 07:08 |
| evrardjp | anyway | 07:08 |
| evrardjp | we'll do it for both queens and master, just in case | 07:08 |
| odyssey4me | ok, so yeah - need a patch for both queens and master to tie that to a group | 07:08 |
| odyssey4me | my plan was to further the python build simplification work today | 07:09 |
| evrardjp | odyssey4me: I think I need a chat with you | 07:10 |
| evrardjp | I need to clarify the process | 07:10 |
| odyssey4me | the selective build keys off of this var: https://github.com/openstack/openstack-ansible/blob/master/playbooks/defaults/repo_packages/openstack_services.yml#L40 | 07:11 |
| evrardjp | I know | 07:11 |
| odyssey4me | if that var doesn't exist for the git repo in queston, then it assumes it must build it | 07:11 |
| niraj_singh | if i mention the same project repo group for masakari and masakari-monitors is it downloaded as a single project in /openstack/venv or two different project. Just curious because after running repo-build playbook i am not able to see masakari-monitors. I mentioned both in a single group masakari_all | 07:11 |
| evrardjp | I want to know how your'e gonna do that with the selective part | 07:12 |
| evrardjp | I mean with the new way of doings things | 07:12 |
| evrardjp | you need some variable resolution to get there | 07:12 |
| odyssey4me | oh, that won't matter any more because the build will only happen if the playbook has hosts, so the exclusion happens automatically via a normal ansible way | 07:12 |
| odyssey4me | I haven't figured it out in detail yet, but that's the idea | 07:12 |
| evrardjp | odyssey4me: that was my question | 07:12 |
| odyssey4me | if there are no horizon hosts, then the wheel/venv build for horizon just doesn't happen | 07:13 |
| evrardjp | do you plan to use the target host like nova_all / horizon_all | 07:13 |
| evrardjp | and then do the wheel at the first, then copy to a destination "repo" and resync repos | 07:13 |
| odyssey4me | the playbook will target horizon_all, but the build task will delegate to the designated build host | 07:13 |
| evrardjp | with this repos as first indexes | 07:13 |
| evrardjp | ok | 07:14 |
| odyssey4me | we'll tag it so that if you want to build in CI and host somewhere, that's an option | 07:15 |
| evrardjp | OK | 07:15 |
| evrardjp | woops caps lock | 07:15 |
| odyssey4me | although it probably makes better sense to execute a build and harvest the resulting artifacts if the build succeeded | 07:15 |
| evrardjp | generic handler behavior ? | 07:16 |
| odyssey4me | dunno, for now will focus on the JIT thing - once this body of work is done I'll look at the staged thing, and that's where offline install capability becomes a target | 07:16 |
| *** epalper has joined #openstack-ansible | 07:16 | |
| odyssey4me | I don't understand the question. | 07:16 |
| evrardjp | it was a suggestion | 07:18 |
| odyssey4me | hmm, something else is still trying to pull horizon in | 07:18 |
| odyssey4me | trying to figure out what it is | 07:18 |
| evrardjp | oh it's horizon | 07:19 |
| evrardjp | not heat-dashbaord? | 07:19 |
| odyssey4me | something is requiring horizon | 07:19 |
| odyssey4me | heat-dashboard was what I thought it was, but apparently that's not the only one | 07:19 |
| evrardjp | networking-bgpvpn | 07:20 |
| evrardjp | https://github.com/openstack/networking-bgpvpn/blob/master/requirements.txt#L18 | 07:20 |
| evrardjp | is that in our path? | 07:20 |
| evrardjp | https://github.com/openstack/openstack-ansible/blob/master/playbooks/defaults/repo_packages/openstack_services.yml#L159 | 07:21 |
| evrardjp | dammit | 07:21 |
| evrardjp | https://github.com/openstack/networking-bgpvpn/commit/e9a0ea199b47f76f69545e04bdb4db44869c388b#diff-b4ef698db8ca845e5845c4618278f29a | 07:22 |
| evrardjp | we found it | 07:22 |
| evrardjp | Andreas broke us | 07:22 |
| evrardjp | 14 march | 07:22 |
| evrardjp | makes sense | 07:22 |
| evrardjp | ok let me still hack the heat_dashboard thing | 07:23 |
| evrardjp | will do it in the sha bump | 07:23 |
| evrardjp | will bump bgpvpn to the patch before that patch | 07:23 |
| evrardjp | and freeze it under a better solution is found | 07:23 |
| odyssey4me | wow, that's a terrible thing to do - why would a driver require horizon | 07:23 |
| odyssey4me | I'm guessing they may have their dashboard in the same repo? | 07:23 |
| odyssey4me | wow - that's a good find - not sure how you managed to reduce the possibilities down to that one | 07:25 |
| evrardjp | I searched? | 07:25 |
| evrardjp | :p | 07:25 |
| evrardjp | grep is my friend | 07:25 |
| evrardjp | will speak to the neutron guys in question | 07:27 |
| *** radeks has joined #openstack-ansible | 07:27 | |
| odyssey4me | ok, meanwhile, I'm trying using that parent sha | 07:27 |
| odyssey4me | what does concern me is that the wheel build doesn't show that it failed | 07:28 |
| odyssey4me | it moves on to the venv build as if the wheel build completed | 07:28 |
| odyssey4me | so that is a concern here | 07:28 |
| evrardjp | yeah, you can deal with that part :p | 07:28 |
| odyssey4me | thanks d34dh0r53 for taking the time to get all this setup and going - it's helpful to already have a system setup that I can poke at | 07:29 |
| odyssey4me | ok, using the parent sha of that breaking patch is making all the wheels build right | 07:31 |
| odyssey4me | so two patches - one to add the heat dashboard to defaults, the other is just to use a different sha for the bgp thingy | 07:32 |
| odyssey4me | evrardjp want the diff from my working test? | 07:33 |
| evrardjp | well I was writing it but why not :p | 07:35 |
| evrardjp | to compare | 07:35 |
| odyssey4me | evrardjp added the diff to the review | 07:36 |
| odyssey4me | right, afk for a little bit | 07:37 |
| *** fghaas has joined #openstack-ansible | 07:38 | |
| *** mbuil has joined #openstack-ansible | 07:39 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_nova stable/pike: Use first conductor for nova-manage cell_v2 discover_hosts https://review.openstack.org/560920 | 07:40 |
| openstackgerrit | Merged openstack/openstack-ansible-os_nova stable/queens: Use first conductor for nova-manage cell_v2 discover_hosts https://review.openstack.org/560924 | 07:41 |
| odyssey4me | evrardjp we need https://review.openstack.org/#/c/560976/1 to get that other os_nova patch in | 07:42 |
| odyssey4me | are we going to bump the nova role in the integrated build to include those patches with the next release? | 07:43 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible master: Update all SHAs for master https://review.openstack.org/558082 | 07:44 |
| *** fghaas has quit IRC | 07:44 | |
| Taseer | evrardjp: I think there is still some problem with the "hacking" package => http://logs.openstack.org/39/560839/3/check/openstack-ansible-deploy-congress-ubuntu-xenial/fc0c116/job-output.txt.gz#_2018-04-13_06_51_06_066062 ? | 07:45 |
| *** fghaas has joined #openstack-ansible | 07:45 | |
| Taseer | evrardjp: did we reach a solution to this last time it hit us ? | 07:46 |
| evrardjp | odyssey4me: I will issue a release today | 07:50 |
| evrardjp | I want all those big patches in for today :p | 07:51 |
| evrardjp | :D | 07:51 |
| evrardjp | let's see how it goes | 07:51 |
| evrardjp | odyssey4me: the way I said it sounded far funnier in my head. | 07:51 |
| evrardjp | Let me rephrase: | 07:52 |
| evrardjp | I will bump the things, so that it gets included in the next release | 07:52 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible master: Update all SHAs for master https://review.openstack.org/558082 | 07:57 |
| *** electrofelix has joined #openstack-ansible | 07:58 | |
| *** cshen has joined #openstack-ansible | 07:58 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_glance stable/ocata: Update tests a-r-r to use newton-eol tag https://review.openstack.org/561148 | 08:00 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_cinder stable/ocata: Update tests a-r-r to use newton-eol tag https://review.openstack.org/561149 | 08:01 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_neutron stable/ocata: Update tests a-r-r to use newton-eol tag https://review.openstack.org/561150 | 08:01 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_swift stable/ocata: Update tests a-r-r to use newton-eol tag https://review.openstack.org/561151 | 08:02 |
| *** tomtom001 has quit IRC | 08:03 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_keystone stable/ocata: Update tests a-r-r to use newton-eol tag https://review.openstack.org/561152 | 08:04 |
| *** tomtom001 has joined #openstack-ansible | 08:04 | |
| *** gkadam has joined #openstack-ansible | 08:07 | |
| *** armaan has quit IRC | 08:07 | |
| *** armaan has joined #openstack-ansible | 08:07 | |
| *** vnogin has joined #openstack-ansible | 08:08 | |
| *** vnogin has quit IRC | 08:13 | |
| *** cshen has quit IRC | 08:20 | |
| *** evin has quit IRC | 08:23 | |
| evrardjp | odyssey4me: thanks a lot for that, I am still discussing with bgpvpn :p | 08:31 |
| odyssey4me | sure thing | 08:32 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible stable/queens: Add heat dashboard into packages https://review.openstack.org/561158 | 08:34 |
| *** shardy has quit IRC | 08:35 | |
| *** radeks has quit IRC | 08:35 | |
| *** vnogin has joined #openstack-ansible | 08:36 | |
| evrardjp | odyssey4me: what needs bumping in q/p/o? nova only? | 08:39 |
| odyssey4me | to my knowledge, yes | 08:39 |
| odyssey4me | oh, could I get some eyes on https://review.openstack.org/559995 please | 08:40 |
| odyssey4me | could also bump os_neutron to include these, but I don't think it's essential: https://review.openstack.org/#/q/I45247cd4fe9a54dde6c866e23b1a62a0632467a3 | 08:41 |
| speedmann | HI all, I just added some new compute nodes to our openstack ansible deployment... Somehow those nodes are missing the neutron-l3-agents. Anyone has any idea how this could be solved? | 08:44 |
| evrardjp | speedmann: could you dump your inventory somewhere? | 08:45 |
| evrardjp | speedmann: maybe I should start by asking what did you run to add your compute node, and on which branch are you | 08:45 |
| evrardjp | and a log of those actions would be cool too | 08:46 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_nova master: Re-order database migrations https://review.openstack.org/559354 | 08:46 |
| speedmann | We're still on newton, and we used the docs from here https://docs.openstack.org/openstack-ansible/latest/admin/scale-environment.html to add the new nodes | 08:47 |
| speedmann | we tried the newton docs, but they failed because of the missing setup step | 08:47 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_nova master: Re-order database migrations https://review.openstack.org/559354 | 08:47 |
| *** vnogin has quit IRC | 08:48 | |
| odyssey4me | speedmann newton is EOL and has some known issues posted in the release notes which came up after EOL, I would recommend going with ocata or later... that said, the l3 agents are in the neutron-agents container on the control plane hosts | 08:49 |
| odyssey4me | not on the compute hosts | 08:49 |
| odyssey4me | evrardjp given https://review.openstack.org/559354's last few edits were only comment clarifications, I think it needs review now | 08:50 |
| odyssey4me | that should unblock master upgrades - at least for nova | 08:50 |
| *** radeks has joined #openstack-ansible | 08:52 | |
| openstackgerrit | Maxime Guyot proposed openstack/openstack-ansible stable/queens: Update os_nova SHA to fix nova-manage cell_v2 discover_hosts https://review.openstack.org/561162 | 08:52 |
| *** shardy has joined #openstack-ansible | 08:52 | |
| evrardjp | Miouge: I am busy doing it :) | 08:52 |
| Miouge | OK no worries, I can abandon 561162 then | 08:53 |
| speedmann | odyssey4me yes I know. Unfortunately we are currently stuck to newton. I guess the l3-agents should be running on the compute n odes also because we are using DVR | 08:53 |
| odyssey4me | oh I see | 08:54 |
| odyssey4me | hmm, ok, I don't know enough about how the dvr setup is done to advise | 08:54 |
| evrardjp | odyssey4me: I am reading the patch | 08:54 |
| odyssey4me | evrardjp if Miouge does it, then we have two cores to vote on it ;) | 08:55 |
| evrardjp | yes indeed | 08:55 |
| evrardjp | Miouge: keep it in | 08:55 |
| evrardjp | I was abandonning mine :p | 08:55 |
| odyssey4me | if Miouge can do all the sha bumps, we're winning | 08:55 |
| evrardjp | yup | 08:55 |
| odyssey4me | the role sha bumps I mean | 08:55 |
| evrardjp | and I like contributions :) | 08:55 |
| evrardjp | yeah yeah | 08:55 |
| evrardjp | I was definitely not clear | 08:56 |
| *** vnogin has joined #openstack-ansible | 08:56 | |
| odyssey4me | Miouge can you add 'Closes-Bug: 1762742' to the commit message | 08:56 |
| openstack | bug 1762742 in openstack-ansible "Some delegated tasks are failing when deployer is node part of a deployment (eg. controller node)" [Undecided,New] https://launchpad.net/bugs/1762742 | 08:56 |
| evrardjp | indeed: "A smiley is not a proper explanation of what's inside my head" | 08:56 |
| openstackgerrit | Merged openstack/openstack-ansible-os_nova stable/ocata: Update tests a-r-r to use newton-eol tag https://review.openstack.org/560976 | 08:56 |
| openstackgerrit | Merged openstack/openstack-ansible-os_nova stable/ocata: Use first conductor for nova-manage cell_v2 discover_hosts https://review.openstack.org/560923 | 08:56 |
| Miouge | I just wonder if I need to add some more info in the commit message like Related-Bug etc� I was checking previous ansible-requirements.yaml to see how you usually do it | 08:56 |
| evrardjp | Miouge: odyssey4me just answered that :) | 08:56 |
| odyssey4me | haha :) | 08:56 |
| Miouge | He is typing faster than me :) | 08:57 |
| odyssey4me | two cups of coffee down ;) | 08:57 |
| odyssey4me | thinking about another one | 08:57 |
| *** sar has joined #openstack-ansible | 08:57 | |
| evrardjp | odyssey4me: WOW | 08:58 |
| evrardjp | EXTRA BOOST | 08:58 |
| evrardjp | rampage! | 08:58 |
| evrardjp | Miouge: for sure he is! | 08:58 |
| evrardjp | :D | 08:58 |
| odyssey4me | https://media.giphy.com/media/26xBEamXwaMSUbV72/giphy.gif | 08:58 |
| evrardjp | HAHAH | 08:58 |
| speedmann | odyssey4me, I see, still, thank you. we are going to figure out why this is broken for us | 08:58 |
| odyssey4me | speedmann I know the dvr documentation was a bit thin in newton - I don't think how it works has changed much in ocata or even pike, so you may find some clues there | 08:59 |
| odyssey4me | by that I mean our dvr setup documentation | 08:59 |
| evrardjp | odyssey4me: https://review.openstack.org/#/c/561158/1 | 09:00 |
| evrardjp | manual thing, so check it twice as it will be completely hidden in our gate testing for now (we haven't introduced metal back yet in Q iirc) | 09:00 |
| evrardjp | oh yes we did | 09:01 |
| evrardjp | glad it merged | 09:01 |
| Miouge | odyssey4me: how do I go about bumping the other SHA, in another patch or all at once? | 09:02 |
| odyssey4me | evrardjp mind if I tweak that just a small bit | 09:02 |
| odyssey4me | the line spacing is wrong, and there's a confusing word in the commit msg | 09:02 |
| odyssey4me | Miouge only the os_nova sha must bump for now | 09:02 |
| odyssey4me | evrardjp will do a general bump after this release | 09:03 |
| Miouge | OK, sounds good | 09:03 |
| openstackgerrit | Maxime Guyot proposed openstack/openstack-ansible stable/queens: Update os_nova SHA to fix nova-manage cell_v2 discover_hosts https://review.openstack.org/561162 | 09:03 |
| odyssey4me | Miouge LGTM, now can you do the same for pike & ocata? | 09:04 |
| openstackgerrit | Maxime Guyot proposed openstack/openstack-ansible stable/pike: Update os_nova SHA to fix nova-manage cell_v2 discover_hosts https://review.openstack.org/561167 | 09:05 |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/openstack-ansible-specs master: specs: rocky: Add new openstack-distribution-packages specification https://review.openstack.org/556832 | 09:07 |
| openstackgerrit | Maxime Guyot proposed openstack/openstack-ansible stable/ocata: Update os_nova SHA to fix nova-manage cell_v2 discover_hosts https://review.openstack.org/561170 | 09:08 |
| odyssey4me | well, that's not fun: http://logs.openstack.org/49/561149/1/check/openstack-ansible-functional-ubuntu-xenial/cdfbda7/job-output.txt.gz#_2018-04-13_08_33_09_330673 | 09:10 |
| evrardjp | odyssey4me: loading and afraid now. | 09:10 |
| odyssey4me | cinder ocata tempest tests failing | 09:10 |
| odyssey4me | with a rather obscure error | 09:10 |
| evrardjp | oh again | 09:11 |
| evrardjp | that damn unicode test | 09:11 |
| evrardjp | I can't remember what I did | 09:11 |
| evrardjp | I knew it was a ticking timebomb | 09:11 |
| odyssey4me | lol https://bugzilla.redhat.com/show_bug.cgi?id=1461192 | 09:11 |
| openstack | bugzilla.redhat.com bug 1461192 in openstack-tempest "AttributeError: 'module' object has no attribute 'wait_for_volume_resource_status'" [High,Closed: duplicate] - Assigned to chkumar | 09:11 |
| evrardjp | I have to go back to my old commits | 09:11 |
| *** pcaruana has joined #openstack-ansible | 09:12 | |
| odyssey4me | well yes, that'd be a problem: https://github.com/openstack/requirements/blob/stable/ocata/upper-constraints.txt#L405 | 09:12 |
| odyssey4me | also https://bugs.launchpad.net/cinder/+bug/1667448 | 09:13 |
| openstack | Launchpad bug 1667448 in Cinder ocata "Change in method name in tempest/common broke cinder's internal tempest tests" [Undecided,Fix released] - Assigned to Eric Harney (eharney) | 09:13 |
| evrardjp | tempest bypassing those, isn't it? | 09:13 |
| evrardjp | isolated and ignorereqs? | 09:13 |
| evrardjp | or the ignorereqs doesn't bypass? | 09:13 |
| odyssey4me | evrardjp for the role tests we're now using whatever's in u-c | 09:14 |
| evrardjp | oh fun. | 09:14 |
| odyssey4me | we used to use the git source with a fixed sha, but changed that | 09:14 |
| evrardjp | yeah | 09:14 |
| evrardjp | I guess we can override that constraint | 09:14 |
| odyssey4me | well, it's ocata and not crucial right now - there are better things to worry about | 09:15 |
| evrardjp | using ==version | 09:15 |
| odyssey4me | yes we can - I'll look at it some time when I'm bored, given that this is role test only I don't think it's worth much of a bother | 09:15 |
| evrardjp | ok | 09:15 |
| odyssey4me | fyi for https://review.openstack.org/559995 - I have no idea what the right things is to do for those services, but to my knowledge we have no SME's active for them... I figured that if anyone uses them in their test environment then the'll pick up issues and can push a patch up | 09:16 |
| odyssey4me | *if they pick up issues | 09:16 |
| evrardjp | odyssey4me: that's my hope too | 09:17 |
| evrardjp | but I'd like to see some ppl chime in | 09:17 |
| odyssey4me | also, for https://review.openstack.org/559791 I don't see the 'in development' warning adding any value... all OSA code is always in development - so I tried to change the initial warning to be clear that not all environments are equal | 09:17 |
| evrardjp | is it in your upgrade testing path? | 09:17 |
| evrardjp | if it's not it doesn't hurt to wait a little. it doesn't do good either though. | 09:18 |
| evrardjp | so that's why I added a deadline | 09:18 |
| odyssey4me | evrardjp I did test that it works, but I don't know the services well enough to validate whether it breaks things or not. I did look for service disable and agent disable commands... but there were none. | 09:18 |
| evrardjp | odyssey4me: yup, I trust you. And it's a close patterns to the others, which I have tested in a intermediate patchset. | 09:19 |
| evrardjp | so I belive it's fine | 09:19 |
| evrardjp | for what its worth | 09:19 |
| evrardjp | trove itself doesn't deploy good so... meh | 09:20 |
| evrardjp | but still | 09:20 |
| *** radeks has quit IRC | 09:20 | |
| evrardjp | for the "in development" I like the commit itself | 09:20 |
| evrardjp | but I don't think it's good enough to say that | 09:20 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible stable/queens: Add heat dashboard into packages https://review.openstack.org/561158 | 09:20 |
| evrardjp | right now it's broken, so we are setting wrong expectations. Maybe worth saying please double check zuul status before upgrading? | 09:21 |
| odyssey4me | evrardjp ^ why is that not patched into master - or are you going to do that within a sha bump patch? | 09:21 |
| evrardjp | it's in the sha bump patch in master | 09:21 |
| evrardjp | https://review.openstack.org/#/c/558082/ | 09:21 |
| odyssey4me | evrardjp no, we can't say 'check zuul status' - but we can say 'try it in a test environment first' | 09:22 |
| evrardjp | ok | 09:22 |
| evrardjp | I won't fight on this, it's not to the user to do the developer's work. | 09:22 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible stable/queens: Add heat dashboard into packages https://review.openstack.org/561158 | 09:23 |
| evrardjp | but we are not yet in a good shape | 09:23 |
| evrardjp | odyssey4me: I wanted to add the heat dashboard before doing the Q release today | 09:23 |
| evrardjp | we should have filed a bug for it | 09:23 |
| evrardjp | but meh | 09:23 |
| evrardjp | too late now | 09:23 |
| odyssey4me | evrardjp there is absolutely no way that we can guarantee that an upgrade will work for everyone - we have to warn them to test in a test environment first | 09:23 |
| evrardjp | that's true but it's not a <=> condition | 09:24 |
| evrardjp | it's a => | 09:24 |
| evrardjp | we can guarantee that some upgrades WILL fail. | 09:24 |
| evrardjp | but it's detail, let's not bother ourselves too much wiht it | 09:25 |
| evrardjp | odyssey4me: just curious why did you move it? https://review.openstack.org/#/c/561158/1..3/playbooks/defaults/repo_packages/openstack_services.yml | 09:26 |
| evrardjp | it's outside alphabetical order now | 09:27 |
| *** DanyC has joined #openstack-ansible | 09:27 | |
| evrardjp | look also a little below with trove dashboard and all | 09:27 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: Remove in-development warning from run-upgrade https://review.openstack.org/559791 | 09:27 |
| *** DanyC has quit IRC | 09:27 | |
| evrardjp | I don't really care but it's inconsistent | 09:28 |
| *** DanyC has joined #openstack-ansible | 09:28 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: Remove double warning from run-upgrade https://review.openstack.org/559791 | 09:28 |
| odyssey4me | evrardjp ^ perhaps that's better? | 09:28 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: Remove double warning from run-upgrade https://review.openstack.org/559791 | 09:28 |
| evrardjp | yup | 09:29 |
| evrardjp | thanks | 09:29 |
| odyssey4me | :) it just took me realising that you didn't think the warning was strong enough... | 09:30 |
| evrardjp | haha | 09:30 |
| *** udesale_ has joined #openstack-ansible | 09:30 | |
| evrardjp | odyssey4me: could you vote on https://review.openstack.org/#/c/559788/ ? | 09:30 |
| evrardjp | that would be useful for testing some backport on q | 09:30 |
| odyssey4me | done | 09:31 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-ceph_client stable/queens: Adapt the list of ceph packages for SUSE https://review.openstack.org/561173 | 09:32 |
| *** udesale has quit IRC | 09:33 | |
| evrardjp | odyssey4me: do you know where we are in jimmy's tempest patch backport? | 09:34 |
| evrardjp | seems ok till P | 09:34 |
| evrardjp | good | 09:34 |
| evrardjp | it's sometimes hard to see if somehting has been backported because the owner of the backport is not the same as the author of the original patch... but it's generally you or me, so it's fine at the end. | 09:35 |
| evrardjp | :D | 09:35 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-os_nova stable/queens: Update paste, policy and rootwrap configurations 2018-03-31 https://review.openstack.org/558086 | 09:36 |
| evrardjp | odyssey4me: I think with the use of kvm on nova role tests, nova goes better. | 09:37 |
| odyssey4me | evrardjp I think the changes giving rise to the issue only went to p - not sure though | 09:37 |
| evrardjp | on role tests in general | 09:37 |
| evrardjp | odyssey4me: yup | 09:38 |
| evrardjp | odyssey4me: I think this: https://review.openstack.org/#/c/557896/ is still worth doing | 09:38 |
| evrardjp | your opinion? | 09:38 |
| odyssey4me | yes, that may help when we are on a host which executes with wemu | 09:38 |
| odyssey4me | *qemu | 09:39 |
| evrardjp | yup. | 09:41 |
| evrardjp | I've seen real tempest issues now | 09:41 |
| evrardjp | now that the timeout is reached for low machines we seem some processed where killed due to OOM | 09:42 |
| evrardjp | swift (and IIRC glance) being the worse ram users | 09:42 |
| odyssey4me | evrardjp did you notice https://review.openstack.org/561012 ? it looks important | 09:49 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-ops master: Update docs and add inventory example https://review.openstack.org/561092 | 09:53 |
| *** cshen has joined #openstack-ansible | 09:54 | |
| *** armaan has quit IRC | 09:55 | |
| *** vnogin has quit IRC | 09:57 | |
| *** hamza21 has joined #openstack-ansible | 10:02 | |
| *** gaoyan has quit IRC | 10:05 | |
| hwoarang | evrardjp: i think you recent ceph_client changes may have broke stuff | 10:08 |
| hwoarang | http://logs.openstack.org/periodic/git.openstack.org/openstack/openstack-ansible/master/openstack-ansible-deploy-ceph-opensuse-423/837decb/job-output.txt.gz#_2018-04-13_07_20_25_360619 | 10:08 |
| hwoarang | if you are not looking at it can i can look at it | 10:08 |
| *** madhu has joined #openstack-ansible | 10:15 | |
| evrardjp | mmm | 10:16 |
| *** Madhu-tescra has joined #openstack-ansible | 10:17 | |
| *** Madhu-tescra has quit IRC | 10:19 | |
| *** vnogin has joined #openstack-ansible | 10:21 | |
| evrardjp | hwoarang: oh I see | 10:23 |
| evrardjp | stupid me | 10:23 |
| *** jwitko has quit IRC | 10:24 | |
| evrardjp | oh no I thought I saw. | 10:30 |
| evrardjp | ok found it | 10:31 |
| evrardjp | I should clean my glasses | 10:31 |
| *** nicolasbock has joined #openstack-ansible | 10:31 | |
| *** vnogin has quit IRC | 10:32 | |
| hwoarang | λολ | 10:32 |
| hwoarang | oh | 10:32 |
| hwoarang | stupid keyboard | 10:32 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-ceph_client master: Add the proper variable name for SUSE https://review.openstack.org/561188 | 10:32 |
| evrardjp | hwoarang: hahaha | 10:32 |
| evrardjp | lambda omicron -- persei 8! | 10:33 |
| evrardjp | For thos who don't know about it, it's the domain of Lrrr | 10:34 |
| openstackgerrit | Merged openstack/openstack-ansible-rabbitmq_server master: Configure plugins and policies after RabbitMQ https://review.openstack.org/559495 | 10:35 |
| hwoarang | what are you talking about! | 10:35 |
| odyssey4me | hwoarang just ignore him, he's talking gibberish again ;) | 10:36 |
| hwoarang | hahaha | 10:36 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-ceph_client stable/queens: Adapt the list of ceph packages for SUSE https://review.openstack.org/561173 | 10:36 |
| evrardjp | OMG | 10:36 |
| evrardjp | CORE DEMOTE | 10:36 |
| odyssey4me | https://media.giphy.com/media/3o8doOlGO3pjQa5h28/giphy.gif | 10:36 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-ceph_client stable/queens: Adapt the list of ceph packages for SUSE https://review.openstack.org/561173 | 10:37 |
| hwoarang | i sensed it might be some sci-fi nonsense | 10:37 |
| evrardjp | http://futurama.wikia.com/wiki/Omicron_Persei_8 | 10:37 |
| hwoarang | -_- | 10:37 |
| evrardjp | you sensed it correctly sir! | 10:38 |
| evrardjp | futurama though | 10:38 |
| evrardjp | that's good kwaltay | 10:38 |
| evrardjp | kwalitay | 10:38 |
| * hwoarang can't parse anything | 10:39 | |
| evrardjp | haha | 10:39 |
| odyssey4me | evrardjp is it fun over there on the plane lerl | 10:39 |
| evrardjp | depends if Ndnd is there or not | 10:40 |
| hwoarang | you are prob talking using base64 or something | 10:40 |
| evrardjp | haha | 10:40 |
| evrardjp | that's not news | 10:40 |
| evrardjp | :) | 10:40 |
| evrardjp | I am sad you both haven't seen futurama | 10:41 |
| evrardjp | or that part at least. | 10:41 |
| odyssey4me | I saw a bit of it. Never got into it. | 10:41 |
| evrardjp | Definitely better than the simpons, but I might be killed by saying that out loud. | 10:42 |
| evrardjp | simpsons* | 10:42 |
| evrardjp | hwoarang: could you vote on this series please: https://review.openstack.org/#/q/topic:do_not_log_passwords+(status:open+OR+status:merged) ? | 10:42 |
| *** udesale__ has joined #openstack-ansible | 10:42 | |
| hwoarang | yep on it | 10:43 |
| *** oanson has quit IRC | 10:44 | |
| *** udesale_ has quit IRC | 10:45 | |
| *** vnogin has joined #openstack-ansible | 10:51 | |
| odyssey4me | yay, a review storm :) | 10:55 |
| openstackgerrit | Merged openstack/openstack-ansible-ops master: Do not log passwords https://review.openstack.org/560377 | 10:56 |
| openstackgerrit | Merged openstack/openstack-ansible-ceph_client master: Add the proper variable name for SUSE https://review.openstack.org/561188 | 10:59 |
| *** cshen has quit IRC | 10:59 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_monasca-agent master: Do not log passwords https://review.openstack.org/560391 | 11:00 |
| *** rodolof has joined #openstack-ansible | 11:02 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_molteniron master: Do not log passwords https://review.openstack.org/560383 | 11:03 |
| *** tomtom001 has quit IRC | 11:04 | |
| *** tomtom001 has joined #openstack-ansible | 11:04 | |
| *** madhu has quit IRC | 11:05 | |
| *** niraj_singh has quit IRC | 11:06 | |
| *** cshen has joined #openstack-ansible | 11:06 | |
| *** niraj_singh has joined #openstack-ansible | 11:07 | |
| *** dave-mccowan has joined #openstack-ansible | 11:17 | |
| *** armaan has joined #openstack-ansible | 11:17 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_cloudkitty master: Do not log passwords https://review.openstack.org/560390 | 11:20 |
| openstackgerrit | Merged openstack/openstack-ansible-os_cloudkitty master: Do not log passwords https://review.openstack.org/560380 | 11:22 |
| *** vnogin has quit IRC | 11:22 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_watcher master: Do not log passwords https://review.openstack.org/560385 | 11:23 |
| *** rodolof has quit IRC | 11:24 | |
| *** rodolof has joined #openstack-ansible | 11:25 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_tacker master: Do not log passwords https://review.openstack.org/560392 | 11:26 |
| gokhan_ | hi team, I modified my provider networks(one of them is deleted) on openstack_user_config.yml file and run setup_hosts.yml but it doesn't apply changes on container interfaces. what is your advice in this situation ? | 11:28 |
| *** udesale_ has joined #openstack-ansible | 11:30 | |
| openstackgerrit | Merged openstack/openstack-ansible-tests master: Do not log passwords https://review.openstack.org/560386 | 11:31 |
| *** udesale__ has quit IRC | 11:32 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_rally master: Do not log passwords https://review.openstack.org/560384 | 11:34 |
| *** udesale_ has quit IRC | 11:35 | |
| *** manous has joined #openstack-ansible | 11:35 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_keystone master: Do not log passwords https://review.openstack.org/560382 | 11:37 |
| *** sar has quit IRC | 11:47 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_nova master: Fix kernel post-installation script https://review.openstack.org/561012 | 11:51 |
| *** vnogin has joined #openstack-ansible | 11:52 | |
| odyssey4me | gokhan_ which branch/tag? | 11:52 |
| *** bhujay has joined #openstack-ansible | 11:53 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_heat master: Update paste, policy and rootwrap configurations 2018-03-31 https://review.openstack.org/558077 | 11:56 |
| *** armaan has quit IRC | 11:57 | |
| *** vnogin has quit IRC | 11:57 | |
| *** sar has joined #openstack-ansible | 11:58 | |
| *** armaan has joined #openstack-ansible | 11:59 | |
| openstackgerrit | Merged openstack/openstack-ansible-tests master: Do not log passwords https://review.openstack.org/560394 | 12:02 |
| gokhan_ | odyssey4me, 16.0.4 tag | 12:05 |
| odyssey4me | gokhan_ I think you need https://github.com/openstack/openstack-ansible/commit/9629a03bf2ceab779ba4f588568bd5e55b28b413 | 12:06 |
| gokhan_ | odyssey4me, ok I am looking now | 12:08 |
| openstackgerrit | Merged openstack/openstack-ansible-os_barbican master: Do not log passwords https://review.openstack.org/560378 | 12:09 |
| *** fghaas has quit IRC | 12:09 | |
| jrosser | odyssey4me: does this make the dynamic inventory always start with the currently defined container_networks https://github.com/openstack/openstack-ansible/blob/master/osa_toolkit/generate.py#L609? I had to hack some out of the json by hand yesterday after fiddling with provider_networks.... | 12:10 |
| odyssey4me | jrosser I have no idea. | 12:11 |
| openstackgerrit | Merged openstack/openstack-ansible stable/queens: Update os_nova SHA to fix nova-manage cell_v2 discover_hosts https://review.openstack.org/561162 | 12:11 |
| odyssey4me | I reference to gokhan_ was just because I knew that part of code had been touched, not necessarily because I thought it would actually fix the problem. | 12:11 |
| gokhan_ | odyssey4me, yep this part is missing on my environment . I will apply it | 12:12 |
| jrosser | gokhan_: i was trying to do similar yesterday and found it difficult | 12:14 |
| jrosser | you can run the dynamic invetory script by hand and look at the output to check if the changes you've make to provider_networks reflect into the container config as you'd like | 12:15 |
| jrosser | it's not worth going much further until that looks right | 12:15 |
| bhujay | odyssey4me: i cant figure out why the timestamp in openstack env ( ocata) is not showing the local time from os . NAy create operation shows the utc time . IS there a setting in OSA or in openstack somewhere to change this ? | 12:16 |
| odyssey4me | hmm, I thought that we copied a file from the host on container/cache creation time, but I don't remember exactly | 12:17 |
| bhujay | that was /etc/locatime | 12:17 |
| gokhan_ | jrosser, thanks for your advices. firstl I will apply this commit . if it doesn't work on my environment, I will do it as you said | 12:18 |
| bhujay | odyssey4me , /etc/localtime , /etc/timezone all these i have manually modified now and restarted all the containers , yet no result | 12:19 |
| openstackgerrit | Merged openstack/openstack-ansible-os_nova master: Revert "Set nova-lxd job to non-voting" https://review.openstack.org/560890 | 12:19 |
| odyssey4me | sorry, I've no idea and I'm busy on some other work now... hopefully someone else can help | 12:20 |
| *** armaan has quit IRC | 12:26 | |
| *** fghaas has joined #openstack-ansible | 12:27 | |
| openstackgerrit | Merged openstack/openstack-ansible master: Do not log passwords https://review.openstack.org/560388 | 12:32 |
| openstackgerrit | Merged openstack/openstack-ansible master: Do not log passwords https://review.openstack.org/560376 | 12:33 |
| openstackgerrit | Merged openstack/openstack-ansible stable/ocata: Update os_nova SHA to fix nova-manage cell_v2 discover_hosts https://review.openstack.org/561170 | 12:33 |
| fghaas | odyssey4me, evrardjp, cloudnull et al: I just wanted to mention that I absolutely love run_upgrade.sh in Queens. With the latest patchset from Jesse it's actually running that thing from a screen session and just check on it when it's done. Beautiful. Could lose the big scary "experimental" warning for Rocky, I think. | 12:40 |
| evrardjp | fghaas: well it's been a long way. I remember it was not so nice in K. | 12:44 |
| evrardjp | I can't count the amount of hours to make that K L upgrade. | 12:44 |
| evrardjp | good old times. | 12:44 |
| fghaas | Well, let me think, what else what wasn't so nice in Kilo... hmmmm... it's on the tip of my tongue... | 12:44 |
| fghaas | OpenStack! That's what | 12:44 |
| odyssey4me | fghaas https://review.openstack.org/559791 :) | 12:45 |
| openstackgerrit | Merged openstack/openstack-ansible stable/queens: Add heat dashboard into packages https://review.openstack.org/561158 | 12:45 |
| odyssey4me | evrardjp L to M was bad too, remember the DNS name changes | 12:45 |
| fghaas | odyssey4me: oh sweet! Yeah good call on dropping that envar check. | 12:46 |
| odyssey4me | oh, and of course the introduction of using ssl certs | 12:46 |
| odyssey4me | fghaas yep, this way we still warn - but it's more like warning people to be responsible | 12:46 |
| fghaas | I think you should just drop that warning and just say "YOLO! Running upgrade." | 12:47 |
| *** vnogin has joined #openstack-ansible | 12:47 | |
| odyssey4me | haha | 12:47 |
| gokhan_ | odyssey4me, on my environment it doesn't work as jrosser said | 12:48 |
| odyssey4me | gokhan_ :/ | 12:48 |
| fghaas | (for the record if a humor-challenged person is reading this, that wasn't serious) | 12:48 |
| odyssey4me | well, I think for many people upgrading openstack is a bit of a YOLO experience | 12:48 |
| odyssey4me | I know I did a few of those back in the essex/folsom/grizzly days | 12:48 |
| odyssey4me | if it breaks somewhere, we'll just have to figure it out | 12:49 |
| fghaas | well yeah but you peeps are making it more like a "meh, another upgrade done" experience which is terrific | 12:49 |
| odyssey4me | yep, that's the hope | 12:49 |
| odyssey4me | the hope is that a minor upgrade is - checkout new tag, pull down the latest roles, run playbooks | 12:50 |
| odyssey4me | and a major upgrade is not much more complex | 12:50 |
| odyssey4me | maybe one day it will always just be the same as a minor upgrade | 12:50 |
| openstackgerrit | Niraj Singh proposed openstack/openstack-ansible-specs master: Add specs to create masakari role https://review.openstack.org/556756 | 12:51 |
| *** vnogin has quit IRC | 12:53 | |
| *** udesale has joined #openstack-ansible | 12:53 | |
| *** sawblade_ has quit IRC | 12:54 | |
| *** sawblade6 has joined #openstack-ansible | 12:54 | |
| openstackgerrit | Damian Cikowski proposed openstack/openstack-ansible-os_cinder master: tasks: cinder_qos: Fix the way how strings are matched https://review.openstack.org/561218 | 12:57 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_nova stable/queens: Fix kernel post-installation script https://review.openstack.org/561018 | 12:58 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_nova stable/pike: Fix kernel post-installation script https://review.openstack.org/561017 | 12:58 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_nova stable/ocata: Fix kernel post-installation script https://review.openstack.org/561219 | 12:59 |
| openstackgerrit | Merged openstack/openstack-ansible master: Update all SHAs for master https://review.openstack.org/558082 | 13:01 |
| openstackgerrit | Merged openstack/openstack-ansible stable/newton: Add known issue for os_neutron handler problem https://review.openstack.org/560906 | 13:01 |
| openstackgerrit | Merged openstack/openstack-ansible stable/queens: Fixes typo to enable log rotate for Octavia https://review.openstack.org/559383 | 13:01 |
| admin0 | openstack_hosts : Drop hosts file entries script locally : fatal: [b5 -> localhost]: FAILED! => {"changed": false, "msg": "AnsibleUndefinedVariable: ansible.vars.hostvars.HostVars object has no element None"} | 13:02 |
| admin0 | doh ! | 13:02 |
| odyssey4me | admin0 stale fact cache? | 13:05 |
| *** kstev has joined #openstack-ansible | 13:12 | |
| admin0 | odyssey4me, deleting the fact cache gives the save result | 13:16 |
| *** kstev has quit IRC | 13:17 | |
| *** kstev has joined #openstack-ansible | 13:19 | |
| *** vnogin has joined #openstack-ansible | 13:23 | |
| *** sar has quit IRC | 13:29 | |
| hwoarang | odyssey4me: evrardjp do you know how we can brake a statement on jinja2 template into multiple lines? | 13:34 |
| hwoarang | like {% if bla bla bla bla <break line here continue on the next one> | 13:35 |
| odyssey4me | hwoarang sure, lemme find an example | 13:35 |
| odyssey4me | just never close the %} for that line | 13:35 |
| odyssey4me | it will read the multiple lines as a string until it finds the close | 13:36 |
| *** lbragstad has joined #openstack-ansible | 13:36 | |
| odyssey4me | I thought I had an easy example, but I don't :/ | 13:36 |
| odyssey4me | ah, there it is hwoarang https://github.com/openstack/openstack-ansible-repo_build/blob/master/templates/venv-build-options.txt.j2#L124-L127 | 13:37 |
| *** vnogin has quit IRC | 13:38 | |
| hwoarang | oh i see | 13:38 |
| *** vnogin has joined #openstack-ansible | 13:38 | |
| hwoarang | thank you odyssey4me | 13:38 |
| *** kstev has quit IRC | 13:41 | |
| *** vnogin has quit IRC | 13:43 | |
| *** mbuil has quit IRC | 13:44 | |
| *** cmart has joined #openstack-ansible | 13:46 | |
| *** hachi has joined #openstack-ansible | 13:52 | |
| *** hamza21 has quit IRC | 13:55 | |
| *** esberglu has joined #openstack-ansible | 14:00 | |
| *** kstev has joined #openstack-ansible | 14:01 | |
| *** vnogin has joined #openstack-ansible | 14:08 | |
| *** armaan has joined #openstack-ansible | 14:10 | |
| *** hachi has quit IRC | 14:16 | |
| *** DanyC has quit IRC | 14:16 | |
| *** DanyC has joined #openstack-ansible | 14:17 | |
| *** DanyC has quit IRC | 14:22 | |
| *** vakuznet_ has joined #openstack-ansible | 14:23 | |
| *** ThomasS has joined #openstack-ansible | 14:26 | |
| *** vakuznet has quit IRC | 14:26 | |
| ThomasS | hi | 14:26 |
| ThomasS | i wanted to set this to nova resize_confirm_window | 14:26 |
| ThomasS | how can i do this if this is not implemented in osa | 14:27 |
| ThomasS | i know i have seen this in the docs but cannot find it anymore :-) | 14:27 |
| *** kstev1 has joined #openstack-ansible | 14:28 | |
| *** vnogin has quit IRC | 14:28 | |
| *** mma has quit IRC | 14:28 | |
| dcdamien | odyssey4me: It looks like task that adds QoS specs and assign it to volume types isn't covered by Zuul checks because default user_config lacks this setting. Can it be added into checks? Where it should be added? `openstack-ansible-os_cinder/tests/host_vars/cinder.yml` ? | 14:28 |
| *** mma has joined #openstack-ansible | 14:28 | |
| ThomasS | i found it in advanced config :-) | 14:29 |
| ThomasS | https://docs.openstack.org/project-deploy-guide/openstack-ansible/pike/app-advanced-config-override.html | 14:29 |
| odyssey4me | ThomasS like this: https://docs.openstack.org/project-deploy-guide/openstack-ansible/pike/app-advanced-config-override.html | 14:29 |
| *** kstev has quit IRC | 14:29 | |
| odyssey4me | evrardjp where is https://docs.openstack.org/project-deploy-guide/openstack-ansible/pike/app-advanced-config-override.html in the new docs? | 14:29 |
| evrardjp | config overrides seem a configuration item for reference ... so reference something about configuration | 14:30 |
| evrardjp | https://docs.openstack.org/openstack-ansible/latest/reference/configuration/advanced-config.html | 14:30 |
| evrardjp | first one is "using overrides" | 14:30 |
| evrardjp | I guess that's what you meant | 14:30 |
| evrardjp | it's also a full view of what's overridable | 14:30 |
| *** maybebuggy has quit IRC | 14:32 | |
| *** jwitko has joined #openstack-ansible | 14:32 | |
| *** mma has quit IRC | 14:33 | |
| odyssey4me | it'd be nice to have that config reference in the base index to find it | 14:35 |
| odyssey4me | I couldn't find it | 14:35 |
| cloudnull | mornings | 14:38 |
| ThomasS | thanks guys! | 14:38 |
| ThomasS | btw i wanted to add trove to an existing osa deployment | 14:38 |
| ThomasS | i added this to trove-infra_hosts: *infrastructure_hosts to openstack_user_config | 14:39 |
| *** sar has joined #openstack-ansible | 14:39 | |
| ThomasS | which playbooks i do need to run? | 14:39 |
| cloudnull | fghaas: thanks! Thank you and odyssey4me for putting a lot of that together for the queens upgrades. | 14:39 |
| spotz | hey | 14:39 |
| ThomasS | os-trove is for sure not enough ;-) | 14:39 |
| odyssey4me | ThomasS as usual, if you add a new service, you need to create the containers, build the software, deploy the software | 14:40 |
| odyssey4me | so lxc-containers-create.yml, then repo-build.yml, then os-trove-install.yml and os-horizon-install.yml (assuming you want the dashboard) | 14:41 |
| odyssey4me | there is also some diskimage building and stuff you need to do to get a functional environment - that's standard openstack which we leave to you | 14:41 |
| ThomasS | ok got it! thanks a lot! | 14:42 |
| odyssey4me | there are role docs, I think | 14:42 |
| ThomasS | you are right as always | 14:44 |
| ThomasS | https://docs.openstack.org/openstack-ansible/ocata/developer-docs/additional-roles.html | 14:44 |
| ThomasS | google helped | 14:44 |
| *** idlemind has joined #openstack-ansible | 14:44 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Add APM Server https://review.openstack.org/561238 | 14:45 |
| ThomasS | oh this is for developing a role ;-) | 14:45 |
| *** Chris___ has joined #openstack-ansible | 14:48 | |
| mnaser | time to run tempest on a newly deployed queens cloud | 14:48 |
| mnaser | lets see how badly this breaks :D | 14:48 |
| odyssey4me | ThomasS https://docs.openstack.org/openstack-ansible-os_trove/latest/configure-trove.html | 14:49 |
| *** openstackgerrit has quit IRC | 14:50 | |
| *** vakuznet has joined #openstack-ansible | 14:51 | |
| *** vakuznet_ has quit IRC | 14:51 | |
| Chris___ | Hey Guys, I'm having an issue when trying to run the setup-hosts playbook with openstack-ansible, I'm seeing 503 service request errors within the logs pertaining to the url containing the host internal_lb_vip_address defined within my user_config file. For ex http://<internal_lb_vip_address>:3142/acng-report.html, are there any suggestions as to why its using the internal_lb_vip_address for this? Or what could be causing | 14:52 |
| fghaas | Chris___: it always helps if you put a screencap (or C&P from your terminal) on paste.openstack.org so that people can see what specific error messages you're seeing. | 14:56 |
| *** cjloader has joined #openstack-ansible | 14:58 | |
| *** openstackgerrit has joined #openstack-ansible | 14:58 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Add grafana https://review.openstack.org/561243 | 14:58 |
| *** cjloader has quit IRC | 14:58 | |
| *** cjloader has joined #openstack-ansible | 14:59 | |
| fghaas | ^^ loving that one cloudnull | 14:59 |
| cloudnull | ++ grafana is nice. | 15:00 |
| *** rodolof has quit IRC | 15:00 | |
| cloudnull | now needs to make dashboards, which is not so nice :) | 15:00 |
| fghaas | :) | 15:01 |
| *** cshen has quit IRC | 15:01 | |
| odyssey4me | cloudnull not sure if you're aware of this, but there's an ansible role for grafana which does a bunch, including importing dashboards: https://github.com/cloudalchemy/ansible-grafana | 15:02 |
| odyssey4me | also, I've been working a bit on using grafyaml with grafana so that dasboards can be defined in code: https://github.com/openstack/ansible-role-grafyaml | 15:02 |
| cloudnull | very nice | 15:03 |
| odyssey4me | it's very basic for now (grafyaml), but it'll grow | 15:03 |
| odyssey4me | the cloudalchemy guys are very responsive to PR's too, and have some other roles of interest | 15:04 |
| odyssey4me | a bunch of prometheus ;) | 15:04 |
| cloudnull | with the recent ELK bits we have access to A LOT of data, and while Kibana is nice, grafana seems to be a lot better when visualizing data. | 15:04 |
| *** vnogin has joined #openstack-ansible | 15:04 | |
| odyssey4me | ja, kabana has a place different to grafana | 15:04 |
| cloudnull | ++ | 15:04 |
| d34dh0r53 | yeah, they fill different shoes imo | 15:06 |
| *** vnogin has quit IRC | 15:09 | |
| *** pcaruana has quit IRC | 15:12 | |
| *** AR2018 has joined #openstack-ansible | 15:13 | |
| *** gokhan_ has quit IRC | 15:13 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Add grafana https://review.openstack.org/561243 | 15:18 |
| cloudnull | odyssey4me: updated to use the linuxacadamy role. | 15:18 |
| odyssey4me | cloudalchemy ;) | 15:19 |
| cloudnull | that one. | 15:19 |
| cloudnull | :P | 15:19 |
| odyssey4me | one small spelling error, otherwise LGTM | 15:20 |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/openstack-ansible-specs master: specs: rocky: Add new openstack-distribution-packages specification https://review.openstack.org/556832 | 15:23 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Update docs and add inventory example https://review.openstack.org/561092 | 15:24 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Add grafana https://review.openstack.org/561243 | 15:24 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-ops master: Add grafana https://review.openstack.org/561243 | 15:24 |
| odyssey4me | oops | 15:24 |
| cloudnull | haha | 15:25 |
| odyssey4me | sorry, I'll leave it to you | 15:25 |
| cloudnull | np | 15:25 |
| odyssey4me | not sure what I broke there | 15:25 |
| *** throwsb1 has joined #openstack-ansible | 15:27 | |
| *** jbadiapa has quit IRC | 15:28 | |
| *** mma has joined #openstack-ansible | 15:30 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Add grafana https://review.openstack.org/561243 | 15:32 |
| Chris___ | fghaas: Here is a paste of my error http://paste.openstack.org/show/719165/ | 15:32 |
| *** mathlin has quit IRC | 15:32 | |
| *** vnogin has joined #openstack-ansible | 15:36 | |
| *** vnogin has quit IRC | 15:37 | |
| *** mma has quit IRC | 15:39 | |
| openstackgerrit | Merged openstack/openstack-ansible-ops master: Update docs and add inventory example https://review.openstack.org/561092 | 15:41 |
| dcdamien | It looks like os_cinder task that adds QoS specs and assign it to volume types isn't covered by Zuul checks because default user_config lacks this setting. Can it be added into checks? What is the right place to create override just for tests? | 15:43 |
| *** mbuil has joined #openstack-ansible | 15:45 | |
| *** armaan has quit IRC | 15:46 | |
| cloudnull | dcdamien: in the role, under the tests directory, you can add group vars for cinder to activate that code path | 15:48 |
| cloudnull | you could also add the overrides to https://github.com/openstack/openstack-ansible-os_cinder/blob/master/tests/os_cinder-overrides.yml | 15:49 |
| cloudnull | or create a new overrides file if you want to have it run as a specific stand-alone test. | 15:49 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Add APM Server https://review.openstack.org/561238 | 15:51 |
| cloudnull | dcdamien: that's all if you want it tested in the role specifically, you could also just add it to the defaults in the integrated openstack-ansible repo to make it deploy those QoS configs by default. | 15:52 |
| dcdamien | cloudnull: thanks for the guidance | 15:53 |
| *** hachi has joined #openstack-ansible | 15:57 | |
| *** admin0 has quit IRC | 16:05 | |
| *** admin0 has joined #openstack-ansible | 16:05 | |
| *** AR2018 has quit IRC | 16:06 | |
| mnaser | has anyone used the tempest role to validate a cloud post deployment? | 16:09 |
| mnaser | is it quite invasive and very ci-oriented or not as much? | 16:09 |
| odyssey4me | I think we'd like to move it in that direction, but it'll take some work. | 16:10 |
| odyssey4me | How invasive you think it is would depend more on your use-case. | 16:10 |
| mnaser | well like in terms of it creating its own networks, adding images, etc.. i guess all that can be disabled | 16:11 |
| odyssey4me | jrosser pushed this up, so it would seem that may be their intent: https://review.openstack.org/560670 | 16:11 |
| odyssey4me | mnaser possibly - not verified | 16:11 |
| odyssey4me | happy to have that be the case | 16:11 |
| mnaser | cause for example i have a cloud that doesnt yet have a public network (yay dealing with network providers) but i'd still like to test out most of the functionality of the cloud | 16:11 |
| *** radeks has joined #openstack-ansible | 16:12 | |
| odyssey4me | config_template is still usable | 16:12 |
| odyssey4me | and there are vars to explicitly disable a bunch of things | 16:12 |
| mnaser | true, but i guess i'll have to work on making some things to disable automatic creation of resources | 16:12 |
| *** pcaruana has joined #openstack-ansible | 16:15 | |
| logan- | yeah there are ways to disable all of that but it is not the easiest to configure imo | 16:16 |
| logan- | you might have better luck just using ansible + shade modules | 16:16 |
| logan- | thats what we do for image build CI. upload to openstack and then build instances with it for testing | 16:17 |
| *** gkadam has quit IRC | 16:21 | |
| jrosser | i figured i'd like to use the tempest role to validate the deploy was good | 16:22 |
| jrosser | got as far as overriding the tempest config with what by external net will be anyway | 16:24 |
| jrosser | *my | 16:24 |
| jrosser | hence this https://review.openstack.org/#/c/560670/ | 16:24 |
| admin0 | mnaser, i used rally .. set it up one of the util containers | 16:24 |
| admin0 | rally uses tempest roles | 16:24 |
| *** radeks has quit IRC | 16:24 | |
| admin0 | mnaser, https://rally.readthedocs.io/en/latest/ to get started | 16:26 |
| jrosser | mnaser: this caught me out for a while https://bugs.launchpad.net/openstack-ansible/+bug/1763336 | 16:29 |
| openstack | Launchpad bug 1763336 in openstack-ansible "stale tempest config remains in the workspace after re-running os_tempest role" [Undecided,New] | 16:29 |
| mnaser | oh yeah the whole tempest workspace things in a bit of a mess | 16:29 |
| mnaser | -_ | 16:29 |
| mnaser | not the role but like | 16:30 |
| mnaser | just the whole tempest workspace idea | 16:30 |
| *** mbuil has quit IRC | 16:30 | |
| spotz | cloudnull odyssey4me anyone else:) = Do we know of anyone who has gotten an OSA AIO to run on AWS? | 16:30 |
| *** sar has quit IRC | 16:32 | |
| * admin0 will not answer :) | 16:32 | |
| spotz | admin0: hehehe | 16:32 |
| mnaser | >:( | 16:34 |
| fghaas | spotz: I routinely run OSA non-AIO on OpenStack if this is generally about nested clouds | 16:34 |
| *** jbadiapa has joined #openstack-ansible | 16:34 | |
| *** jbadiapa has quit IRC | 16:35 | |
| *** jbadiapa has joined #openstack-ansible | 16:35 | |
| spotz | fghaas: Yeah I can build it on Rackspace, but AWS has messed up networking. Apparently someone did a presentation at a meetup about it | 16:35 |
| fghaas | I think the whole messed up networking was Ravello's raison d'ĂȘtre | 16:36 |
| spotz | fghaas: Yeah | 16:36 |
| fghaas | Chris___: looks like it's trying to connect to your repo container while either that repo container is unavailable, or there's no apt-cacher-ng running therein... hence why haproxy would throw that 503 | 16:37 |
| *** cshen has joined #openstack-ansible | 16:38 | |
| fghaas | Or, of course, you could have messed up your networking such that the host running haproxy is unable to connect to any services in the repo container | 16:38 |
| openstackgerrit | Damian Cikowski proposed openstack/openstack-ansible master: [WIP] Add extra volume types to AIO https://review.openstack.org/561266 | 16:40 |
| *** cshen has quit IRC | 16:42 | |
| *** epalper has quit IRC | 16:46 | |
| *** esberglu has quit IRC | 16:52 | |
| *** foutatoro has joined #openstack-ansible | 16:54 | |
| *** cshen has joined #openstack-ansible | 17:01 | |
| *** cshen has quit IRC | 17:06 | |
| *** shardy has quit IRC | 17:09 | |
| foutatoro | hi all, I'm deploy OSA with one ceph node for testing purpose. Ceph is correctly runnig and all pools required by glance and cinder are created. | 17:12 |
| foutatoro | but I get this error when running setup-openstack http://paste.openstack.org/show/719179/ | 17:13 |
| foutatoro | how to setup the ceph key in OSA vars ? | 17:13 |
| *** vakuznet_ has joined #openstack-ansible | 17:21 | |
| *** vakuznet has quit IRC | 17:23 | |
| openstackgerrit | Matthew Thode proposed openstack/openstack-ansible-os_ironic master: update for depricated driver definition https://review.openstack.org/561277 | 17:27 |
| shananigans | foutatoro: I think that task goes into one of the ceph devices defined in ceph_mons from your override variables and pulls it with something like 'ceph auth get client.<servicename>' and does a get-or-create if that fails. | 17:28 |
| shananigans | foutatoro: You might want to add it yourself to make sure you have access to the correct pool from one of the ceph boxes: ceph auth get-or-create client.glance mon 'allow r' osd 'allow class-read object_prefix rbd_children, allow rwx pool=glance-images' | 17:31 |
| shananigans | foutatoro: I think the client name needs to match what you specify in your overrides (glance_ceph_client, cinder_ceph_client...) | 17:37 |
| *** electrofelix has quit IRC | 17:38 | |
| evrardjp | fghaas: wow, properly written french:p | 17:38 |
| *** udesale has quit IRC | 17:53 | |
| *** esberglu has joined #openstack-ansible | 17:56 | |
| *** manous has quit IRC | 17:57 | |
| foutatoro | shananigans: Thanks so much the "ceph auth ..." for glance works correctly | 17:58 |
| foutatoro | I will check the correction for cinder | 17:58 |
| foutatoro | shananigans: It seems that OSA try to install again ceph in my node | 17:59 |
| foutatoro | shananigans: http://paste.openstack.org/show/719182/ | 18:00 |
| openstackgerrit | Damian Cikowski proposed openstack/openstack-ansible master: [WIP] Add extra volume types to AIO https://review.openstack.org/561266 | 18:05 |
| *** cshen has joined #openstack-ansible | 18:05 | |
| openstackgerrit | Damian Cikowski proposed openstack/openstack-ansible master: Add extra volume types to AIO https://review.openstack.org/561266 | 18:09 |
| *** cjloader_ has joined #openstack-ansible | 18:14 | |
| *** cjloader has quit IRC | 18:14 | |
| *** cjloader_ has quit IRC | 18:16 | |
| *** bhujay has quit IRC | 18:16 | |
| *** cjloader has joined #openstack-ansible | 18:17 | |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible stable/queens: Update all SHAs for 17.0.3 https://review.openstack.org/561288 | 18:20 |
| *** cjloader_ has joined #openstack-ansible | 18:21 | |
| *** cjloader has quit IRC | 18:21 | |
| *** cshen has quit IRC | 18:24 | |
| *** DanyC has joined #openstack-ansible | 18:24 | |
| *** cshen has joined #openstack-ansible | 18:24 | |
| prometheanfire | notmyfault? http://logs.openstack.org/77/561277/1/check/openstack-ansible-functional-ubuntu-xenial/7313a36/job-output.txt.gz#_2018-04-13_17_36_39_016828 | 18:31 |
| openstackgerrit | German Eichberger proposed openstack/openstack-ansible-os_octavia master: Downloads amphora images from artifact storage https://review.openstack.org/555084 | 18:32 |
| *** SimAloo has joined #openstack-ansible | 18:38 | |
| *** cshen has quit IRC | 18:41 | |
| *** cshen has joined #openstack-ansible | 18:52 | |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible stable/pike: Update all SHAs for 16.0.12 https://review.openstack.org/561299 | 19:05 |
| DimGR | hi | 19:06 |
| *** exodusftw has quit IRC | 19:12 | |
| openstackgerrit | Merged openstack/openstack-ansible stable/pike: Add missing Ubuntu packages https://review.openstack.org/560003 | 19:14 |
| *** idlemind has quit IRC | 19:15 | |
| *** idlemind has joined #openstack-ansible | 19:16 | |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible stable/ocata: Update all SHAs for 15.1.20 https://review.openstack.org/561302 | 19:16 |
| *** exodusftw has joined #openstack-ansible | 19:17 | |
| fghaas | evrardjp, am I being old school by using the circonflexe? | 19:17 |
| evrardjp | fghaas: you're doing great :) | 19:17 |
| *** cshen has quit IRC | 19:22 | |
| *** chudster has quit IRC | 19:24 | |
| *** chudster has joined #openstack-ansible | 19:28 | |
| *** adcpdk has quit IRC | 19:29 | |
| *** cjloader_ has quit IRC | 19:31 | |
| *** cjloader has joined #openstack-ansible | 19:34 | |
| *** cshen has joined #openstack-ansible | 19:35 | |
| *** cjloader has quit IRC | 19:36 | |
| *** cjloader has joined #openstack-ansible | 19:39 | |
| *** cjloader has quit IRC | 19:39 | |
| *** cjloader has joined #openstack-ansible | 19:39 | |
| *** fghaas has quit IRC | 19:47 | |
| prometheanfire | evrardjp: is this the right way to do nested conditionals? https://gist.github.com/prometheanfire/e6af23d8af5648322c05f35a03eab211 | 19:58 |
| prometheanfire | or anyone | 19:58 |
| *** radeks has joined #openstack-ansible | 20:00 | |
| openstackgerrit | Matthew Thode proposed openstack/openstack-ansible-os_ironic master: update for depricated driver definition https://review.openstack.org/561277 | 20:01 |
| prometheanfire | guess we'll see | 20:05 |
| openstackgerrit | German Eichberger proposed openstack/openstack-ansible-os_octavia master: Downloads amphora images from artifact storage https://review.openstack.org/555084 | 20:16 |
| *** kstev1 has quit IRC | 20:21 | |
| *** cshen has quit IRC | 20:23 | |
| *** sar has joined #openstack-ansible | 20:28 | |
| *** throwsb1 has quit IRC | 20:30 | |
| *** gokhan has joined #openstack-ansible | 20:37 | |
| *** radeks has quit IRC | 20:45 | |
| *** hachi has quit IRC | 20:47 | |
| *** Chris___ has quit IRC | 21:06 | |
| *** jwitko_ has joined #openstack-ansible | 21:08 | |
| *** jwitko has quit IRC | 21:12 | |
| logan- | prometheanfire whats the difference between the two big blocks you have split by the 'or' there | 21:14 |
| logan- | they look identical | 21:14 |
| *** esberglu has quit IRC | 21:15 | |
| *** jwitko_ has quit IRC | 21:16 | |
| *** fdsa_ is now known as fdsa | 21:17 | |
| *** gkadam has joined #openstack-ansible | 21:30 | |
| *** dave-mccowan has quit IRC | 21:36 | |
| *** evrardjp has quit IRC | 21:40 | |
| *** cshen has joined #openstack-ansible | 21:54 | |
| prometheanfire | logan-: one is openstack, one is standalone | 21:55 |
| prometheanfire | logan-: kinda annoying to have to diferentiate, but that's how it was, and if changed should be another patchset | 21:57 |
| * prometheanfire is happy it passed gate | 21:57 | |
| prometheanfire | just needs a doc update to be done now | 21:57 |
| *** gkadam has quit IRC | 21:58 | |
| *** cshen has quit IRC | 21:59 | |
| *** jbadiapa has quit IRC | 21:59 | |
| *** jra has joined #openstack-ansible | 22:16 | |
| jra | So in the OSA Operations Guide, it says that "MySQL data is automatically backed up." I can't find any other references to this in the docs; what's meant by that statement? | 22:17 |
| *** esberglu has joined #openstack-ansible | 22:20 | |
| openstackgerrit | German Eichberger proposed openstack/openstack-ansible-os_octavia master: Downloads amphora images from artifact storage https://review.openstack.org/555084 | 22:25 |
| *** AR2018 has joined #openstack-ansible | 22:31 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Add variables to connect ELK and Grafana https://review.openstack.org/561365 | 22:38 |
| cloudnull | jra: IDK if that's true any longer. | 22:39 |
| cloudnull | we used to have holland doing automatic backups | 22:39 |
| cloudnull | however I think that was removed | 22:39 |
| cloudnull | a while back. | 22:39 |
| cloudnull | likely something that was forgotten to be removed from the docs | 22:40 |
| cloudnull | though I could be wrong here. | 22:40 |
| jra | Yikes! That's kind of alarming! | 22:43 |
| cloudnull | yea, idk if holland works anymore... | 22:43 |
| jra | So, how are people backing up their databases these days? | 22:44 |
| cloudnull | I mean its still a DB cluster so the DB lives on >1 node (assuming you have >1 infra node) | 22:44 |
| cloudnull | but not a static backup. | 22:44 |
| jra | Yeah, static backups are a requirement for production deployments, as far as I can imagine | 22:44 |
| cloudnull | we use xtrabackup directly. | 22:44 |
| jra | Just call it yourselves on a cron and back up what comes out? | 22:44 |
| * cloudnull goes to really look how our teams do it :) | 22:45 | |
| cloudnull | https://github.com/rsoprivatecloud/openstack-ops/blob/master/playbooks/install-holland-db-backup.yml | 22:46 |
| cloudnull | looks like we still use holland :) | 22:46 |
| * cloudnull was mis-remembering | 22:46 | |
| openstackgerrit | Merged openstack/openstack-ansible-ops master: Add grafana https://review.openstack.org/561243 | 22:46 |
| cloudnull | which seems to be running on a cron | 22:46 |
| cloudnull | and is running with xtrabackup. | 22:47 |
| cloudnull | So i had the where and what just not the how :P | 22:47 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Add variables to connect ELK and Grafana https://review.openstack.org/561365 | 22:54 |
| *** AR2018 has quit IRC | 22:56 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_tempest master: Allow public network name to be changed https://review.openstack.org/560670 | 22:56 |
| *** sar has quit IRC | 22:58 | |
| *** AR2018 has joined #openstack-ansible | 23:10 | |
| jra | cloudnull: thanks very much for digging up those details! | 23:19 |
| openstackgerrit | German Eichberger proposed openstack/openstack-ansible-os_octavia master: Downloads amphora images from artifact storage https://review.openstack.org/555084 | 23:45 |
| *** DanyC has quit IRC | 23:45 | |
| *** AR2018 has quit IRC | 23:55 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!
