| *** harlowja has joined #ara | 01:16 | |
| harlowja | so dmsimard neats that this exists | 01:16 |
|---|---|---|
| harlowja | trying to integrate it into a bot we have that is going to do the deployments and such (that itself runs ansible) | 01:16 |
| harlowja | *slack bot | 01:17 |
| harlowja | and ara will provide a nice UI for all of that imho | 01:17 |
| *** themurph has quit IRC | 01:25 | |
| dmsimard | harlowja: sexy | 01:27 |
| harlowja | ya, this is a missing piece, so thx :-P | 01:27 |
| harlowja | will this just replace what i'd assume is in ansible tower eventually | 01:28 |
| harlowja | pretty sure tower has similar(?) code | 01:28 |
| dmsimard | harlowja: there's no real API for ARA, I definitely want to better split the components into clear server/client/api to allow people to further integrate with it | 01:28 |
| harlowja | fair nuff | 01:29 |
| dmsimard | Tower has it's own way of recording things, I didn't look at how they do it | 01:29 |
| dmsimard | I saw a demo of Tower once in my life | 01:29 |
| dmsimard | And I wasn't paying attention | 01:29 |
| dmsimard | So any similarities is mostly a coincidence :) | 01:29 |
| *** themurph has joined #ara | 01:31 | |
| dmsimard | Something I really wanna do and I haven't figured out yet how, mostly because I suck at logstash | 01:31 |
| dmsimard | Is to send ARA data to it | 01:32 |
| harlowja | ya, seems nice | 01:32 |
| dmsimard | Like, cliff CLI to json, dump that into logstash pre-indexed | 01:32 |
| dmsimard | It's good that you're here | 01:33 |
| dmsimard | Cause ara should probably be using (at least) oslo.log and oslo.db :p | 01:34 |
| harlowja | meh, up to u | 01:35 |
| dmsimard | Part of the challenge is that ARA uses the same (in fact outright imports) Ansible's config mechanism | 01:37 |
| dmsimard | So oslo.config is a no go | 01:37 |
| dmsimard | Unless there'd be a driver or something, not super familiar with oslo internals | 01:38 |
| harlowja | nah, i'd recommend to just stick with what u got | 01:39 |
| harlowja | similar with oslo.db | 01:47 |
| harlowja | not sure it'd buy u that much | 01:47 |
| dmsimard | Well, I have to fix logging in general, that's for sure. It has been very neglected. | 01:56 |
| harlowja | dmsimard something i just hit https://gist.github.com/harlowja/50e076a72993d50d18b047de039fb4cd | 01:59 |
| harlowja | though, i guess i'll fix that, just weird that pbr is blowing up there | 01:59 |
| *** openstackgerrit has joined #ara | 02:05 | |
| openstackgerrit | Joshua Harlow proposed openstack/ara master: If overriden PBR version isn't PBR compliant just ignore it https://review.openstack.org/449396 | 02:05 |
| dmsimard | Huh, never seen that before | 02:05 |
| harlowja | ya, pbr is to strict, lol | 02:05 |
| dmsimard | pbr is black magic to me | 02:06 |
| harlowja | ya, to much so imho, though i mean i get it | 02:06 |
| harlowja | just it's to strict for itself :-P | 02:06 |
| harlowja | like it has an ability to override that version | 02:06 |
| harlowja | but then u get stuff like this, where we overrode it, then it pukes (because we used its capability to override the version) | 02:07 |
| *** larsks has quit IRC | 02:18 | |
| dmsimard | harlowja: patch lgtm but how do you reproduce that particular issue ? | 02:18 |
| harlowja | do some like export PBR_VERSION=1.4.4.4.j2 should work i think | 02:20 |
| harlowja | *something like | 02:20 |
| *** larsks has joined #ara | 02:20 | |
| harlowja | https://gist.github.com/harlowja/4a87b6662d2e58dac40b43a558c24315 | 02:25 |
| dmsimard | ¯\_(ツ)_/¯ | 02:35 |
| openstackgerrit | Merged openstack/ara master: If overriden PBR version isn't PBR compliant just ignore it https://review.openstack.org/449396 | 02:41 |
| *** cliles has quit IRC | 04:01 | |
| *** karimb has joined #ara | 06:50 | |
| *** karimb has quit IRC | 06:51 | |
| *** jrist has quit IRC | 09:34 | |
| *** jrist has joined #ara | 13:22 | |
| openstackgerrit | Merged openstack/ansible-role-ara master: Remove weird non-ascii quote character https://review.openstack.org/448648 | 14:20 |
| dmsimard | leifmadsen: just looked at your dockerfile review again | 14:21 |
| dmsimard | leifmadsen: do I remember properly that you said there'd be a readme ? | 14:21 |
| leifmadsen | dmsimard: there should be? | 14:22 |
| leifmadsen | it was in the directory... | 14:22 |
| dmsimard | leifmadsen: it didn't make it's way to the review | 14:22 |
| leifmadsen | ok, I'll double check that | 14:22 |
| leifmadsen | maybe I forgot to ADD... | 14:22 |
| leifmadsen | git add* | 14:22 |
| dmsimard | my finger type "git add ." "git commit --amend" ":wq" and "git review" so fast | 14:23 |
| dmsimard | fingers* | 14:23 |
| dmsimard | it's muscle memory by now | 14:23 |
| openstackgerrit | Leif Madsen proposed openstack/ara master: Add Dockerfile for building ARA https://review.openstack.org/445058 | 14:58 |
| leifmadsen | dmsimard: ^^ | 14:58 |
| leifmadsen | yep, forgot the `git add` :) | 14:58 |
| leifmadsen | so I have a question! | 14:58 |
| dmsimard | leifmadsen: me too, you first | 14:59 |
| leifmadsen | I have a jenkins slave that runs oooq and uses ARA for callbacks, and then that generates a sqlite DB. I then have a plugin that copies that data back to the master into a centralized location. So now I have multiple sqlite DBs for each run on each slave (1 per workspace). Is there a way I "merge" those DBs so that I can get a centralized view of all the runs? | 14:59 |
| dmsimard | leifmadsen: not natively ? I mean, you could do like the equivalent of mysqldump with no create table statements and then pipe that to sqlite3 to import data into one database | 15:01 |
| leifmadsen | (I know the real answer is to write this data to a MySQL location, but I don't want to build that infra right now...) | 15:01 |
| dmsimard | there's probably a way with sqlite3 to dump data to another database with the same schema | 15:01 |
| leifmadsen | let me look on google | 15:01 |
| leifmadsen | it might even have the answer :) | 15:01 |
| leifmadsen | dmsimard: what was your question? | 15:02 |
| dmsimard | leifmadsen: I do have this bookmark, however http://www.tylerlesmann.com/2009/apr/27/copying-databases-across-platforms-sqlalchemy/ | 15:02 |
| dmsimard | That I wanted to try in the context of importing sqlite to mysql | 15:02 |
| leifmadsen | ah neato | 15:03 |
| leifmadsen | yea, the other idea I had, and not sure this can work currently, was to effectively "replay" from the logs in a centralized location, so that the DB being written to was the only one | 15:03 |
| leifmadsen | but don't think that's how ARA works | 15:04 |
| leifmadsen | oh the other real question I guess is... if I have multiple runs, and they write to the same DB, does the web interface provide a "list" of the different runs and their results? | 15:04 |
| leifmadsen | if not, then this is really a moot point :D | 15:04 |
| dmsimard | leifmadsen: asked my question in the Dockerfile review | 15:05 |
| dmsimard | leifmadsen: yeah, of course if you write to the same db, you have all the results in one place | 15:06 |
| leifmadsen | +1 | 15:06 |
| leifmadsen | dmsimard: looking | 15:06 |
| dmsimard | leifmadsen: http://ara-demo.dmsimard.com/reports/index.html | 15:06 |
| leifmadsen | yea, just found the youtube video too | 15:06 |
| dmsimard | is from *one* job from openstack-ansible | 15:06 |
| leifmadsen | dmsimard: I see your questions, and those are good points. I'll try and address them. | 15:07 |
| leifmadsen | dmsimard: are different runs groups differently? | 15:11 |
| leifmadsen | grouped* | 15:11 |
| leifmadsen | so like... I run jenins_job_1 and jenkins_job_2 | 15:11 |
| leifmadsen | but use ARA and write to the same DB file | 15:11 |
| leifmadsen | (for example) | 15:11 |
| dmsimard | leifmadsen: not sure I understand | 15:12 |
| dmsimard | an ansible-playbook execution is an ansible-playbook execution | 15:13 |
| leifmadsen | so you have all these rows in this demo | 15:13 |
| leifmadsen | are each of these one execution of ansible-playbook ? | 15:13 |
| dmsimard | each of those is a separate playbook run, a separate execution of ansible-playbook | 15:13 |
| leifmadsen | gotcha, ok cool | 15:13 |
| leifmadsen | so I run: ansible-playbook site.yml, and then run it again, I should see 2 rows | 15:14 |
| dmsimard | right | 15:14 |
| leifmadsen | ++ | 15:14 |
| dmsimard | and if you run ansible-playbook site.yml on another host pointing to the same db, you'll see 3 rows | 15:14 |
| leifmadsen | man I think it might be early lunch day :) | 15:14 |
| leifmadsen | ok awesome, so that's what I was hoping, perfect | 15:14 |
| leifmadsen | run now I don't see that because the runs are remote on the slave, so there is no centralized source of info | 15:15 |
| leifmadsen | they are all independent of each other, so I hadn't really tried it with a single DB :) | 15:15 |
| leifmadsen | 0.12.3 gives me this currently... | 16:39 |
| leifmadsen | /home/stack/quickstart/lib/python2.7/site-packages/ara/cli/generate.py:60: MissingURLGeneratorWarning: Nothing frozen for endpoints reports.ajax_results, reports.ajax_records, file.show_file, reports.ajax_stats, reports.ajax_plays, result.show_result, reports.ajax_files, host.show_host. Did you forget a URL generator? | 16:39 |
| leifmadsen | freezer.freeze() | 16:39 |
| leifmadsen | guess I'm missing a dep for some reason | 16:39 |
| leifmadsen | oh, or maybe flask changed something of how this is called | 16:41 |
| leifmadsen | dmsimard: ^ | 16:44 |
| dmsimard | leifmadsen: that means you generated from an empty databasr | 17:09 |
| dmsimard | brb | 17:09 |
| leifmadsen | hrmmm | 17:10 |
| leifmadsen | dmsimard: ah, did the path to the callbacks move? | 17:20 |
| leifmadsen | (that's what it looks like) | 17:20 |
| dmsimard | leifmadsen: not in a long time | 17:23 |
| leifmadsen | :) | 17:23 |
| dmsimard | leifmadsen: since like 5 months ? | 17:23 |
| leifmadsen | probably used to be like that WAY back in the day :) | 17:23 |
| leifmadsen | yea, so I guess I haven't tested this in a while :D | 17:23 |
| leifmadsen | well, mostly that our deploy at a partner is probably old at this point, so might have been using ARA 0.10 or something | 17:24 |
| leifmadsen | all good, found it, testing now, but thanks for the tip, that lead me in the right direction | 17:24 |
| leifmadsen | bah... I wonder if my issues are because there is an export ARA_DATABASE in quickstart.sh... | 17:37 |
| dmsimard | leifmadsen: maybe | 17:37 |
| dmsimard | leifmadsen: there's some magic there | 17:37 |
| dmsimard | leifmadsen: because, like, even browbeat gets ara reports for free or something | 17:38 |
| leifmadsen | I think the issue is that I have a mismatch between the path for ARA_DATABASE. I set it, then I think a DB is generated, then... quickstart.sh runs, but there is no DB in the location hard coded there | 17:38 |
| leifmadsen | going to try and match things up, and see if the DB is generated in the default path... | 17:39 |
| leifmadsen | dmsimard: so yea, that was it... | 18:40 |
| leifmadsen | removed all my exports for ARA and now the callbacks seem to write data to the sqlite db in ~/quickstart/ara.sqlite | 18:41 |
| leifmadsen | however... an ara generate html gave me this... :( https://paste.fedoraproject.org/paste/M-HHXC-2cxwMchTyxwKA~V5M1UNdIGYhyRLivL9gydE= | 18:41 |
| dmsimard | that's bad yo | 18:43 |
| dmsimard | This is not a connection string /home/stack/quickstart/ara.sqlite | 18:43 |
| harlowja | dmsimard qq, if we have extra_vars that have secrets in them, will those show up in the ara UI? | 18:43 |
| dmsimard | This is a connection string: sqlite:////home/stack/quickstart/ara.sqlite | 18:44 |
| dmsimard | harlowja: ara does not store vars/extra-vars or tags (yet) | 18:44 |
| harlowja | k | 18:44 |
| harlowja | if it does, can we run them through https://github.com/openstack/oslo.utils/blob/master/oslo_utils/strutils.py#L316 :) | 18:44 |
| harlowja | *if it ever does | 18:45 |
| dmsimard | harlowja: if you have tasks that would expose them (i.e in results' stdout), I suggest you use no_log: yes | 18:45 |
| harlowja | k | 18:45 |
| dmsimard | harlowja: ansible's no_log: yes is basically that | 18:45 |
| harlowja | hmmm, though i still want a log :-P | 18:45 |
| dmsimard | harlowja: http://docs.ansible.com/ansible/faq.html#how-do-i-keep-secret-data-in-my-playbook | 18:45 |
| harlowja | hmmm, k | 18:46 |
| harlowja | will have to try that | 18:46 |
| dmsimard | harlowja: I'm not sure what the result ends up being | 18:46 |
| dmsimard | harlowja: but yeah, secrets are already mentioned in the feature request https://storyboard.openstack.org/#!/story/2000747 | 18:47 |
| harlowja | kk | 18:47 |
| dmsimard | harlowja: there's also an "ara_no_log" directive that will maybe exist one day: https://storyboard.openstack.org/#!/story/2000840 | 18:47 |
| dmsimard | which wouldn't be recorded by ara at all | 18:47 |
| harlowja | ya, i like what jenkins does with ansible | 18:48 |
| harlowja | except for the jenkins part | 18:48 |
| dmsimard | leifmadsen: see my reply ? | 18:48 |
| harlowja | lol | 18:48 |
| dmsimard | harlowja: what does jenkins do ? | 18:48 |
| leifmadsen | dmsimard: oh sorry, I didn't, reading scrollback | 18:48 |
| leifmadsen | dmsimard: /me facepalms | 18:48 |
| leifmadsen | you're right | 18:48 |
| dmsimard | harlowja: my usage of jenkins with ansible is just a shell builder | 18:48 |
| leifmadsen | layer 8 ftl | 18:48 |
| dmsimard | harlowja: so nothing smart | 18:48 |
| harlowja | https://gist.githubusercontent.com/harlowja/be7247c4c380490cb470309a73fad1e8/raw/6946b391baa3d3c84cb12821c63bd49c8b274e44/gistfile1.txt | 18:48 |
| harlowja | that for example is running the script to deploy the control bot that i am making | 18:49 |
| dmsimard | harlowja: interesting | 18:49 |
| dmsimard | if you're using jenkins, there's this junit export thing too | 18:49 |
| harlowja | well i want to use ara and just python instead of jenkins for that | 18:49 |
| harlowja | *via the bot | 18:50 |
| dmsimard | that works too | 18:50 |
| harlowja | but still retain the extra vars (even if they turn into '****' for certain ones) | 18:50 |
| dmsimard | In the context of ara, if it's implemented it'd probably be an all-or-nothing deal | 18:51 |
| dmsimard | show the var values or don't show them | 18:51 |
| harlowja | well i'd assume u'd be able to pass it through at least https://github.com/openstack/oslo.utils/blob/master/oslo_utils/strutils.py#L248 | 18:51 |
| harlowja | right? | 18:51 |
| dmsimard | s/don't show them/don't record them/ | 18:51 |
| harlowja | ya, i guess the other thing is ansible vault for these vault | 18:52 |
| dmsimard | harlowja: I'd debate recording anything at all, recording obfuscated things in the first place is a bit pointless -- and recording them in clear and obfuscating only the output is worse | 18:53 |
| harlowja | fair | 18:53 |
| leifmadsen | dmsimard: so close! | 18:53 |
| leifmadsen | I feel like such a rookie... | 18:53 |
| leifmadsen | database doesn't appear to be empty now, but still getting a freezer.freeze() error | 18:53 |
| dmsimard | leifmadsen: what is it ? ajax_records ? | 18:53 |
| leifmadsen | yea | 18:54 |
| leifmadsen | /home/stack/quickstart/lib/python2.7/site-packages/ara/cli/generate.py:60: MissingURLGeneratorWarning: Nothing frozen for endpoints reports.ajax_records. Did you forget a URL generator? | 18:54 |
| dmsimard | leifmadsen: https://review.openstack.org/#/c/447553/ | 18:54 |
| dmsimard | leifmadsen: it's a warning, not an error -- your thing got generated properly | 18:54 |
| leifmadsen | oic | 18:54 |
| leifmadsen | looking | 18:54 |
| leifmadsen | ah nice | 18:54 |
| dmsimard | the warning is going away by default in 0.12.4 | 18:54 |
| leifmadsen | I looked in the ONE directory that had nothing diff :) | 18:54 |
| leifmadsen | ++ | 18:55 |
| leifmadsen | thanks! | 18:55 |
| dmsimard | I'll probably cut 0.12.4 next week, some fixes in there | 18:56 |
| leifmadsen | +1 | 19:07 |
| leifmadsen | rock on, thanks! | 19:07 |
| leifmadsen | things seem to be working now | 19:07 |
| dmsimard | leifmadsen: it's awesome when things work | 19:08 |
| leifmadsen | hell yea | 19:09 |
| leifmadsen | I've now exposed the generated logs for our jenkins slave runs | 19:09 |
| leifmadsen | much nice :) | 19:09 |
| dmsimard | magic | 19:09 |
| leifmadsen | http://i.imgur.com/iZcUNxH.gif | 19:10 |
| dmsimard | https://media.giphy.com/media/26ufdipQqU2lhNA4g/giphy.gif | 19:11 |
| leifmadsen | http://i.imgur.com/VvpBTG9.jpg | 19:11 |
| dmsimard | we must go deeper | 19:13 |
| dmsimard | http://i.imgur.com/p0DkDRG.jpg | 19:14 |
| leifmadsen | I feel like this is the nerdy version of a guitar battle | 19:15 |
| dmsimard | nah, that happens on /r/HighQualityGifs/ | 19:16 |
| leifmadsen | :) | 19:17 |
| *** tbielawa has joined #ara | 19:19 | |
| tbielawa | dmsimard: stickersstickersstickersstickersstickersstickersstickers | 19:34 |
| dmsimard | tbielawa: stickersstickersietksicekriasdhjlakjhgfdsg | 19:36 |
| tbielawa | lksjfd;lkasjd;flkjsdf yeah! | 19:40 |
| *** dougbtv|laptop has joined #ara | 20:01 | |
| *** russellb has joined #ara | 20:01 | |
| *** dougbtv|laptop has quit IRC | 20:35 | |
| openstackgerrit | David Moreau Simard proposed openstack/ara master: Test ansible-2.3.0.0-0.1.rc1 https://review.openstack.org/449813 | 20:50 |
| openstackgerrit | David Moreau Simard proposed openstack/ara master: Test ansible-2.2.2.0-0.2.rc2 https://review.openstack.org/449815 | 20:51 |
| openstackgerrit | David Moreau Simard proposed openstack/ara master: Test ansible-2.1.5.0-0.2.rc2 https://review.openstack.org/449816 | 20:52 |
| *** tbielawa is now known as tbielawa|weekend | 20:55 | |
| dmsimard | harlowja: what I told you about no_log earlier | 21:40 |
| dmsimard | that was a lie | 21:40 |
| harlowja | lol | 21:40 |
| dmsimard | I just learned that no_log places the burden on the callback to scrap the data | 21:40 |
| dmsimard | not the modules | 21:40 |
| harlowja | interesting | 21:40 |
| dmsimard | so the modules send the whole thing to the callbacks and they're the ones that have to "respect" no_log | 21:40 |
| dmsimard | harlowja: i.e, https://github.com/ansible/ansible/blob/9a05113daf4ab90b43c5570ef69b43e42ecd0ecd/lib/ansible/plugins/callback/__init__.py#L76-L78 | 21:41 |
| harlowja | wouldn't have expected that, but TIL | 21:41 |
| dmsimard | harlowja: yeah so ARA just slurps everything | 21:42 |
| dmsimard | I noticed because there was this one task (os_auth) that's pretty darn verbose and tells you all about tokens, endpoints and everything | 21:42 |
| harlowja | right right | 21:42 |
| dmsimard | had no_log on it | 21:42 |
| dmsimard | nope | 21:42 |
| harlowja | weirds | 21:42 |
| harlowja | the more u know, the less u want to | 21:42 |
| harlowja | lol | 21:42 |
| dmsimard | so I'll go and be a good little white hat | 21:43 |
| dmsimard | and patch that in | 21:43 |
| harlowja | :-P | 21:43 |
| * dmsimard sighs | 21:43 | |
| harlowja | at least patch in https://github.com/openstack/oslo.utils/blob/master/oslo_utils/strutils.py#L248 ? | 21:43 |
| dmsimard | harlowja: ara has access to that method I linked | 21:44 |
| dmsimard | it just doesn't use it, it dumped the raw data | 21:45 |
| openstackgerrit | David Moreau Simard proposed openstack/ara master: Use Ansible's CallbackBase._dump_results to dump results https://review.openstack.org/449827 | 21:51 |
| openstackgerrit | David Moreau Simard proposed openstack/ara master: Use Ansible's CallbackBase._dump_results to dump results https://review.openstack.org/449827 | 21:55 |
| harlowja | dmsimard kk | 21:56 |
| openstackgerrit | David Moreau Simard proposed openstack/ara master: Use Ansible's CallbackBase._dump_results to dump results https://review.openstack.org/449827 | 22:19 |
| *** logan- has joined #ara | 22:31 | |
| openstackgerrit | David Moreau Simard proposed openstack/ara master: Use Ansible's CallbackBase._dump_results to dump results https://review.openstack.org/449827 | 22:34 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!
