| *** dyasny__ has joined #ara | 01:25 | |
| *** dyasny__ is now known as dyasny | 01:25 | |
| dyasny | dmsimard, ping - are you here? | 01:26 |
|---|---|---|
| dmsimard | dyasny: kinda | 01:26 |
| dyasny | dmsimard, just wondering - does ara give an indication on ansible tasks that have a long-ish run, e.g. with repeats+delay? | 01:27 |
| dyasny | dmsimard, like, if I have a few of those in a play, can I get a feel for how long they typically take | 01:28 |
| dmsimard | dyasny: ara doesn't give you stats but it gives you the data you need to do your stats | 01:28 |
| dmsimard | dyasny: for example, the tripleo folks wrote an ara_influxdb module that sends task metrics from ara and then they have graphs on top of that | 01:29 |
| dmsimard | https://github.com/openstack/tripleo-quickstart-extras/blob/master/roles/collect-logs/library/ara_influxdb.py | 01:29 |
| dmsimard | https://review.rdoproject.org/grafana/dashboard/db/tripleo-ci?orgId=1 | 01:30 |
| dmsimard | It's pretty neat :) | 01:30 |
| dyasny | dmsimard, I see. I guess I could parse the output, I'm just looking for a way to get a report on those specific tasks, because I'm getting them failing sometimes and I increase the repeats to a larger number to get the play to pass. I am looking for a way to actually know when I expect to see 20 repeats and it actually abnormally takes 200, for example | 01:30 |
| dyasny | dmsimard, because openshift-ansible is such a beast, going through jenkins logs can be daunting :) | 01:31 |
| dmsimard | dyasny: yeah, I know what you mean, I'm working on a 3.9 upgrade right now :) | 01:31 |
| dyasny | dmsimard, same boat :) only with 3.10 and kubevirt :) | 01:32 |
| dmsimard | rebasing two patches because I have two PRs, one of which hasn't merged in >1yr | 01:32 |
| dyasny | that's no fun at all | 01:32 |
| dmsimard | kubevirt looks awesome (and scary) | 01:32 |
| dyasny | dmsimard, tell me about it :) still, it's more fun than openstack - less loosely fitting parts | 01:33 |
| dmsimard | dyasny: anyway, the UI and the CLI won't really be able to help you but you can export the raw data and analyze it after the fact (like tripleo does) | 01:34 |
| dmsimard | influxdb is super easy, I don't know in what context you're running with | 01:34 |
| dmsimard | they wrote something for graphite too but I don't think they ended up using it | 01:34 |
| dyasny | dmsimard, cool, I'll give it a try. I'm deploying a openshift first, and then kubevirt via an apb. Typically, openshift fails to deploy, and quite often it's environmental, because of a timeout being too short | 01:35 |
| dmsimard | dyasny: wait, you're setting up ara to track openshift-ansible ? | 01:36 |
| dmsimard | or for the apb ? | 01:36 |
| dyasny | I was thinking to inject a task that saves the start time before the long task, then ignore the long task error state and another task to log the time it's done in, and fail on errors there | 01:37 |
| dyasny | couldn't find anything native in ansible to deal with that, and then I remembered ara, for the "analysis" part | 01:37 |
| dmsimard | dyasny: when you want to do stuff like that, think about callbacks | 01:39 |
| dmsimard | dyasny: http://paste.openstack.org/show/723253/ | 01:40 |
| dmsimard | super simple | 01:40 |
| dyasny | dmsimard, yeah, I was doing that (changing the output to json and parsing it with python) for other projects, but that's huge. Was hoping for something that already knows how to catch tasks that use repeats and reports on those only | 01:40 |
| dmsimard | dyasny: something that you *could* try | 01:41 |
| dmsimard | dyasny: tag the tasks where you import/include the things, ara saves tags as well so you can search for those | 01:42 |
| dyasny | dmsimard, I just ran into an issue, when the same service takes 5 times longer to start using cri-o than docker. Nobody could debug it, because the service actually starts, just not within the default timeout. So I come in in the morning, see the job failed, start looking at the setup, and everything is working. Had to go through a huge log with no actual errors. Good thing my patch to make the repeats configurable got merged quickly, but I'd like to be able | 01:44 |
| dyasny | to catch such things on the fly, you know | 01:44 |
| *** bcoca has quit IRC | 02:20 | |
| *** mhu has quit IRC | 05:11 | |
| *** mhu has joined #ara | 05:11 | |
| *** myoung|off has quit IRC | 05:11 | |
| *** myoung has joined #ara | 05:17 | |
| *** ara-slack3 has joined #ara | 05:48 | |
| *** ara-slack has quit IRC | 05:49 | |
| *** ara-slack3 has quit IRC | 05:49 | |
| *** ara-slack has joined #ara | 05:50 | |
| *** ara-slack has joined #ara | 05:50 | |
| *** ChanServ sets mode: +v ara-slack | 05:50 | |
| openstackgerrit | zhangzhaoshan proposed openstack/ara master: replace http with https https://review.openstack.org/574575 | 06:29 |
| *** jparrill_ has joined #ara | 07:34 | |
| *** jparrill_ has quit IRC | 07:37 | |
| *** jparrill_ has joined #ara | 07:37 | |
| *** jparrill_ has quit IRC | 07:39 | |
| Naphta | So I'm having a bit of trouble with the web gui getting stuck on "loading file contents", the actual endpoint responds fine (e.g. "/file/5878c8c4-36ea-454f-88b8-9452599d2f94/") but it never loads in it's pop up frame. | 07:56 |
| *** gvincent_ has joined #ara | 08:04 | |
| *** gvincent_ has quit IRC | 08:05 | |
| *** gvincent_ has joined #ara | 08:05 | |
| *** gvincent has quit IRC | 08:07 | |
| Naphta | Scratch that, our default settings for nginx had X-Frame-Options deny. Just looking around and is there any way to tell the web client that it's running under https? Getting mixed content as I'm terminating the HTTPS at a proxy server but the html has http links. Trying to avoid a rewrite tbh. | 08:09 |
| *** jparrill has quit IRC | 08:31 | |
| *** jparrill has joined #ara | 08:32 | |
| *** jlozadad_ has quit IRC | 08:56 | |
| openstackgerrit | Tristan Cacqueray proposed openstack/ara master: Prevent exception when LOG_FILE is empty https://review.openstack.org/574669 | 09:08 |
| *** gvincent_ has quit IRC | 11:36 | |
| *** gvincent has joined #ara | 11:36 | |
| dmsimard | Naphta: are you using the embedded web server ? | 11:53 |
| dmsimard | Naphta: flask exposes a PREFERRED_URL_SCHEME variable: http://flask.pocoo.org/docs/1.0/config/#PREFERRED_URL_SCHEME | 11:56 |
| dmsimard | However ARA doesn't set it up. If you'd like to send a patch I can point you in the right direction :) | 11:56 |
| *** rvgate has joined #ara | 12:17 | |
| *** dyasny has quit IRC | 12:47 | |
| hiroki | Naphta: this is a known bug: https://ptpb.pw/HrCv | 13:09 |
| hiroki | ARA (embedded webserver) doesn't add a trailing slash for /file/* URL's, therefore breaking a reverse HTTPS proxy you might have put in front of ARA | 13:09 |
| hiroki | All other links work well, it's just /file/* URL's which fail, because of the missing trailing slash | 13:10 |
| *** tbielawa has joined #ara | 13:15 | |
| *** themurph_ has joined #ara | 13:23 | |
| SpamapS | dmsimard: have you seen any bugs that manifest as play records not getting inserted? | 13:31 |
| dmsimard | SpamapS: I have not | 13:36 |
| dmsimard | hiroki: that's interesting | 13:38 |
| hiroki | dmsimard: it only happens when you request files. any other links are fine and -do- get the trailing slash appended | 13:40 |
| dmsimard | hiroki: files are not particularly different than other types of content being loaded into modals, let me check | 13:41 |
| dmsimard | hiroki: facts and results work just fine ? | 13:44 |
| *** Bakey has joined #ara | 13:45 | |
| hiroki | dmsimard: yes, if i click on hosts and then a host link for instance, works fine. If i click on the main playbook file (in my case site.yml) on the left, it works fine. If i click on a clickable link in tasks, it works fine. As soon as I click on a file, it does not work | 13:46 |
| dmsimard | hiroki: ok so notice how there is a trailing slash on the data-load item here: https://github.com/openstack/ara/blob/master/ara/templates/report.html#L20 | 13:47 |
| hiroki | Yeah, I see the trailing slash | 13:47 |
| dmsimard | sec | 13:48 |
| dmsimard | hiroki: if you click on a module name in the task list, does that work ? | 13:49 |
| hiroki | You can spot the difference if you click on site.yml on the left, have the pop-up frame come up, and hover over the permanent link file top right. It shows a trailing slash. Repeat for a file under files, it doesn't show a trailing slash there | 13:49 |
| dmsimard | hiroki: the link that opens up the file with the line highlighted | 13:49 |
| hiroki | dmsimard: under files or under tasks ? | 13:50 |
| dmsimard | hiroki: tasks | 13:50 |
| hiroki | dmsimard: it opens up the site.yml (we include individual task files from site.yml, so i'm guessing that's why) | 13:50 |
| dmsimard | hiroki: ok, but it works 6 | 13:50 |
| dmsimard | ? | 13:50 |
| hiroki | Yes it does | 13:50 |
| dmsimard | hiroki: ok so it's really just the file tab then ? | 13:51 |
| hiroki | Oh actually it shows the actual task file for included tasks, so anything under tasks works well. Perhaps because it adds /#line-<number> at the end of the link ? | 13:52 |
| hiroki | Yes exactly, just the file tab | 13:52 |
| dmsimard | I'm narrowing in on the issue, hang on | 13:53 |
| hiroki | sure :) | 13:53 |
| *** tbielawa is now known as tbielawa|brb | 13:56 | |
| SpamapS | dmsimard: thanks, I'll keep debugging | 13:59 |
| dmsimard | hiroki: want to try a patch ? | 14:04 |
| hiroki | Sure, I can probably try it out later today | 14:05 |
| dmsimard | ok, hang on, just confirming I didn't break something when fixing that | 14:08 |
| *** tbielawa|brb is now known as tbielawa | 14:08 | |
| openstackgerrit | David Moreau Simard proposed openstack/ara master: Add a trailing slash to file links in the file tab https://review.openstack.org/574739 | 14:12 |
| dmsimard | hiroki: ^ | 14:12 |
| dmsimard | in patch format: https://git.openstack.org/cgit/openstack/ara/patch/?id=56582b261624e40926a392bc53454f197f1d3659 | 14:12 |
| hiroki | dmsimard: tested, works like a charm, thank you ! | 14:16 |
| dmsimard | hiroki: sweet | 14:16 |
| dmsimard | Naphta: so hopefully that should also work for you ^, I'll land it in 0.15.1. | 14:17 |
| dmsimard | hiroki: so much for trying it out later today ? :D | 14:20 |
| hiroki | Well, I looked at the code change and it was a single line, that falls under my 90-second rule :P | 14:21 |
| dmsimard | ah, the famous 90 second rule | 14:23 |
| dmsimard | doesn't work for bread+peanut butter though | 14:23 |
| dmsimard | (that sounded funnier in my head) | 14:24 |
| hiroki | That's because you're doing it in reverse. Spreading peanut butter on bread takes too much time. Wrapping the bread around the peanut butter is faster and chunkier | 14:26 |
| dmsimard | haha, I meant how you have N seconds to pick it up from the floor | 14:27 |
| dmsimard | (and how it always falls face down) | 14:28 |
| hiroki | Ah ! Well, I come from Africa, so we actually adhere to the 2 week rule | 14:31 |
| dmsimard | o_O | 14:32 |
| *** DrWaluigi has quit IRC | 14:55 | |
| *** DrWaluigi has joined #ara | 15:00 | |
| *** DrWaluigi has quit IRC | 16:05 | |
| *** DrWaluigi has joined #ara | 16:07 | |
| *** jlozadad[m] has quit IRC | 16:08 | |
| ara-slack | <harlowja> @dmsimard have u seen anything where a playbook failure doesn't get recorded | 16:18 |
| ara-slack | <harlowja> clint and i have seen this happen a couple of times | 16:19 |
| ara-slack | <harlowja> wonder if u recall anything like that | 16:19 |
| ara-slack | <dmsimard> @harlowja if the playbook outright fails to even start (i.e, a loading/syntax/parse failure), there won't be a record for it | 16:20 |
| ara-slack | <dmsimard> It's likely that the same applies for plays | 16:20 |
| ara-slack | <dmsimard> That's because it fails so early in the process that there's no hook sent to the ara callback | 16:21 |
| ara-slack | <harlowja> hmmmm, i'll see if we can get some kind of reproducible case | 16:22 |
| ara-slack | <harlowja> it seems to have actually started | 16:22 |
| *** jlozadad[m] has joined #ara | 16:37 | |
| ara-slack | <harlowja> but something odd happens; will see if can get some more details | 16:37 |
| ara-slack | <dmsimard> cool thanks | 16:50 |
| SpamapS | Yeah we have task_result records and task records | 16:55 |
| SpamapS | but no play record | 16:55 |
| dmsimard | SpamapS: that's weird. Any stack trace in the raw stdout/stderr ? | 16:56 |
| *** rvgate has quit IRC | 16:56 | |
| *** tbielawa is now known as tbielawa|lunch | 16:59 | |
| SpamapS | dmsimard: nothing that we saw, but I"ll look for that | 17:13 |
| *** tbielawa|lunch is now known as tbielawa|lunchMt | 17:31 | |
| *** tbielawa|lunchMt is now known as tbielawa | 18:34 | |
| *** tbielawa is now known as tbielawa|brb | 18:53 | |
| *** tbielawa|brb is now known as tbielawa | 19:04 | |
| *** pmdharris has joined #ara | 19:36 | |
| pmdharris | dmsimard: /msg NickServ VERIFY REGISTER pmdharris tyzehnuvayfw | 19:37 |
| pmdharris | i'm awesome. stand back or you might get splashed by the awesome :) | 19:37 |
| dmsimard | pmdharris:oops | 19:37 |
| dmsimard | pmdharris: that was logged, make sure that no longer works or something like that | 19:38 |
| pmdharris | dmsimard: Yesterday I mentioned some UTF-8 warnings; I've tracked it down somewhat (just in case this is useful to you); setting binary_prefix to True in the MySQL connector gets rid of the errors. In my /usr/local/lib/python3.6/site-packages/pymysql/connections.py , line 584, def __init__() | 19:38 |
| pmdharris | dmsimard: (I haven't yet found a way to pass that down through SQLAlchemy though, so it looks like it's a problem there) | 19:39 |
| pmdharris | dmsimard: Just FYI | 19:39 |
| dmsimard | pmdharris: oh, neat, let me know if you manage to find out how to plumb it all the way | 19:39 |
| pmdharris | dmsimard: will do :) | 19:39 |
| *** pmdharris has quit IRC | 19:40 | |
| *** tbielawa has quit IRC | 20:30 | |
| *** jlozadad[m] has quit IRC | 20:49 | |
| *** jlozadad[m] has joined #ara | 21:10 | |
| *** evrardjp_ has joined #ara | 21:41 | |
| *** evrardjp has quit IRC | 21:43 | |
| *** myoung is now known as myoung|bbl | 22:07 | |
| *** Bakey has quit IRC | 23:02 | |
| *** gvincent_ has joined #ara | 23:17 | |
| *** jlozadad[m] has quit IRC | 23:17 | |
| *** gvincent has quit IRC | 23:17 | |
| *** jlozadad[m] has joined #ara | 23:37 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!