| *** ministry is now known as __ministry | 02:13 | |
| *** ysandeep|out is now known as ysandeep | 05:10 | |
| *** bhagyashris is now known as bhagyashris|ruck | 05:33 | |
| *** jpena|off is now known as jpena | 06:59 | |
| *** ysandeep is now known as ysandeep|lunch | 07:09 | |
| opendevreview | daniel.pawlik proposed openstack/ci-log-processing master: Improve logsender tool https://review.opendev.org/c/openstack/ci-log-processing/+/835419 | 07:46 |
|---|---|---|
| *** ysandeep|lunch is now known as ysandeep | 09:11 | |
| opendevreview | Merged openstack/ci-log-processing master: Improve logsender tool https://review.opendev.org/c/openstack/ci-log-processing/+/835419 | 09:27 |
| icey | does anybody know if it's possible to get the tox bindep handling to add a PPA? | 09:27 |
| icey | it looks like the answer will be no, as the unittest pre step just runs the bindep role, which doesn't do anything except system dependencies | 09:29 |
| fungi | icey: well, also bindep itself doesn't install things, it's just a mechanism for listing required packages and checking to see whether they're installed. how you install those packages and where you install them from is outside the scope of bindep itself. the current ansible role we use there just assumes they'll be available to the package manager on the system (whatever package | 11:12 |
| fungi | manager that is), so if you need additional package repositories added to the system you'd want to express that somehow outside bindep.txt and preempt the existing role with some tasks to add the additional package repos you'll need | 11:12 |
| *** dviroel|out is now known as dviroel|rover | 11:14 | |
| icey | fungi: yeah, that was what I was leaning towards - thank you for explaining the details :) | 11:58 |
| *** ysandeep is now known as ysandeep|afk | 11:58 | |
| fungi | yw | 12:04 |
| opendevreview | Dr. Jens Harbott proposed openstack/project-config master: Start bulding ubuntu-jammy images https://review.opendev.org/c/openstack/project-config/+/839057 | 12:49 |
| *** dasm|off is now known as dasm | 13:16 | |
| *** ysandeep|afk is now known as ysandeep | 13:16 | |
| *** Guest2246 is now known as prometheanfire | 13:54 | |
| *** ysandeep is now known as ysandeep|out | 14:43 | |
| *** dviroel|rover is now known as dviroel|rover|lunch | 15:09 | |
| dpawlik | dansmith: hey, by doing: tags: "performance" you should be able to get informations in Opensearch for performance.json file | 15:11 |
| dansmith | dpawlik: something I wasn't clear about in the meeting yesterday, is logstash.o.o the "new" one now? | 15:12 |
| dpawlik | https://opensearch.logs.openstack.org/_dashboards/app/discover/?security_tenant=global this one is a new one | 15:12 |
| dpawlik | https://docs.openstack.org/project-team-guide/testing.html#checking-status-of-other-job-results | 15:13 |
| dansmith | ah, I see, .. different | 15:13 |
| dansmith | so, cool, I tags:performance works | 15:15 |
| *** akekane_ is now known as abhishekk | 15:15 | |
| dansmith | I was thinking clarkb was saying that this would be digested not just as a message string, but so that we can query and filter on the actual values in there | 15:15 |
| dansmith | but maybe I misunderstood? | 15:15 |
| dansmith | clarkb: is the json file in as a string what we need to use it a a source for graphing? | 15:16 |
| dpawlik | dansmith: I would not to proceed the json in the logsender.... the service is doing very simple things, but it got a lot work to do | 15:18 |
| dpawlik | maybe some script that will query the opensearch, parse the json and maybe push into other index? | 15:19 |
| dpawlik | s/other/another/ | 15:19 |
| clarkb | each elasticsearch document is a json blob | 15:19 |
| clarkb | in the log case we construct a json blob with a mesage and time and priority etc | 15:19 |
| clarkb | in the case of dansmith's data we'd send it in as is to a different index so that we don't pollute the log data with the metric data | 15:20 |
| clarkb | then you can query service.foo.memory or whatever the actual path is and tools like grafana can graph all the resulting data points | 15:20 |
| dansmith | clarkb: right so this is just a document with message: $performance_json_string | 15:20 |
| clarkb | dansmith: no that makes it hard/impossible to query | 15:20 |
| dansmith | okay, right that's why I was surprised to see this.. I didn't expect this to be workable | 15:21 |
| clarkb | the log messages are sent like { message: "I'm a log", timestamp: "20220101T12:12:12.123", priority: "INFO"} | 15:21 |
| dansmith | yeah | 15:21 |
| clarkb | then you can query elasticsearch for all mesasges matching "a" or whatever | 15:21 |
| clarkb | for this data you would send { "service": stuff, "db": stuff} and then you can just say give me all the service nova memory records over a threshold for this time period | 15:22 |
| dansmith | and would the value for 'service" be a json string of the stuff under there? | 15:23 |
| clarkb | dansmith: no everything between the {} is your raw json document that you ar ealready producing | 15:23 |
| clarkb | we would send it as is. We might need to add a timestamp and a job name now that I think of it though | 15:24 |
| clarkb | but I think your generated info is largely the actual document we want ot store in the database | 15:24 |
| dansmith | clarkb: okay I wasn't sure if you mean the json doc itself, or pick off the top-level bits and stringify the contents underneath, but yeah, full ingest is what I was expecting | 15:25 |
| clarkb | ya if we stringify them then the records become less serachable | 15:25 |
| clarkb | with logs we accept that because logs are inteded to be human readable first | 15:26 |
| clarkb | but with metrics like this the document should be stored largely as is | 15:26 |
| fungi | though structured logs are also a thing these days | 15:26 |
| dansmith | right, since this is just a string of json, I thought maybe I was misunderstanding what you were proposing, but sounds like you're expecting us to get the actual json doc in as it is so we can query individual values | 15:26 |
| sean-k-mooney | i mean oslo support exporting logs in json format unfortually there is nothign to rexport them in human readbale formant | 15:27 |
| dpawlik | it might create a many fields in the index that will be not used | 15:27 |
| sean-k-mooney | we have one downstream custoemr that use json fromated logs in there env and i hate trying to debug thigns when they file bugs | 15:27 |
| dansmith | that's not the topic here.. just using logs as an example | 15:27 |
| clarkb | dansmith: yes exactly | 15:27 |
| clarkb | dpawlik: possibly though dansmith has identified useful info and is distilling it all down into a single doc | 15:28 |
| clarkb | dpawlik: but also I would index these records in a new index not the log index | 15:28 |
| clarkb | then people who query for logs won't also see the metrics and vice versa. You would query the index you want depending on the info you want | 15:28 |
| clarkb | thi shas the additional advantange of allowing different pruning periods for the different data | 15:29 |
| dpawlik | clarkb: right | 15:29 |
| dpawlik | what if create an other tool that will make a query, take the message and parse the json? | 15:30 |
| clarkb | ya that would probably be fine too | 15:30 |
| dpawlik | so the logsender does not need to handle that and we can specify the endpoint whatever it is | 15:30 |
| clarkb | the important bit here I htink is to not treat this as log data but as metrics stored in a slightly different way in the db. If that implies a different ingest tool that is fine | 15:31 |
| dpawlik | I will think during the weekend what will be the best way. Let's talk about it on monday | 15:33 |
| dpawlik | enjoy your weekend | 15:34 |
| dansmith | thanks dpawlik ! | 15:35 |
| clarkb | I know that you've said you don't want to use logstash or that it isn't necessary but this is where a tool like logstash can be helpful | 15:35 |
| clarkb | rathe rthan need to rewrite a tool you just update a small config file to say "input of this form gets treated this way" | 15:35 |
| *** jpena is now known as jpena|off | 15:55 | |
| *** dviroel|rover|lunch is now known as dviroel|rover | 16:14 | |
| opendevreview | Luciano Lo Giudice proposed openstack/project-config master: Add the cinder-three-par to Openstack charms https://review.opendev.org/c/openstack/project-config/+/837782 | 16:37 |
| *** dviroel|rover is now known as dviroel|rover|biab | 20:31 | |
| *** dviroel|rover|biab is now known as dviroel|rover | 21:02 | |
| opendevreview | Gage Hugo proposed openstack/project-config master: End project gating for openstack-helm-docs https://review.opendev.org/c/openstack/project-config/+/839103 | 21:10 |
| *** dviroel|rover is now known as dviroel|rover|biab | 21:16 | |
| *** dasm is now known as dasm|off | 22:03 | |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!