nathang15 | Hi everyone, I wonder if it is normal if only the TestReplicatorFunctions within probe tests fails? I did not modify any code and I tried running resetswift, reboot my pc but it still fails. Every other tests passed, some were skipped though. Here is the output of my probe tests: https://paste.opendev.org/show/bAkeq8da8gwLhL9rZClY/ | 02:33 |
---|---|---|
nathang15 | Thank you | 02:34 |
opendevreview | Merged openstack/swift feature/mpu: mpu: add x-backend-etag-is-at header to GET/HEAD requests https://review.opendev.org/c/openstack/swift/+/933932 | 02:45 |
timburke | nathang15, is this off master, or some change you're working on? i'd expect the tests to all pass... | 03:12 |
timburke | have you tried running that test in isolation? `pytest test_replication_servers_working.py::TestReplicatorFunctions::test_main` should target just the one. then it's easier to check logs after the failure -- though off hand, i'm not sure what could have caused the failure | 03:13 |
nathang15 | Yep this is off master, maybe I'll try stable 2024.2 | 03:20 |
nathang15 | I did not work on any changes yet | 03:20 |
opendevreview | Matthew Oliver proposed openstack/swift master: Configurable expiring_objects_task_container_per_day https://review.opendev.org/c/openstack/swift/+/933373 | 04:13 |
mattoliver | nathang15: it seems to indicate that the directory is still there. If you just run that probe test (as Tim showed), it should leave it in the state it failed. and we can look at the directory and try and figure out why it wasn't removed. | 04:34 |
opendevreview | Matthew Oliver proposed openstack/swift master: db_auditor: add vacuum support https://review.opendev.org/c/openstack/swift/+/916861 | 06:31 |
mattoliver | ^ just moving the guage definition to somewhere else in the file to stop a downstream conflict / collision. fun times | 06:32 |
opendevreview | Jianjian Huo proposed openstack/swift master: Object-server: add periodic greenthread yielding during file write https://review.opendev.org/c/openstack/swift/+/934200 | 06:49 |
opendevreview | Matthew Oliver proposed openstack/swift master: db_auditor: add vacuum support https://review.opendev.org/c/openstack/swift/+/916861 | 08:31 |
opendevreview | Matthew Oliver proposed openstack/swift master: add statds guage and use it for auditor vacuum sizes https://review.opendev.org/c/openstack/swift/+/934206 | 08:31 |
opendevreview | Merged openstack/swift feature/mpu: mpu: fix param passing to sub requests https://review.opendev.org/c/openstack/swift/+/933933 | 12:31 |
opendevreview | Alistair Coles proposed openstack/swift master: reconciler: Record queue-clean-up later than old-policy-clean-up https://review.opendev.org/c/openstack/swift/+/933683 | 14:33 |
opendevreview | Yan Xiao proposed openstack/swift master: proxy-logging: Add real-time transfer bytes counters https://review.opendev.org/c/openstack/swift/+/930918 | 14:59 |
opendevreview | Merged openstack/swift feature/mpu: mpu: change some response codes to keep s3api happy https://review.opendev.org/c/openstack/swift/+/933934 | 16:29 |
opendevreview | Alistair Coles proposed openstack/swift feature/mpu: mpu-auditor: audit sessions https://review.opendev.org/c/openstack/swift/+/931517 | 16:40 |
opendevreview | Merged openstack/swift master: Prevent installation with pyeclib 1.6.2 and 1.6.3 https://review.opendev.org/c/openstack/swift/+/933203 | 18:45 |
opendevreview | ASHWIN A NAIR proposed openstack/swift master: s3api: improve error handling for part-num req's https://review.opendev.org/c/openstack/swift/+/934154 | 18:48 |
opendevreview | ASHWIN A NAIR proposed openstack/swift master: s3api: improve error handling for part-num req's https://review.opendev.org/c/openstack/swift/+/934154 | 19:06 |
opendevreview | Tim Burke proposed openstack/swift master: utils: Add crc32c function https://review.opendev.org/c/openstack/swift/+/909800 | 20:53 |
opendevreview | Tim Burke proposed openstack/swift master: s3api: Add support for additional checksums https://review.opendev.org/c/openstack/swift/+/909801 | 20:53 |
mattoliver | Morning | 21:02 |
timburke | oh, right! | 21:04 |
timburke | #startmeeting swift | 21:04 |
opendevmeet | Meeting started Wed Nov 6 21:04:13 2024 UTC and is due to finish in 60 minutes. The chair is timburke. Information about MeetBot at http://wiki.debian.org/MeetBot. | 21:04 |
opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 21:04 |
opendevmeet | The meeting name has been set to 'swift' | 21:04 |
timburke | who's here for the swift team meeting? | 21:04 |
mattoliver | o/ | 21:04 |
nathang15 | hi everyone, this is my first time joining the meeting | 21:05 |
timburke | hi nathang15! | 21:05 |
boosungkim-dson | Hello, I am Boosung from Dickinson College. | 21:05 |
jamesng | Hi everyone, I'm James, from Dickinson College. This is my first time either. | 21:05 |
timburke | sorry i wasn't around longer to help debug your probe test issue yesterday -- i had to get my kids down for bed :-) | 21:05 |
nathang15 | No worries, I somehow fixed it :) | 21:06 |
timburke | hi boosungkim-dson, jamesng! | 21:06 |
nathang15 | all the tests passed now | 21:06 |
timburke | nathang15, oh good! i wonder what the issue was | 21:06 |
timburke | eh, as long as it works now, i suppose :-) | 21:06 |
fulecorafa | Hey everbody | 21:07 |
nathang15 | Yeah, so I head over to install vsaio, and the tests for saio passed after I finished setting up vsaio. | 21:07 |
nathang15 | So I think this might be some issue with my local environment | 21:07 |
mattoliver | Nice | 21:07 |
timburke | fulecorafa, o/ | 21:07 |
mattoliver | Some race condition.. keep and eye on it, and if it happens again might be nice to know what it was :) | 21:08 |
nathang15 | Yep, I'll keep an eye on it | 21:08 |
timburke | i suppose it could have been the random chance to re-hash... not sure how best to write the test to avoid that, though | 21:08 |
timburke | https://github.com/openstack/swift/blob/2.34.0/swift/obj/replicator.py#L666-L668 | 21:09 |
timburke | i'm afraid i haven't updated the agenda | 21:10 |
timburke | but i *did* finally get around to digging into https://bugs.launchpad.net/swift/+bug/2028175 | 21:10 |
patch-bot | Bug #2028175 - intermittent probe test failure: test_reconciler_move_object_twice (In Progress) | 21:10 |
timburke | and i think i have a fix | 21:11 |
timburke | #link https://review.opendev.org/c/openstack/swift/+/933683 | 21:11 |
patch-bot | patch 933683 - swift - reconciler: Record queue-clean-up later than old-p... - 2 patch sets | 21:11 |
mattoliver | Nice | 21:11 |
timburke | i wanted to be sure to bring it up since it changed how the reconciler bumps timestamp offsets, which i don't think has ever changed since we introduced storage policies | 21:12 |
timburke | er, changes* -- not actually merged yet, though acoles likes the look of it | 21:13 |
mattoliver | thanks for going to the effort of groking the whole reconciler offset stuff! | 21:13 |
mattoliver | I'll take a look today and see if we can get it over the line | 21:13 |
timburke | thanks mattoliver! | 21:14 |
timburke | in related gate-health news, i think i managed to nerd-snipe someone into looking at https://bugs.launchpad.net/openstacksdk/+bug/2085654 -- so hopefully those openstacksdk-functional-devstack failures we've been seeing will be fewer and further between sometime soon 🤞 | 21:15 |
patch-bot | Bug #2085654 - Two intermittent functional test failures (Confirmed) | 21:15 |
mattoliver | nerd-snipe for the win.. and it wasn't me! so extra good news :P | 21:16 |
timburke | the one other top-of-my-head topic i've got is on https://review.opendev.org/c/openstack/swift/+/933697 | 21:17 |
patch-bot | patch 933697 - swift - Pull out a bunch of liberasurecode requirements - 3 patch sets | 21:17 |
mattoliver | oh cool, and pete has already +2ed it | 21:18 |
timburke | now that pyeclib is publishing binary wheels, we have an opportunity to remove some of the liberasurecode stuff from the swift repo, since we won't need to build pyeclib from sdists | 21:18 |
timburke | note that it doesn't remove the need for a build environment entirely (though that would be a nice end-goal) since various other entries in test-requirements.txt pull in deps (i remember seeing netifaces at least) that still need to be built from source | 21:21 |
mattoliver | kk, might be a nice side project or task for someone. | 21:22 |
timburke | it also doesn't change anything for https://docs.openstack.org/swift/latest/development_saio.html -- though maybe we could? | 21:22 |
mattoliver | ideally we shouhld probably build liberasurecode manually in SAIO seeing as it is one of our projects we maintain.. and so having the build env there is probably a good thing | 21:23 |
timburke | figured i'd see what people thought about how integral it is to easily hack on pyeclib in an SAIO | 21:23 |
mattoliver | but maybe I only say that because I've been looking at it lately :P | 21:24 |
timburke | yeah, the "it is one of our projects we maintain" is an excellent point -- i was thinking maybe we could treat pyeclib/liberasurecode something like how we treat eventlet, but maybe that's the big dividing line | 21:25 |
mattoliver | I guess we could have a developement_pyeclib and link to that.. but if a SAIO is a main dev env, might be nice to put it in there too. | 21:26 |
timburke | not changing things is pretty cheap & easy :-) we'll keep it as it is | 21:27 |
mattoliver | to build it, it's only installing the compiler tools and make (build-essential for ubuntu or "Development tools" in fedora/centos) then a make, make install. | 21:28 |
mattoliver | sounds like another side task.. we should write these down somewhere.. on the ideas page maybe in a new section. like project low hanging fruit :) | 21:28 |
timburke | huh, interesting... looks like pyeclib is only mentioned on the SAIO page to install it from system packages on Fedora, while liberasurecode-dev(el) is mentioned for all of them (presumably because you used to need it to install pyeclib from pypi) | 21:30 |
mattoliver | no wonder we never show pyeclib no love :( | 21:31 |
timburke | writing up bugs on launchpad and tagging it "low-hanging-fruit" seems like a reasonable plan | 21:32 |
mattoliver | kk | 21:32 |
timburke | one or two other things on my mind (since we've got the Dickinson College folks) | 21:33 |
timburke | first, (carrying on from thinking about low-hanging-fruit) what projects might be nice for them to work on? | 21:34 |
boosungkim-dson | Yeah, we just finished our presentation and small demo and are looking to start contributing soon! | 21:35 |
timburke | and second, do y'all have any questions about swift already? would it be nice to have an overview of the architecture or anything? | 21:35 |
timburke | nice! what was the presentation about? | 21:35 |
boosungkim-dson | The presentation was about introducing swift to our classmates, swift architecture, and a small swift client application we made (upload/download/auth). | 21:36 |
zaitcev | nice | 21:36 |
zaitcev | hopefully better than this https://www.youtube.com/watch?v=lSUSZ8xGF9o | 21:37 |
mattoliver | hey zaitcev o/ | 21:37 |
timburke | you did great, zaitcev! | 21:37 |
zaitcev | thanks | 21:37 |
boosungkim-dson | We watched that video to prepare for our presentation! | 21:38 |
zaitcev | But boosungkim-dson had a demo! | 21:38 |
zaitcev | Oh, really? | 21:38 |
boosungkim-dson | We watched a few reference videos to prepare | 21:38 |
mattoliver | did we want to organise a jitsi (meetpad) video chat sometime soon if that's an easier medium to ask questions in? worked well for the qatar students. I found them useful for the qatar students as they did their projects. | 21:39 |
boosungkim-dson | Yes, that would be very helpful! | 21:39 |
mattoliver | maybe we can find some smaller low langing fruit to wet their appitites and then find something more meaty for them to really dig into. So they can start getting used to the gerrit and openstack dev workflows. | 21:40 |
opendevreview | ASHWIN A NAIR proposed openstack/swift master: s3api: improve s3 sigv4 error responses https://review.opendev.org/c/openstack/swift/+/934087 | 21:40 |
timburke | when would be a good time for you guys? i might recommend some afternoon M-Th; then we could try to have both mattoliver and i there | 21:41 |
mattoliver | kk. how about I set up an etherpad (that we can link to any future meetpad sync meetings) and I'll seed it with some ideas. | 21:41 |
mattoliver | +1 | 21:41 |
timburke | if you wanted any more reference videos, https://www.youtube.com/@OpenInfraFoundation/search?query=swift has a bunch of good talks from years past | 21:42 |
boosungkim-dson | Thank you! | 21:43 |
opendevreview | ASHWIN A NAIR proposed openstack/swift master: s3api: improve error handling for part-num requests https://review.opendev.org/c/openstack/swift/+/934154 | 21:43 |
mattoliver | I can also do a tour with the tracing code so you can watch some paths.. although there probably already is a talk about that already (this is a bit old, but can do a live one with newer code) https://www.youtube.com/watch?v=iP_qy8KKDno | 21:44 |
mattoliver | The code has mostly compeltely changed from there, because it's now all open-telemtry and using a decorator implementation rather then mixins, but you get so see some traces :) | 21:45 |
timburke | another fun thing to do is look at extending swift with your own custom middleware -- teaches you a good bit about how the WSGI pipeline works | 21:45 |
nathang15 | awesome thank you timburke, we also read through the swift book so we have a general idea about the architecture. Anyway, Monday 7PM, Tuesday 5PM, and Wednesday 5PM EST works for us | 21:45 |
mattoliver | yeah! | 21:46 |
timburke | mattoliver, have any preference? for reference, 5PM EST is ~12mins away | 21:48 |
mattoliver | oh that's 9am this time of year. thats not bad at all! | 21:49 |
timburke | i think any of those would work well for me | 21:49 |
mattoliver | so for me, that's 11am Tues, 9am weds or 9 am thurs. I can make them work. I do have a downstream meeting at Tues 9am EST.. so maybe one of the other 2? | 21:50 |
mattoliver | but I run it, so can always move it ;) | 21:50 |
timburke | and it rarely goes two hours ;-P | 21:51 |
mattoliver | oh no i dont. that's monday 5pm EST (man timezones are hard) | 21:51 |
mattoliver | yup they;re all ok for me too! | 21:51 |
mattoliver | Whatever is easiest for the students. would you rather a 5pm or 7pm? :P | 21:52 |
nathang15 | Monday 7pm works best for us, thank you | 21:53 |
timburke | sounds good! mattoliver will work on getting an etherpad/meetpad seeded, and we'll meet monday 👍 | 21:54 |
boosungkim-dson | Sounds great! | 21:54 |
mattoliver | great! let's lock that in! | 21:54 |
mattoliver | I'll paste a link to the ehterpad/meetpad here when we create it. | 21:54 |
timburke | thanks mattoliver | 21:55 |
timburke | all right, we're running a little low on time -- anything else we ought to discuss this week? | 21:55 |
boosungkim-dson | I have a quick question! | 21:55 |
timburke | go ahead! | 21:56 |
boosungkim-dson | I am also trying out the vagrant install, but I am noticing that not all the functional tests pass for me, and the probe tests are really really slow (over an hour). | 21:56 |
boosungkim-dson | Not sure what I exactly did wrong though... | 21:56 |
timburke | did you create your own localrc based on https://github.com/NVIDIA/vagrant-swift-all-in-one/blob/master/localrc-template? i feel like the default VAGRANT_CPUS=1 is probably too low these days... | 21:57 |
timburke | func test failures are a little surprising to me -- do you remember which ones failed? | 21:58 |
boosungkim-dson | Eh I just cloned the repo and did "vagrant up" | 21:58 |
timburke | makes sense -- we should probably also fix the default ;-) | 21:58 |
boosungkim-dson | I'm trying to get the terminal output history for the functional tests, since the probe functions are still running | 21:58 |
mattoliver | you can copy rclocal.template to rclocal. and modify it. Then just source rclocal then vagrant up. | 21:59 |
indianwhocodes | missed it :-/ | 21:59 |
timburke | no rush -- we can also help debug it more | 21:59 |
timburke | indianwhocodes, can always read scrollback ;-) | 21:59 |
mattoliver | yeah probe test take a while too because it calls resetswift inbetween and the vsaio's resetswift tis more involved | 22:00 |
mattoliver | I did rework the vacuum patch, and ended up splitting it into 2. one with the vacuum stuff and recon dumps to dump the bucket sizes and then a follow to add a statsd guage and start using it. I split it because of scope and because the guage stuff really conflicts with the current labeled metric work.. so thought it can be more of a follow up. | 22:00 |
boosungkim-dson | I will try that and see if that changes things, thank you mattoliver | 22:00 |
mattoliver | p https://review.opendev.org/c/openstack/swift/+/916861 | 22:00 |
patch-bot | patch 916861 - swift - db_auditor: add vacuum support - 13 patch sets | 22:00 |
mattoliver | and | 22:01 |
mattoliver | #link https://review.opendev.org/c/openstack/swift/+/934206 | 22:01 |
patch-bot | patch 934206 - swift - add statds guage and use it for auditor vacuum sizes - 1 patch set | 22:01 |
mattoliver | but that's all I got and we're at time | 22:01 |
timburke | (oh yeah, if anyone doesn't have an irc bouncer to help catch messages when you're not around, there's also https://meetings.opendev.org/irclogs/%23openstack-swift/ which has years of channel logs) | 22:02 |
timburke | you're right mattoliver -- thank you all for coming, and thank you for working on swift! | 22:02 |
mattoliver | oh good point. otherwise just run a bouncer, or a matrix client and connect with the irc bridge :) | 22:02 |
timburke | #endmeeting | 22:02 |
opendevmeet | Meeting ended Wed Nov 6 22:02:45 2024 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 22:02 |
opendevmeet | Minutes: https://meetings.opendev.org/meetings/swift/2024/swift.2024-11-06-21.04.html | 22:02 |
opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/swift/2024/swift.2024-11-06-21.04.txt | 22:02 |
opendevmeet | Log: https://meetings.opendev.org/meetings/swift/2024/swift.2024-11-06-21.04.log.html | 22:02 |
boosungkim-dson | Thank you, and see you on Monday! | 22:03 |
timburke | i figured it'd be good to mention since we have some new faces | 22:03 |
timburke | boosungkim-dson, nathang15, jamesng thanks for coming! looking forward to Monday! | 22:03 |
jamesng | See you on Monday. | 22:04 |
nathang15 | likewise, thank you! | 22:04 |
boosungkim-dson | Bye! | 22:04 |
opendevreview | Shreeya Deshpande proposed openstack/swift master: Remove PrefixLoggerAdapter and SwiftLoggerAdapter https://review.opendev.org/c/openstack/swift/+/931060 | 22:22 |
opendevreview | Tim Burke proposed openstack/swift master: s3api: Add support for additional checksums https://review.opendev.org/c/openstack/swift/+/909801 | 22:36 |
opendevreview | ASHWIN A NAIR proposed openstack/swift master: s3api: improve error handling for part-num requests https://review.opendev.org/c/openstack/swift/+/934154 | 23:15 |
opendevreview | ASHWIN A NAIR proposed openstack/swift master: s3api: error handling of complete mpu and part-num https://review.opendev.org/c/openstack/swift/+/934154 | 23:19 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!