| *** diablo_rojo has joined #openstack-tc | 00:21 | |
| *** tosky has quit IRC | 00:36 | |
| *** diablo_rojo has quit IRC | 00:51 | |
| mnaser | \o/ | 01:00 |
|---|---|---|
| mnaser | hi all | 01:01 |
| *** mriedem_away has quit IRC | 01:01 | |
| gmann | o/ | 01:02 |
| fungi | that time once more | 01:04 |
| fungi | it's also first contact sig meeting time in #openstack-upstream-institute | 01:05 |
| mnaser | i had a fun kinda-technical-but-maybe-not-so-tc topic | 01:05 |
| mnaser | http://lists.openstack.org/pipermail/openstack-discuss/2019-May/005956.html was a discussion that was had a while ago regarding upper-constraints | 01:06 |
| *** diablo_rojo has joined #openstack-tc | 01:06 | |
| mnaser | osa (as a deployment tool) relies on that, we have users that are having problems with buggy libraries in our upper constraints (amqp for example for ssl) | 01:06 |
| mnaser | in this case, we actually have a scenario where the upper constraints are an actual test of *non-functional* (if using ssl) dependencies | 01:07 |
| mnaser | from an osa perspective, it's nice that we can just slurp a file that we can maintain upstream (and that perhaps others can refer to to get a working set of constraints) | 01:07 |
| mnaser | i dont think it would be productive if we maintained our own.. | 01:07 |
| *** lbragstad has quit IRC | 01:08 | |
| fungi | my take is that users should not be relying on the security of pip installing dependencies of our stable branches. we're not a linux distro, we don't get advance warning of pending security advisories, our ability to update them at all for stable branches is iffy at best, and evenif we can the delay in doing so means deployments sit vulnerable for faaar too long | 01:09 |
| mnaser | it does seem like it would be a productive thing for this type of thing to be shared across deployment tools / packagers.. | 01:09 |
| fungi | they should get dependencies from distro packages | 01:09 |
| mnaser | you cant install those in a virtual environment :( | 01:10 |
| fungi | sounds like virtualenvs are a security risk then | 01:10 |
| fungi | (but also you can expose them into virtualenvs if system site packages are enabled when the env is built) | 01:11 |
| mnaser | yeah, i figured as much.. but i don't know if that would be easy to accomplish | 01:11 |
| fungi | i have no idea | 01:12 |
| mnaser | i.e. install all these python{2,3}-{foo} things and then create a virtualenv and install these packages from system site packges, i see wher you're coming from though | 01:12 |
| mnaser | it does mean if something breaks we are at the mercy of distros | 01:12 |
| fungi | the upper-constraints.txt is an artifact of how we test newer versions of software than our target distros package. we freeze them when we branch so that we reduce unanticipated test regressions without having to switch to completely different test models for releases once they branch | 01:14 |
| fungi | i don't think we should be encouraging production deployments of stable branches using packages versions listed in our frozen upper-constraints.txt files since we're never going to be able to make that anywhere close to as secure as a security-supported linux distro | 01:15 |
| mnaser | but do linux distros ship anything newer than upper constraints? | 01:16 |
| fungi | they backport security fixes to those things yes, and generally have those backported fixes applied to coincide with the corresponding security advisory publication dates | 01:16 |
| mnaser | what about things that aren't security issues but actual bugs where the library is simply broken | 01:17 |
| mnaser | i.e. ssl + amqp is just plain broken | 01:17 |
| fungi | broken such that our tests stop working? | 01:18 |
| mnaser | context: https://bugs.launchpad.net/oslo.messaging/+bug/1800957/ | 01:19 |
| openstack | Launchpad bug 1800957 in oslo.messaging (Ubuntu) "Upgrading to pike version causes rabbit timeouts with ssl" [Undecided,New] | 01:19 |
| fungi | updating entries in upper-constraints.txt to better stabilize the tests it's designed to stabilize makes some sense, yes | 01:19 |
| mnaser | i assume there was just not a test that caught that | 01:20 |
| fungi | if there's no test then the constraints not providing stability to that nonexistent test seems irrelevant | 01:20 |
| mnaser | i guess the issue here was that there was no test that checked for this (but we promise ssl support in oslo.messaging), a bug was caught, a fix was done, but.. we still don't test what someone might actually be running in prod | 01:21 |
| mnaser | i guess i view upper-constraints differently | 01:21 |
| fungi | again, the upper-constraints.txt file is there to stabilize our testing. reusing it for other purposes and then asking to make it less useful for the purpose it was designed for seems like a bad idea | 01:21 |
| mnaser | it just seems to be like if OSA starts maintaing its own set of constraints, we're just going to start duplicating efforts across projets | 01:23 |
| mnaser | i think kolla even builds containers using upper-constraints too.. i htink | 01:23 |
| fungi | it's quite good at stabilizing tests, which is why we chose that solution. it's really not that good for managing dependencies of a production deployment (for a variety of reasons, security vulnerabilities chief amongst those). making it less effective at stabilizing tests so that people can use it in other dangerous ways isn't something i can get behind | 01:24 |
| mnaser | i'm just trying to visualize the ideal solution for this which doesn't create a lot of duplicate work across deployment tooling | 01:25 |
| fungi | i honestly don't see making your own distro out of a constant churn of packages from pypi as anything other an sisyphean. if you want to talk about duplicating effort, what you're actually describing is kolla and osa duplicating the same work well-funded and staffed linux distros already do, but very, very poorly | 01:26 |
| fungi | there is a reason they freeze package versions and then carefully try to backport critical fixes to those frozen versions rather than just constantly mixing in new releases of random upstream projects | 01:27 |
| fungi | and it's because the latter doesn't work for an lts model, full stop | 01:28 |
| mnaser | that's fair, i totally agree with you. | 01:28 |
| fungi | it works well enough in a rolling release model | 01:28 |
| fungi | which is why i think it's feasible for folks who want to deploy openstack from master | 01:28 |
| mnaser | it's just trying to balance the usability / usefulness of users who might want to make a change or two. and also, it's tough because distros are generally not good at running multiple versions of a library | 01:29 |
| mnaser | i.e. if i am doing a rolling upgrade and doing keystone first, i cant just have a 'virtualenv' or whatever that runs newer packages | 01:29 |
| mnaser | and openstack doesn't exactly test with those debs/rpms so it's just this inconsistency | 01:29 |
| fungi | i get the desire, and have empathy for users who wish that could be less complicated | 01:30 |
| mnaser | to me, the biggest value of virtualenv deploys is having something with all of its dependencies live in a little bucket to help make upgrades painless | 01:30 |
| mnaser | and maybe that isn't openstack's problem to solve.. but its there.. | 01:30 |
| fungi | i think it's entirely reasonable for us to want to solve it, but the problem is way bigger than openstack yes, and we're not the only people who want to work on solving that same problem | 01:31 |
| fungi | so i think the solution also needs to be bigger than openstack, and a much wider-reaching effort | 01:32 |
| clarkb | fwiw venvs dont require you use our constraints or constraints at all | 01:33 |
| clarkb | seema orthogonal to me tbh | 01:33 |
| mnaser | if you deploy stuff without constraints, you're likely going to end up with a broken system | 01:33 |
| mnaser | s/system/service/ | 01:33 |
| fungi | right, i gather the desire there is the usual contradictory one: you want something stable but with updates | 01:33 |
| fungi | more updates means less stable. more stable means less updates | 01:34 |
| mnaser | i can't disagree with that | 01:35 |
| fungi | the task of making updates as stable as possible is increasingly complicated the closer you try to drive that margin, and is why lts server distros invest sooooo much effort in that (it's really probably their greatest selling point, and they know it) | 01:35 |
| mnaser | i agree that they know what they're doing, but i'm just trying to figure out a way where we can use all those lts dependencies, but with upstream openstack. | 01:36 |
| mnaser | to me that's the hard thing to juggle | 01:36 |
| mnaser | and a lot of those lts operating systems put openstack in a special spot where it gets mucher newer packages | 01:37 |
| fungi | for our stable branch testing we opted for as stable as possible, which means no updates. production deployments need updates to fix untested bugs (including newly discovered security vulnerabilities) | 01:37 |
| mnaser | (i.e. UCA, RDO) | 01:37 |
| fungi | yeah, we move "too fast" for stable distros | 01:37 |
| mnaser | so the whole lts promise is pretty much not that useful, do a yum install openstack-nova and you'll find msot things are being pulled from rdo rather than base/updates | 01:38 |
| fungi | well, yes and no. i gather the "newer" packages in, say, the rocky version of rdo will be held basically unchanging for as long as red hat supports rocky | 01:39 |
| fungi | which may well be years | 01:40 |
| fungi | and that's attractive to a certain sort of businesses who dislike the idea of upgrading major versions of software ore often than every 3-4 years | 01:40 |
| fungi | the other thing we've said in the past, which deployment projects have more recently not understood, disagreed with, ignored, or something, is that our stable branches are a place for coordinating backports of critical fixes but are not intended as a comprehensive maintained distribution. the idea was that distributions would pick patches from the stable branches to integrate | 01:44 |
| fungi | so the stable branches as branches provide us a way to test all those backports together to make sure they're hopefully coherent and work as a whole | 01:45 |
| mnaser | well that puts OSA in a weird place i guess | 01:45 |
| fungi | but we weren't expecting stable branch deployments from source outside of using them to test future backports | 01:46 |
| mnaser | i guess it kinda implies we should maintain branches for all services as well | 01:46 |
| fungi | i do think that the fact some of our deployment projects have wanted to deploy stable branches from source (or that their users have asked them for that model) suggests that they may be more useful for other purposes than those which they were originally intended | 01:49 |
| fungi | but it's worth noting that's not a purpose they were designed for, and so the way in which we designed them may make them less effective at that as a result | 01:50 |
| mnaser | i mean i think i see a huge value in our stable branches being something you can deploy IMHO | 01:50 |
| fungi | it may be that we should change fundamental aspects of our stable branch maintenance and testing to better suit those use cases, but that means we also need to 1. be realistic about what we can actually promise even if we do, and 2. determine whether making them less effective at the things we originally designed them for is acceptable | 01:51 |
| mnaser | i think a lot of distros do a straight rebuild of stable branches (i.e. rdo afaik?) | 01:53 |
| fungi | it's also possible we could end up with a compromise like the linux kernel has: two kinds of stable series | 01:53 |
| fungi | yes, distros have generally gotten accustomed to just being able to slurp in everything in our stable branches *because* we have a fastidious approach to keeping them stable (not increasing dependencies in them, for example) | 01:54 |
| fungi | so if we choose to destabilize our stable branches to accommodate a source distribution model which demands updated deps, then distros may no longer be able to make the assumption they can directly consume our stable branches | 01:55 |
| fungi | and will have to go back to selectively importing patches from them | 01:56 |
| fungi | and that may be fine, but it's a trade-off we need to take into account | 01:57 |
| mnaser | fungi: thanks for that conversation :> | 01:59 |
| * mnaser has some thinking to do about how we can make this work for everyone (and be a win-win-win) | 02:00 | |
| fungi | my mentioning the linux kernel model was to point out that it's not necessarily wrong to have different branches catering to different stability needs. it may seem like a duplication of effort, but that can still be justifiable given the right mix of needs | 02:01 |
| fungi | my bigger concern is i think we still can't deliver on a promise to provide a security-supported distribution of software we don't write, at least not without a lot more volunteers to make it happen (and basically becoming a small-scale/derivative distro in our own right). that's not a decision to be taken lightly | 02:03 |
| *** whoami-rajat has joined #openstack-tc | 02:08 | |
| fungi | the starlingx community went down that path, and now they're spending massive amounts of effort on something which is not their core competency (and i think are actively working toward *not* having to maintain their own distro in the future) | 02:08 |
| *** diablo_rojo has quit IRC | 02:49 | |
| *** ricolin has joined #openstack-tc | 02:55 | |
| *** e0ne has joined #openstack-tc | 05:15 | |
| *** e0ne has quit IRC | 05:20 | |
| *** dklyle_ has joined #openstack-tc | 05:24 | |
| *** david-lyle has quit IRC | 05:27 | |
| *** purplerbot has quit IRC | 05:27 | |
| *** purplerbot has joined #openstack-tc | 05:28 | |
| *** irclogbot_2 has quit IRC | 05:30 | |
| *** irclogbot_0 has joined #openstack-tc | 05:32 | |
| *** Luzi has joined #openstack-tc | 05:50 | |
| *** spsurya has joined #openstack-tc | 06:15 | |
| *** ricolin has quit IRC | 06:18 | |
| *** ricolin has joined #openstack-tc | 06:19 | |
| *** tosky has joined #openstack-tc | 07:15 | |
| *** iurygregory has joined #openstack-tc | 07:24 | |
| *** iurygregory has quit IRC | 07:35 | |
| *** iurygregory has joined #openstack-tc | 07:40 | |
| *** dtantsur|afk is now known as dtantsur | 07:56 | |
| *** jpich has joined #openstack-tc | 07:59 | |
| asettle | o/ | 08:25 |
| *** e0ne has joined #openstack-tc | 08:34 | |
| openstackgerrit | Alexandra Settle proposed openstack/project-team-guide master: Reviewing the PTL document for accessibility https://review.opendev.org/665699 | 09:15 |
| *** tosky has quit IRC | 09:47 | |
| asettle | ricolin, I've finally got some time to review https://etherpad.openstack.org/p/explain-team-formate-differentiate | 09:47 |
| asettle | Was there anything else I needed to review for this? | 09:48 |
| *** tosky has joined #openstack-tc | 09:48 | |
| *** lpetrut has joined #openstack-tc | 09:49 | |
| ricolin | asettle, not for now, I will update that etherpad today and try to targeting those feedback | 09:54 |
| asettle | Okay thanks :) going through and placing edits and occasional comments | 09:55 |
| ricolin | asettle, that will be super helpful! | 09:58 |
| asettle | np :) nearly done | 09:58 |
| *** lpetrut has quit IRC | 10:13 | |
| ricolin | asettle, try to explain those unknow meaning part in https://etherpad.openstack.org/p/explain-team-formate-differentiate | 10:23 |
| ricolin | asettle, let me know if it's still confusing | 10:23 |
| asettle | Okay looking :) | 10:24 |
| asettle | Good good! | 10:24 |
| asettle | Thank you :) | 10:24 |
| ricolin | asettle, will also add another paragraph to introduce other formats(which you may not create but it's good to know they exists like Foundation level working group) | 10:26 |
| asettle | Cool sounds good. Shoot me a ping when that's done and I can look at that oto. | 10:26 |
| *** lpetrut has joined #openstack-tc | 10:51 | |
| jroll | fungi: mnaser: thanks for the good conversation in scrollback. I'm not sure I have anything to add right now, but it's certainly making my brain churn. :) | 11:09 |
| smcginnis | fungi: Since you've been able to articulate it better than anyone else I've seen, would you have a few minutes to add some comments to the top of https://opendev.org/openstack/requirements/src/branch/master/upper-constraints.txt to make it a little more clear what the expectations are for that file and the package/versions listed there? | 11:46 |
| *** tdasilva has joined #openstack-tc | 12:05 | |
| fungi | smcginnis: sure, i'll see if i can distil some useful nuggets for that once i'm more thoroughly caffeinated | 12:10 |
| smcginnis | Thanks! :) | 12:10 |
| openstackgerrit | Adam Spiers proposed openstack/project-team-guide master: Document running tests with stestr https://review.opendev.org/666273 | 12:13 |
| *** ricolin_phone has joined #openstack-tc | 12:39 | |
| *** ricolin_phone has quit IRC | 12:42 | |
| *** ricolin_phone has joined #openstack-tc | 12:52 | |
| *** ricolin_phone has quit IRC | 13:09 | |
| *** mriedem has joined #openstack-tc | 13:11 | |
| *** ricolin_phone has joined #openstack-tc | 13:14 | |
| *** ijolliffe has joined #openstack-tc | 13:16 | |
| *** ricolin_phone has quit IRC | 13:17 | |
| *** lbragstad has joined #openstack-tc | 13:20 | |
| *** jamesmcarthur has joined #openstack-tc | 14:05 | |
| *** lpetrut has quit IRC | 14:14 | |
| *** zaneb has quit IRC | 14:30 | |
| *** zaneb has joined #openstack-tc | 14:30 | |
| *** Luzi has quit IRC | 14:36 | |
| *** dklyle_ has quit IRC | 14:42 | |
| *** dklyle has joined #openstack-tc | 14:42 | |
| *** iurygregory has quit IRC | 14:52 | |
| *** lpetrut has joined #openstack-tc | 14:58 | |
| *** lpetrut has quit IRC | 14:59 | |
| *** lpetrut has joined #openstack-tc | 14:59 | |
| *** tdasilva_ has joined #openstack-tc | 15:06 | |
| *** tdasilva has quit IRC | 15:07 | |
| *** iurygregory has joined #openstack-tc | 15:18 | |
| *** Sundar has joined #openstack-tc | 15:24 | |
| *** lpetrut has quit IRC | 15:30 | |
| *** thiago__ has joined #openstack-tc | 15:39 | |
| *** tdasilva_ has quit IRC | 15:42 | |
| *** iurygregory_ has joined #openstack-tc | 15:45 | |
| *** iurygregory has quit IRC | 15:46 | |
| *** diablo_rojo has joined #openstack-tc | 16:00 | |
| *** cdent has joined #openstack-tc | 16:08 | |
| *** cdent has left #openstack-tc | 16:09 | |
| *** lpetrut has joined #openstack-tc | 16:10 | |
| *** spsurya has quit IRC | 16:18 | |
| *** thiago__ is now known as tdasilva | 16:19 | |
| *** tellesnobrega has joined #openstack-tc | 16:27 | |
| *** jpich has quit IRC | 16:27 | |
| tellesnobrega | tonyb, hey, I was the PTL for sahara for last releases but now the new PTL is jeremy freudberg, I would like to know how can I add him to the sahara-stable-maint | 16:29 |
| tellesnobrega | and also update the team, we have a lot of people that are not working on sahara anymore that needs to be cleaned out of that team | 16:29 |
| mugsie | tellesnobrega: https://docs.openstack.org/project-team-guide/stable-branches.html#stable-maintenance-teams is how new people are added to the sable-maint team | 16:31 |
| tellesnobrega | thanks mugsie | 16:31 |
| mugsie | PTLs are not automatically added to that group - they have to show their knowledge of the stable policy like everyone else | 16:32 |
| tellesnobrega | mugsie, thanks | 16:33 |
| tellesnobrega | as a member of that team, I have two names that should be there | 16:33 |
| tellesnobrega | jeremyfreudberg (Jeremy Freudberg) and tosky (Luigi Toscano) they both have more than proved their knowledge of review process and stable branches policies | 16:34 |
| mugsie | emailing the list with the [stable] tag, or pinging the team in #openstack-stable is a good place to get them added then | 16:35 |
| tellesnobrega | also, thanks | 16:35 |
| smcginnis | tellesnobrega: Do you know off hand - does Sahara assert the stable:follows-policy tag? | 16:36 |
| smcginnis | I suppose I could look that up. | 16:36 |
| mugsie | smcginnis: it does | 16:36 |
| smcginnis | OK, then everything above is a good start. ;) | 16:37 |
| *** tosky has quit IRC | 16:47 | |
| *** ricolin has quit IRC | 16:48 | |
| *** lpetrut has quit IRC | 16:49 | |
| *** diablo_rojo has quit IRC | 16:56 | |
| *** dtantsur is now known as dtantsur|afk | 16:56 | |
| *** e0ne has quit IRC | 17:20 | |
| *** diablo_rojo has joined #openstack-tc | 17:26 | |
| openstackgerrit | Adam Spiers proposed openstack/project-team-guide master: Document running tests with stestr https://review.opendev.org/666273 | 17:36 |
| *** brett-soric has joined #openstack-tc | 17:41 | |
| *** tellesnobrega has left #openstack-tc | 17:49 | |
| *** melwitt is now known as jgwentworth | 17:55 | |
| *** diablo_rojo has quit IRC | 18:05 | |
| *** tdasilva has quit IRC | 18:17 | |
| *** brett-soric has left #openstack-tc | 18:30 | |
| *** diablo_rojo has joined #openstack-tc | 18:31 | |
| *** tdasilva has joined #openstack-tc | 18:35 | |
| *** e0ne has joined #openstack-tc | 18:35 | |
| *** e0ne has quit IRC | 18:36 | |
| *** jamesmcarthur has quit IRC | 18:40 | |
| *** jamesmcarthur has joined #openstack-tc | 18:41 | |
| *** jamesmcarthur has quit IRC | 18:46 | |
| *** diablo_rojo has quit IRC | 18:46 | |
| *** diablo_rojo has joined #openstack-tc | 18:57 | |
| *** jamesmcarthur has joined #openstack-tc | 19:04 | |
| *** e0ne has joined #openstack-tc | 19:05 | |
| *** tdasilva has quit IRC | 19:06 | |
| *** jamesmcarthur has quit IRC | 19:10 | |
| *** jamesmcarthur has joined #openstack-tc | 19:23 | |
| *** bsilverman has quit IRC | 19:47 | |
| *** jgwentworth is now known as melwitt | 19:52 | |
| *** ijolliffe has quit IRC | 19:53 | |
| *** e0ne has quit IRC | 20:11 | |
| *** jamesmcarthur has quit IRC | 20:13 | |
| *** jamesmcarthur has joined #openstack-tc | 20:15 | |
| *** jamesmcarthur has quit IRC | 20:23 | |
| openstackgerrit | Adam Spiers proposed openstack/project-team-guide master: Document running tests with stestr https://review.opendev.org/666273 | 20:41 |
| *** whoami-rajat has quit IRC | 20:47 | |
| *** e0ne has joined #openstack-tc | 20:55 | |
| *** dirk has quit IRC | 20:55 | |
| *** e0ne has quit IRC | 20:59 | |
| *** e0ne has joined #openstack-tc | 21:00 | |
| *** e0ne has quit IRC | 21:39 | |
| *** jamesmcarthur has joined #openstack-tc | 21:40 | |
| *** jamesmcarthur has quit IRC | 21:51 | |
| *** mriedem has quit IRC | 21:59 | |
| *** diablo_rojo has quit IRC | 22:05 | |
| *** diablo_rojo has joined #openstack-tc | 22:35 | |
| *** Sundar has quit IRC | 22:44 | |
| openstackgerrit | Adam Spiers proposed openstack/project-team-guide master: Document running tests with stestr https://review.opendev.org/666273 | 23:36 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!