| *** jasondotstar has quit IRC | 01:24 | |
| *** jasondotstar has joined #openstack-infra-incident | 01:29 | |
| *** crinkle_ has joined #openstack-infra-incident | 02:52 | |
| *** crinkle has quit IRC | 02:52 | |
| *** crinkle_ is now known as crinkle | 03:08 | |
| *** ig0r_ has joined #openstack-infra-incident | 09:17 | |
| *** ig0r_ has quit IRC | 09:41 | |
| *** nibalizer has joined #openstack-infra-incident | 17:20 | |
| jeblair | anyone here? | 17:20 |
|---|---|---|
| fungi | yeah, so we should take care of it, though the urgency is not insane | 17:20 |
| jeblair | clarkb: what's the ubuntu package version? | 17:20 |
| clarkb | lrt me pull it back up | 17:20 |
| jeblair | fungi: i agree re puppetmaster -- though some important machines ssh to the backup server | 17:21 |
| clarkb | http://www.ubuntu.com/usn/usn-2869-1/ | 17:21 |
| clarkb | I dont see anything on centos announce yet | 17:21 |
| jeblair | 1:6.6p1-2ubuntu2.4 is the updated package for trusty | 17:21 |
| jeblair | 1:6.6p1-2ubuntu2.3 is what's on puppetmaster | 17:22 |
| clarkb | so we should be able to do an ansible !git* 'apt-get update && apt-get dist-upgrade' | 17:22 |
| clarkb | but actually make that valid ansible | 17:22 |
| fungi | patched on precise yet? | 17:22 |
| jeblair | dist-upgrade is not showing an option to upgrade to ubuntu2.4 on puppetmaster | 17:22 |
| clarkb | then modify ssh_config on the git servers ubtil centos has packages | 17:23 |
| jeblair | fungi: 1:5.9p1-5ubuntu1.8 | 17:23 |
| clarkb | fungi ya usn url has precise package too | 17:23 |
| fungi | okay, cool | 17:23 |
| clarkb | jeblair: that may mean we aeent using upstream ubuntu for security updates :/ | 17:23 |
| jeblair | Get:28 http://mirror.rackspace.com trusty-security/main Sources [103 kB] | 17:23 |
| jeblair | clarkb: yep | 17:23 |
| clarkb | rax has a habit of overwriting those with their own mirrors | 17:23 |
| jeblair | :( | 17:23 |
| clarkb | ya that | 17:23 |
| nibalizer | hi | 17:23 |
| jeblair | so, task 1: use ansible to find apt configs that have a mirror as the security repo | 17:24 |
| jeblair | task2: figure out a way to fix that | 17:24 |
| jeblair | task3: use ansible to upgrade all | 17:24 |
| jeblair | oh | 17:24 |
| jeblair | task0: apply the config file mitigation on puppetmaster :) | 17:25 |
| fungi | ideally, upgrade your clients first, before you ssh to the servers to upgrade them | 17:25 |
| clarkb | jeblair +1 | 17:25 |
| jeblair | fungi: yeah task -1 is that :) | 17:25 |
| fungi | heh | 17:25 |
| clarkb | jeblair I think the easy mode fix for sources is to have ansible just copy a file over for what it should be | 17:25 |
| fungi | working on step -1 while trying to scarf down a quick lunch | 17:26 |
| jeblair | clarkb: probably; i just wanted to get an idea of what our sources actually look like first -- we do still have a mix of oses | 17:26 |
| clarkb | jeblair ya | 17:26 |
| nibalizer | i juts put the NoRoaming directive in my .ssh/config | 17:26 |
| fungi | it's faster for me to upgrade my openssh-client package | 17:27 |
| jeblair | my step [-1] complete! :) | 17:27 |
| jeblair | i'll do [0] | 17:27 |
| jeblair | [0] done | 17:28 |
| fungi | https://lists.debian.org/debian-security-announce/2016/msg00015.html for the debian advisory | 17:29 |
| fungi | if you're running sid or stretch, install 1:6.7p1-5+deb8u1 from jessie | 17:29 |
| mordred | o/ | 17:30 |
| Clint | if you're not using an ssh-agent you might wanna replace your keys too | 17:31 |
| fungi | yep, depending on how much trust you place in the servers to which you've been ssh'ing with a given key | 17:31 |
| fungi | i pretty heavily compartmentalize my keys, but for my infra ssh key it's probably time to do another annual rotation anyway | 17:33 |
| clarkb | I wonder if the ssh agent confirm is sufficient for protecting against this | 17:34 |
| Clint | it means they can't get your private key | 17:35 |
| jeblair | ansible zm* -a "sh -c 'grep security /etc/apt/sources.list | grep rackspace'" | 17:35 |
| clarkb | oh good I wont switch my key then | 17:35 |
| jeblair | i think something like that ^ should tell us which hosts have problems, yeah? | 17:36 |
| jeblair | (zm* was just me testing on a subset) | 17:36 |
| fungi | if i'm reading correctly, it's the ssh client's process memory which is at risk of leaking, so the agent is maintaining the key itself in another process anyway with its own separate allocation | 17:36 |
| clarkb | jeblair that looks right | 17:36 |
| jeblair | so i'll go do that with 'all' and report back here shortly | 17:36 |
| clarkb | jeblair exclude git* though as they are centos | 17:36 |
| fungi | pbx too? | 17:36 |
| clarkb | pbx is trusty now | 17:37 |
| fungi | oh, right! | 17:37 |
| clarkb | was part of removing centos6 | 17:37 |
| fungi | how quickly i forget | 17:37 |
| jeblair | https://etherpad.openstack.org/p/pYJ6fttQIU | 17:37 |
| nibalizer | https://review.openstack.org/267730 would configure the client not to use roaming, if we want that | 17:38 |
| jeblair | clarkb: no need to exclude since they don't match anyway | 17:38 |
| jeblair | ("SUCCESS" here means "found a rackspace security mirror") | 17:39 |
| jeblair | (so "SUCCESS" is "bad" :) | 17:39 |
| nibalizer | jeblair: what are the uuid hosts? | 17:39 |
| mordred | nibalizer: hosts where more than one machine has the same name | 17:39 |
| jeblair | nibalizer: good question! i think we have to look that up in nova -- they are hosts with duplicate hostnames | 17:39 |
| mordred | you can also look them up in the inventory cache | 17:39 |
| jroll | jeblair: if our mirrors are out of date let me know and I'll bug people | 17:40 |
| jeblair | jroll: your mirrors are out of date :) | 17:40 |
| jroll | :| | 17:40 |
| jeblair | jroll: but the real issue is that we shouldn't be using your mirrors for security updates | 17:40 |
| jeblair | jroll: that's actually an ubuntu-recommended way of doing things | 17:41 |
| jeblair | jroll: i believe rax has fixed that now | 17:41 |
| jroll | jeblair: ah | 17:41 |
| jroll | right. | 17:41 |
| jeblair | jroll: and standard configs are split, with main a mirror and security not, but we have some old hosts | 17:41 |
| jroll | ok, I'll refrain from yelling then | 17:41 |
| fungi | in the past i thought i'd fixed our sources.list files to not use the rackspace mirrors for security updates? | 17:41 |
| fungi | or is this something nova-agent is helpfully replacing for us? | 17:42 |
| jeblair | fungi: the list seems rather small for nova agent to be doing that | 17:42 |
| fungi | i'll start on a system-config patch for that | 17:42 |
| fungi | now that we're on a new enough apt platform we can just drop it into a /etc/apt/sources.list.d/something file | 17:43 |
| jeblair | fungi: hrm | 17:43 |
| fungi | oh, precise may still be too old for that. i'll have to check | 17:43 |
| jeblair | fungi: i feel like this is an error we should correct -- like it would be better to remove them from sources.list | 17:44 |
| clarkb | is it something we need to puppet or just one shot fix? | 17:45 |
| fungi | we could rewrite or sources.list files entirely, sure | 17:45 |
| clarkb | I guess until we control the images puppet is nice | 17:45 |
| mordred | yeah | 17:45 |
| fungi | we could one-shot fix it and write the puppet change to keep it that way for new systems | 17:45 |
| jeblair | we have 6 distinct sources.list files across all hosts | 17:45 |
| jeblair | according to sha1sum | 17:45 |
| nibalizer | apt::repo is pretty goo | 17:46 |
| nibalizer | good* | 17:46 |
| jeblair | my hypothesis is that this is not an ongoing problem | 17:46 |
| mordred | btw - /var/cache/ansible-inventory/ansible-inventory.cache is where the ansible inventory cache goes, in case people need to look things up in the data | 17:46 |
| jeblair | i believe the latest rax images don't do this, and i don't think we have confirmed that nova-agent overwrites this | 17:47 |
| jeblair | so i suspect that if we correct it once, we won't have to do it again... | 17:47 |
| jeblair | i'm fine with also using puppet to protect us from this happening again | 17:47 |
| nibalizer | mordred: thanks | 17:47 |
| mordred | jeblair: yes to all of your statements | 17:48 |
| jeblair | i just want to make sure we understand that we don't _need_ to do that right now because i don't think we're under constant sources.list changing attack :) | 17:48 |
| jeblair | i could be wrong -- just i don't think we've proved that yet :) | 17:48 |
| jeblair | (we should definitely pay close attention) | 17:48 |
| nibalizer | one of the uuid hosts is release.slave.openstack.org | 17:48 |
| nibalizer | the other is openstackid-dev.openstack.org | 17:49 |
| fungi | so we really just need one cleanup for trusty servers and a separate one for precise | 17:49 |
| jeblair | fungi: probably; i'm figuring out what the 6 different sources.list files are now | 17:50 |
| mordred | nibalizer: so there are two release.slave.openstack.org's and two openstackid-dev.openstack.org's ? | 17:50 |
| clarkb | two openstackids makes sense since I think fungi was triyng to trusty them at one point | 17:51 |
| clarkb | for the release slave I don't know why that is, but its probably safe to deltee the one that isn't connected to jenkins | 17:51 |
| nibalizer | no I don't think so | 17:51 |
| fungi | openstackid-dev? yeah, i have been booting and deleting a replacement for that | 17:51 |
| nibalizer | oh wait yes | 17:51 |
| fungi | release.slave got replaced semi-recently and the old one (whatever has addresses not matching dns) can be ignored or deleted | 17:52 |
| *** AJaeger has joined #openstack-infra-incident | 17:52 | |
| jeblair | fungi, nibalizer: is this evidence that new rackspace images may have bad mirror configs? | 17:52 |
| jeblair | maybe they relapsed | 17:53 |
| fungi | jeblair: do you have a summary of the variances? | 17:53 |
| jeblair | fungi: working on that | 17:54 |
| jeblair | fungi: going into https://etherpad.openstack.org/p/pYJ6fttQIU as we speak | 17:54 |
| nibalizer | again I think 267730 sets up the correct client-configuration to be safe until we get the apt-repos sorted | 17:54 |
| fungi | awesome, thanks | 17:54 |
| jeblair | nibalizer: you want to go ahead and merge that now? | 17:54 |
| jeblair | nibalizer: i think i found a syntax error in it, see comment | 17:55 |
| nibalizer | looking | 17:55 |
| nibalizer | jeblair: good catch | 17:56 |
| AJaeger | team, this is about the ssh client incident, correct? Anything that I you need my help with? I doubt it - but will listen in... | 17:56 |
| mordred | AJaeger: yes - that is the current incident - you're always welcome | 17:57 |
| fungi | AJaeger: reviewing puppet bits maybe? but basically under control, and not crazily urgent, just performing due diligence | 17:57 |
| * AJaeger is not an expert but will have a look - reading backscroll now | 17:58 | |
| nibalizer | jeblair: AJaeger https://review.openstack.org/#/c/267730 updated | 17:58 |
| jeblair | nibalizer: +2 | 17:59 |
| nibalizer | kk approving | 18:00 |
| fungi | so our precise servers are getting security updates from ubuntu directly it looks like | 18:02 |
| fungi | deb http://security.ubuntu.com/ubuntu precise-security main restricted | 18:02 |
| jeblair | fungi: except ones with sha1sum of 12670adc87fd7296e430d450f7712058876348ea | 18:03 |
| jeblair | fungi: i just completed the precise section of the etherpad | 18:03 |
| fungi | okay, so we have a mix i guess | 18:03 |
| jeblair | fungi: 4 hosts with that | 18:03 |
| jeblair | fungi: listed in etherpad | 18:04 |
| fungi | those 4 were likely the last precise builds we did | 18:05 |
| fungi | at least they seem more recent than the other precise servers we still have | 18:05 |
| fungi | so this suggests a change in rax or we stopped puppeting in a correct sources.list at some point | 18:06 |
| jeblair | i think ci-backup-rs-ord and zuul-dev are old | 18:06 |
| jeblair | fungi: we puppet sources.list? | 18:06 |
| nibalizer | afaict we only apt::source once, to add the puppetlabs repo | 18:08 |
| fungi | jeblair: i don't find evidence that we were puppeting security.ubuntu.com in a sources.list for anything besides jenkins slaves, and that seems to have ceased when we stopped using natty | 18:09 |
| jeblair | fungi: okay, analysis of 4 trusty sources.list complete | 18:09 |
| fungi | another possibility is that we manually fixed it at some point | 18:10 |
| jeblair | one of them is from ovh. | 18:10 |
| mordred | jeblair: oh yeah. we have other clouds | 18:10 |
| jeblair | mordred: pypi.bhs1.o.o is not showing up in the list | 18:13 |
| jeblair | mordred: clouds.yaml need updating or something? | 18:13 |
| mordred | hrm. maybe? lemme look | 18:13 |
| jeblair | the other pypis are all there | 18:14 |
| mordred | jeblair: yes. patch coming | 18:14 |
| fungi | okay, the issue has finally made it into a post on the oss-security ml | 18:14 |
| fungi | though word seemed to get around earlier than planned | 18:15 |
| clarkb | still nothing on https://lists.centos.org/pipermail/centos-announce/2016-January/thread.html | 18:15 |
| mordred | jeblair: https://review.openstack.org/267758 | 18:16 |
| jeblair | mordred: thx | 18:16 |
| clarkb | what is the difference between all clouds and ansible clouds? | 18:16 |
| mordred | clarkb: all clouds includes nodepool regions | 18:17 |
| mordred | clarkb: it's not actually used anywhere | 18:17 |
| clarkb | ah | 18:17 |
| jeblair | well, it's for operator convenience | 18:17 |
| jeblair | so you can use it with 'openstack' cli | 18:17 |
| mordred | clarkb: it's in tree in case we wanted to drop a clouds.yaml somewhere to be able to do crazy things | 18:17 |
| mordred | yah. that | 18:17 |
| clarkb | right, but we separate so ansible does't get confused talking to 1200 test nodes | 18:17 |
| mordred | yup | 18:17 |
| jeblair | once we nail it down, we can probably get rid of the .sh scripts in ci-launch | 18:17 |
| jeblair | fungi: okay, so on trusty, we have only one variant... | 18:18 |
| jeblair | in rax | 18:18 |
| jeblair | fungi: if i'm reading this right, i think all our trusty hosts are getting security updates from the rax mirror :( | 18:19 |
| jeblair | so i guess they relapsed to the old behavior | 18:19 |
| fungi | that's unfortunate | 18:19 |
| jeblair | i'm now more inclined to believe we should hard-fix this with puppet | 18:20 |
| fungi | yeah, i'm inclined to blow away sources.list entirely and have puppet install a minimal useful one | 18:20 |
| jeblair | should we attempt to maintain usage of cloud-local mirrors at all, or should we just drop in a standard ubuntu one everywhere? | 18:20 |
| jeblair | i'm leaning toward standardized ubuntu | 18:21 |
| fungi | using a common non-provider mirror will be easier to puppet | 18:21 |
| jeblair | (and hope their geodns does something useful for them there) | 18:21 |
| clarkb | the only potential problem with standard ubuntu one is the test slaves | 18:21 |
| fungi | since we have to handle it separately for rax vs ovh vs... otherwise | 18:21 |
| mordred | I vote for standard-ubuntu for long-lived servers | 18:21 |
| clarkb | since they all apt-get update and potentially pull packages that aren't cached | 18:21 |
| clarkb | the flip side is the mirrors break semi frequently | 18:22 |
| fungi | we might want to do this only to our non-slave servers | 18:22 |
| mordred | yah | 18:22 |
| mordred | that's my vote | 18:22 |
| mordred | and we can solve slave servers once we have the mythical mirroring infrastructure ourselves | 18:22 |
| fungi | glean presumably gets separate code to manage sources lists on dynamic workers in the puppetless worker build future utopia | 18:22 |
| nibalizer | ya in theory we'll have our own mirroring up soon™ | 18:23 |
| jeblair | that seems safe for now; i might want to explore the idea of changing it on the slaves too -- i wonder how bad it would really be, but i'm okay considering that a future scope expansion. | 18:23 |
| fungi | or we do some trick with relative domain name search resolution | 18:23 |
| jeblair | nibalizer: tbh we have a plan, but no one working on it. | 18:23 |
| mordred | fungi: I'd say something similar to what we do with pypi mirrors and ready scripts | 18:23 |
| jeblair | nibalizer: so i wouldn't necessarily say we'll have it soon | 18:24 |
| fungi | mordred: oh, true, nodepool is a better candidate than glean | 18:24 |
| * mordred is good at batting features away from glean | 18:24 | |
| fungi | i don't know why i was thinking we needed that determined before nodepool connects to the worker | 18:25 |
| fungi | we definitely don't | 18:25 |
| jeblair | so who wants to write a puppet change to install distro-specific sources.list on non-nodepool workers? | 18:25 |
| fungi | so, yeah, i say puppet for non-dynamic servers, nodepool for the dynamic ones | 18:25 |
| fungi | i'll get working on that now. i've sufficiently degreased my lunchfingers | 18:26 |
| jeblair | fungi: that will be an even easier split when we stop puppeting nodepool workers :) | 18:26 |
| jeblair | fungi: ack, thanks | 18:26 |
| clarkb | in the mean time, have rax mirrors updated enough to allow us to patch without the upstream ubuntu security repos? | 18:27 |
| fungi | should be able to check by grepping the Packages.gz | 18:28 |
| clarkb | git01 has 6.6.1p1 installed fwiw | 18:28 |
| clarkb | nothing on https://rhn.redhat.com/errata/rhel-server-7-errata.html either | 18:40 |
| clarkb | so we may be running with just the updated config on centos for a bit | 18:40 |
| fungi | first stab is https://review.openstack.org/267778 | 19:07 |
| fungi | feel free to recommend adjustments to the list files there, but the uncommented lines appear to correspond to what we were getting from rackspace mirrors previously | 19:09 |
| jeblair | fungi: +2d with a suggestion if you feel like an update | 19:11 |
| fungi | glad to update | 19:12 |
| jeblair | mordred: i'm assumed the extra lines were to keep them as close as possible to the standard ones? | 19:12 |
| jeblair | mordred: though, fungi did add extra comments, so they already aren't exactly the same :) | 19:13 |
| mordred | jeblair: yah to both statements | 19:13 |
| fungi | yeah, and also deleted some trailing lines and in the trusty case switched from gb to us hostnames | 19:13 |
| fungi | so another edit can't hurt | 19:13 |
| fungi | i debated just removing all the comment lines | 19:13 |
| jeblair | fungi: i wouldn't object to that, i'm always surprised how much shorter and readable they are that way :) | 19:13 |
| fungi | yeah, i don't bother with comment lines for all that crap on my personal servers | 19:14 |
| fungi | though also my debian sources.list files are way shorter | 19:14 |
| fungi | also, any reason to leave the deb-src lines there? do we actually ever do anything with source packages on these servers? | 19:15 |
| jeblair | i hope not | 19:15 |
| fungi | yeah, removing those too | 19:15 |
| fungi | i know i don't anyway | 19:15 |
| fungi | if i do, it'll be on my workstation and then end up on a repository somewhere | 19:16 |
| jeblair | i've worked with folks who like to ensure that gcc is _not_ installed on production servers because it's just helping the haxors. | 19:16 |
| jeblair | that might be feasible for us once we finish detangling nodepool from puppet | 19:16 |
| jeblair | though pip may throw a wrench in that | 19:17 |
| fungi | yeah, pip installing non-pure-python stuff will get painful | 19:18 |
| fungi | unless wheels everywhere | 19:18 |
| clarkb | and even then if you don't keep up with making wheels pip will want to builkd from source | 19:18 |
| clarkb | pip will always take newest version it can and wheels only win if newest version has a wheel | 19:19 |
| fungi | okay, minor consistency tweak in that last patchset | 19:19 |
| fungi | ready for any other comments | 19:19 |
| clarkb | I reall hate new gerrits inter patchset diffing | 19:20 |
| jeblair | fungi: ha! you de-normalized the trailing / | 19:20 |
| jeblair | fungi: gertty shows deletions only for precise, but deletions and changes for trusty due to removal of some trailing slashes | 19:20 |
| fungi | alternatively i can add missing trailing / to them all if anyone cares | 19:20 |
| * jeblair pretends not to care | 19:21 | |
| clarkb | note us.archive may be not a great option for ovh | 19:21 |
| clarkb | but we have few permanent hosts in ovh so not a huge deal | 19:21 |
| jeblair | clarkb: yeah, i don't think it's going to be a huge uptick in our international traffic from ovh | 19:21 |
| fungi | clarkb: well, that brings us to additional parameterization, which i'm not opposed to but | 19:21 |
| jeblair | it's one host | 19:22 |
| jeblair | okay 2 | 19:22 |
| fungi | let's consider that maybe a future opportunity for improvement if we care | 19:22 |
| fungi | easy enough to expand this to templates later | 19:22 |
| clarkb | +2'd no approving in case someone else wants to review | 19:22 |
| jeblair | approved | 19:23 |
| clarkb | us.archive seems to be on the east coast at least :) | 19:23 |
| clarkb | still no centos package that I see | 21:29 |
| clarkb | nibalizer: jeblair fungi mordred the puppet change to add the ssh config line fails on slave builds | 22:41 |
| clarkb | I think that means puppet is broken everywhere | 22:41 |
| fungi | clarkb: yeah, see scrollback in -infra. it is | 22:41 |
| fungi | i pinged nibalizer with the puppet error i saw from it | 22:42 |
| clarkb | in that case Ican't actually test that nodepool image builds are working | 22:42 |
| fungi | i was just getting around to checking whether our sources.list change landed | 22:42 |
| jeblair | clarkb: how does it fail? | 22:43 |
| fungi | oh grr, the apply jobs are failing on mine | 22:43 |
| clarkb | jeblair: Error: 6 lines match pattern 'Host *' in file '/etc/ssh/ssh_config'. One or no line must match the pattern. | 22:43 |
| clarkb | I am pulling up file_line docs now to see if we can just put it at the end of the file | 22:44 |
| mordred | so ... | 22:44 |
| jeblair | clarkb: ah | 22:44 |
| mordred | we put out ssh_config files ... why don't we just edit the file template? | 22:44 |
| jeblair | mordred: we do sshd_config -- do we also do ssh_config ? | 22:44 |
| mordred | OH - you're right. we only do sshd_config | 22:45 |
| mordred | I saw the line wrong in my brainhole | 22:45 |
| jeblair | mordred: but regardless, maybe we could just switch to doing that if we aren't already | 22:45 |
| clarkb | https://github.com/puppetlabs/puppetlabs-stdlib#parameters | 22:45 |
| mordred | I mean - this current thing clearly doesn't work - and we grok how file templates work | 22:45 |
| nibalizer | hi | 22:46 |
| fungi | argh! we have a duplicate definition for /etc/apt/sources.list in (unsurprisingly) the apt module | 22:46 |
| nibalizer | sorry was doing a thing | 22:46 |
| clarkb | so we could make that work if ruby regex can match EOF | 22:46 |
| clarkb | hrm I don't think we can set the /m flag | 22:48 |
| clarkb | oh! | 22:50 |
| clarkb | default behavior is to append if you don't set a match or after | 22:50 |
| clarkb | should I write the change? | 22:50 |
| fungi | nibalizer: should we be punching our custom sources.list files through into here instead? https://github.com/puppetlabs/puppetlabs-apt/blob/master/manifests/init.pp#L101 | 22:51 |
| nibalizer | https://review.openstack.org/267854 should work | 22:52 |
| nibalizer | clarkb: ^ | 22:52 |
| clarkb | oh except if we have multiple matches each one needs to be fixed | 22:53 |
| *** ChanServ changes topic to "CVE-2016-0777 openssh-client https://www.qualys.com/2016/01/14/cve-2016-0777-cve-2016-0778/openssh-cve-2016-0777-cve-2016-0778.txt" | 22:54 | |
| nibalizer | clarkb: yea the errror was | 22:54 |
| nibalizer | Error: 8 lines match pattern 'Host *' in file '/etc/ssh/ssh_config'. One or no line must match the pattern. | 22:54 |
| clarkb | ya | 22:54 |
| nibalizer | but putting the ^ in ther makes it match only the uncommented one | 22:54 |
| clarkb | gotcha | 22:55 |
| nibalizer | tested on an ubuntu node and a centos node | 22:55 |
| clarkb | I did read an ssh_config to check too | 22:55 |
| clarkb | so +2 | 22:55 |
| nibalizer | okay i gotta run | 22:55 |
| nibalizer | fungi: i consult the crinkle is my best advice | 22:55 |
| nibalizer | I can look later | 22:55 |
| fungi | nibalizer: oh, right, you had a thing. sorry! | 22:56 |
| * fungi is forgetty | 22:56 | |
| crinkle | how can i help | 22:57 |
| nibalizer | fungi: if we have custom apt sources and we want to puppet them we should probably use apt::source | 23:01 |
| nibalizer | if we want to just have a file and dump it in /etc/apt/sourecs.list.d/ we should probably just use a file resource | 23:02 |
| fungi | crinkle: trying to figure out the best path forward on https://review.openstack.org/267778 since we want a custom sources.list on our servers but that conflicts with the apt module's desire to manage that file in places | 23:02 |
| clarkb | I tink we want to control the actual sources.list | 23:03 |
| nibalizer | clarkb: i approved the puppet fix | 23:04 |
| fungi | yeah, we want to, at best, feed the sources_list_content into that file resource | 23:06 |
| fungi | and i need to go cook dinner. i'll look at this again in a bit, but more recommendations welcome | 23:08 |
| crinkle | why do we need to control the actual sources.list instead of adding to sources.list.d? | 23:18 |
| crinkle | you could do this http://paste.openstack.org/show/483942/ but that is sort of gross hax | 23:24 |
| clarkb | crinkle: because we aren't adding we are replacing, though maybe that just works | 23:24 |
| fungi | crinkle: because our servers have sources.list content we don't want | 23:25 |
| fungi | so we would like puppet to replace the sources.list content that comes on our servers | 23:26 |
| fungi | reasons explained in the commit message for that change | 23:26 |
| crinkle | you could set purge => { 'sources.list' => true } in the apt class and then add an apt::source resource or file resource | 23:27 |
| crinkle | purge just replaces the whole file with a comment | 23:27 |
| crinkle | i will comment | 23:29 |
| fungi | crinkle: thanks! that will probably be good enough | 23:32 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!