Skip to content

Auto-update when running `brew install`. #50192

Open
wants to merge 1 commit into from

5 participants

@mikemcquaid
Homebrew member
  • Have you added an explanation of what your changes do and why you'd like us to include them?
  • Have you successfully ran brew tests with your changes locally?

Opening this for discussion rather than a quick merge. I'm interested in the thoughts of @Homebrew/maintainers about edge-cases we should handle but the brew update no-op is quick enough that the overhead here is minimal and it will help us with a lot of support issues.

@mikemcquaid mikemcquaid Auto-update when running `brew install`.
Slightly tweak the behavior of `brew update` in this case so that it
doesn't print annoying output.
e93dcd8
@apjanke

Enthusiasm! In general, I think auto-update would be a good feature and very helpful to both users and maintainers. And I agree that the no-op case is fast enough now that it's not a burden.

Two concerns: a knob to disable, and cross-formula version skew.

Knob to disable

A couple populations might want a knob to disable the auto-update. For developers that are reproducing issues as of specific commits, or working on things that themselves might involve interaction with git updates, they'll need a way of turning off updates. And there are some users who might just want it turned off, either because they need to do some auditing, or they just prefer having more control. Homebrew has a bit of a DIY/hands-on history, so wouldn't be surprised if a good number of users want that, and maybe we shouldn't second-guess them, even if it means some more maintainer work.

If I read the source changes right, it looks like $HOMEBREW_DEVELOPER disables the auto-update. But maybe there should be a separate knob to disable just the auto-update without turning on the other developer behaviors.

(Though "auto-updates on" sure seems like the right default setting for non-DEVELOPERs.)

Version skew

For Homebrew users doing updates now – and this seems like it should be the normal thing to do – you don't just brew update; you brew update && brew upgrade && brew cleanup. Doing an implicit frequent brew update means that you'd see newer versions of packages whose dependencies are already installed but with older versions. So doing a brew install foo might well end up installing a new foo with older deps in a combination that hasn't been tested by either the contributor updating the foo formula or the test bot. That's not a good situation and might introduce its own maintenance issues.

If we're going to do this automatically, maybe it should check to see if there were updates to currently-installed formulae (or better, currently-installed formulae in the recursive dependency set of the requested install formulae) and prompt the user about upgrading them before doing the install. Maybe with another option to auto-answer "yes"/"no"/"abort", to support scripted installations.

@bfontaine
Homebrew member

I’m all in favor for this but have three concerns: network connectivity, unneeded updates and an option to enable/disable the behavior.

  • Network connectivity: what happens if I’m not connected to the Internet? brew update will fail but that’ll take some time depending on how many taps you have. I just tested and it took 5 seconds with 27 taps; not that long. What happens with low connections? Say I’m sharing my phone’s 3G network and I made sure to brew fetch formulae before; I don’t want Homebrew to use my data plan every time I install something.
  • Unneeded updates: This is related with the low connection issue; we probably don’t need to update all taps when installing a core formula. It becomes tricky if it had a(n optional) dep on a tap formula.
  • Enable/disable the behavior: Sometimes you want to test something or go back in the history to install a previous version; it’d be helpful to be able to disable the default behavior. On the other hand as maintainers we might want to enable the behavior so we’ll be able to detect issues early.
@UniqMartin UniqMartin commented on the diff
Library/brew.sh
@@ -160,6 +160,16 @@ EOS
esac
fi
+if [[ "$HOMEBREW_COMMAND" = "install" ]] && [[ -n "$HOMEBREW_DEVELOPER" ]]
+then
+ # Hide shellcheck complaint:
+ # shellcheck source=/dev/null
+ (
+ source "$HOMEBREW_LIBRARY/Homebrew/cmd/update.sh"
+ { homebrew-update --preinstall; }
@UniqMartin
Homebrew member

This line is equivalent to homebrew-update --preinstall. No need for the curly braces and the semicolon.

However, this is probably gonna create grief every once in a while. If the update happens to include changes to Library/brew.sh, then execution will continue after this line, no matter what the new code at that file offset is. In most cases this will result in an obscure error. This is exactly the reason why we have that conflated piece of code ({ "homebrew-$HOMEBREW_COMMAND" "$@"; exit $?; }) a few lines below, that makes sure to exit the current shell process (that started out with the old code) before leaving the already parsed piece of code in the curly braces.

A scheme that could work, but is a bit more complex: User runs brew install <install-arguments>. This gets transformed into brew update --preinstall -- <install-arguments>. Once that finishes, a new process that uses the new code needs to be spawned from within brew update, e.g. brew install --already-updated <install-arguments>.

@mikemcquaid
Homebrew member

A scheme that could work, but is a bit more complex: User runs brew install . This gets transformed into brew update --preinstall -- . Once that finishes, a new process that uses the new code needs to be spawned from within brew update, e.g. brew install --already-updated .

This sounds like a very good idea :+1:

@xu-cheng
Homebrew member

I think we should try to not over complicate the logic in brew.sh when handling this. Instead, how about create a cmd/install.sh and cmd/install-without-update.rb. So

  • cmd/install.sh is like:
# code to set HOMEBREW_DEBUG and HOMEBREW_VERBOSE based on args.
{ brew update --preinstall && brew install-without-update "$@"; }
  • cmd/install-without-update.rb is the current cmd/install.rb.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@UniqMartin UniqMartin commented on the diff
Library/Homebrew/cmd/update.sh
@@ -347,11 +348,20 @@ EOS
--header "If-None-Match: \"$UPSTREAM_BRANCH_LOCAL_SHA\"" \
"https://api.github.com/repos/$UPSTREAM_REPOSITORY/commits/$UPSTREAM_BRANCH")"
[[ "$UPSTREAM_SHA_HTTP_CODE" = "304" ]] && exit
+ elif [[ -n "$HOMEBREW_UPDATE_PREINSTALL" ]]
+ then
+ exit
@UniqMartin
Homebrew member

I'm not sure I fully understand the intention of this. Does that mean brew update --preinstall is a no-op if the remote URL doesn't start with https://github.com/, i.e. we cannot check its status sufficiently quickly?

@mikemcquaid
Homebrew member

Yep. Will add a comment.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@UniqMartin
Homebrew member

I'm generally in favor, but otherwise pretty much what Andrew and Baptiste expressed much better than I could. I'd like to add to that discussion that maybe we want to limit how often we kick off an automatic update due to brew install. To me it feels reasonable to update at most once every 6/12/24 hours (not sure about the best value) to ensure CLI responsiveness and avoid hitting remotes unnecessarily often.

As long as there's a way to disable the auto-update and the technical issue I raised (as a code comment) gets addressed, I'm happy.

@mikemcquaid
Homebrew member

Network connectivity: what happens if I’m not connected to the Internet? brew update will fail but that’ll take some time depending on how many taps you have.

I wrote this on a plane so that case is handled pretty well :grinning:. It fails quickly and produces no error output.

I just tested and it took 5 seconds with 27 taps; not that long. What happens with low connections? Say I’m sharing my phone’s 3G network and I made sure to brew fetch formulae before; I don’t want Homebrew to use my data plan every time I install something.

That's a trickier situation and I think it may just come down to end-users adapting to the change in behaviour.

@mikemcquaid
Homebrew member

Unneeded updates: This is related with the low connection issue; we probably don’t need to update all taps when installing a core formula. It becomes tricky if it had a(n optional) dep on a tap formula.

I've worked a lot on making brew update extremely quick to do unneeded updates including a new GitHub API. Both the data usage and the speed in this case should be pretty much optimal.

Enable/disable the behavior: Sometimes you want to test something or go back in the history to install a previous version; it’d be helpful to be able to disable the default behavior. On the other hand as maintainers we might want to enable the behavior so we’ll be able to detect issues early.

Yeh, I was going to enable this just for developers to start with.

@mikemcquaid
Homebrew member

Doing an implicit frequent brew update means that you'd see newer versions of packages whose dependencies are already installed but with older versions. So doing a brew install foo might well end up installing a new foo with older deps in a combination that hasn't been tested by either the contributor updating the foo formula or the test bot. That's not a good situation and might introduce its own maintenance issues.

I believe in this case we autoupgrade dependencies. Note that this situation is identical whether you brew update during the install process or not.

@mikemcquaid
Homebrew member

To me it feels reasonable to update at most once every 6/12/24 hours (not sure about the best value) to ensure CLI responsiveness and avoid hitting remotes unnecessarily often.

I'm less concerned with remote load. CLI responsiveness could be achieved by setting an aggressive timeout. I'm OK with not updating every time but the hours feels kinda arbitrary; a formula could have been updated in that time and that would make this behaviour unreliable for us. Perhaps there's some approach where we can check if the formula dependency tree have received any updates? This could also mean we could filter the taps that we update in that time too.

@UniqMartin
Homebrew member

I'm OK with not updating every time but the hours feels kinda arbitrary; a formula could have been updated in that time and that would make this behaviour unreliable for us.

I agree that any number of hours is bound to be arbitrary and I have no good answer for what the “right” interval is. But I think we're mostly trying to solve the issue where people don't update for weeks or even months; that they haven't updated in the last few hours only very rarely will make a difference. At least this is my perception—please correct me if I'm wrong.

@xu-cheng
Homebrew member

I think you have discussed most of the issues better than I could. I just want to add:

  • This could break brew edit foo; brew install foo. Because brew update will reset the DIY edition. I don't know whether we should address this or just declare to not support it. Update: this would make us impossible to run brew install foo inside a branch as brew update will reset it as well.
  • Since brew update will to migration of formula rename. I wonder will it interfere with the installation. For example, say the migration fails. This could provide a bad UX.
  • Should we add a config option to disable such feature. I would expect someone to request this.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.