Opened 5 years ago

Closed 5 years ago

#7035 closed Task/To do item (fixed)

host a view of MF/PL puppet git repository here on SMO

Reported by: https://id.mayfirst.org/dkg Owned by: https://id.mayfirst.org/dkg
Priority: Medium Component: Tech
Keywords: git puppet trac support.mayfirst.org Cc:
Sensitive: no

Description

Now that this site is running trac 1.0.1, we can provide a clean web interface for git repositories to make our work more visible.

This will also let us refer to changes in those git repositories from the ticketing system, so there can be easy reference to see what's going on.

The first git repository i think we should move should be MF/PL's puppet git repo. Consider this ticket a request for such a setup.

I don't think we need to make SMO the canonical location for the git repo (that can stay at git.mayfirst.org if we want), so maybe the way to do this is with a post-commit hook that pushes to a repo on SMO?

Change History (6)

comment:1 Changed 5 years ago by https://id.mayfirst.org/dkg

  • Owner set to https://id.mayfirst.org/dkg
  • Status changed from new to assigned

I've set up a simple initial synchronization for a repository:

0 moses:~# adduser --disabled-password --gecos 'MF/PL puppet git synchronization,,,' git-puppet
Adding user `git-puppet' ...
Adding new group `git-puppet' (1004) ...
Adding new user `git-puppet' (1004) with group `git-puppet' ...
Creating home directory `/home/git-puppet' ...
Copying files from `/etc/skel' ...
0 moses:~# mkdir -p /srv/git/puppet
0 moses:~# chown git-puppet /srv/git/puppet
0 moses:~# su - git-puppet
0 git-puppet@moses:~$ git clone --bare git://git.mayfirst.org/mfpl/puppet /srv/git/puppet
Cloning into bare repository /srv/git/puppet...
remote: Counting objects: 17571, done.
remote: Compressing objects: 100% (8721/8721), done.
remote: Total 17571 (delta 11068), reused 12943 (delta 7967)
Receiving objects: 100% (17571/17571), 2.30 MiB | 2.73 MiB/s, done.
Resolving deltas: 100% (11068/11068), done.
0 git-puppet@moses:~$ echo '*/5 * * * * cd /srv/git/puppet && git fetch' | crontab -
0 git-puppet@moses:~$  

This should probably be changed from a pull model to a push model, and it should probably happen over cryptographically secured transport.

i'm now following wiki:TracRepositoryAdmin#Migration for the next steps.

Last edited 5 years ago by https://id.mayfirst.org/dkg (previous) (diff)

comment:2 Changed 5 years ago by https://id.mayfirst.org/dkg

I migrated the old svn repo with the following change:

diff --git a/conf/trac.ini b/conf/trac.ini
index f03aa26..d2fbf32 100644
--- a/conf/trac.ini
+++ b/conf/trac.ini
@@ -93,6 +93,15 @@ url = https://mayfirst.org/
 default_anonymous_query = status=assigned&status=new&changetime=1 week ago..now&col=id&col=changetime&col=summary&col=status&col=reporter&group=priority&order=changetime&desc=1
 default_query = status=assigned&status=feedback&status=new&reporter=$USER&or&status=assigned&status=new&owner=$USER&col=id&col=changetime&col=summary&col=reporter&col=status&col=owner&group=priority&order=changetime&desc=1

+[repositories]
+mfpl.dir = /srv/svn/mfpl
+mfpl.description = This is the main MF/PL repository.
+mfpl.type = svn
+mfpl.url = https://support.mayfirst.org/browser
+mfpl.hidden = true
+
+.alias = mfpl
+
 [search]
 min_query_length = 3

@@ -181,7 +190,7 @@ mainnav = wiki,timeline,roadmap,browser,tickets,newticket,search
 metanav = login,logout,settings,help,about
 permission_policies = SensitiveTicketsPolicy, DefaultPermissionPolicy, LegacyAttachmentPolicy, VirtualTicketPermissionsPolicy
 permission_store = DefaultPermissionStore
-repository_dir = /srv/svn/mfpl
+repository_sync_per_request = mfpl
 repository_type = svn
 secure_cookies = true
 timeout = 20

followed by:

su - supportdb -c 'trac-admin /srv/trac/support repository resync mfpl'

we should probably set up some post-commit synchronization hooks so that we do not have to rely on repository_sync_per_request for the main repo.

Last edited 5 years ago by https://id.mayfirst.org/dkg (previous) (diff)

comment:3 Changed 5 years ago by https://id.mayfirst.org/dkg

And i've added the puppet repo like this:

diff --git a/conf/trac.ini b/conf/trac.ini
index 0574668..1341e5a 100644
--- a/conf/trac.ini
+++ b/conf/trac.ini
@@ -20,6 +20,7 @@ includemacro.* = enabled
 openidauth.* = disabled
 sensitivetickets.* = enabled
 trac.ticket.report.* = disabled
+tracopt.versioncontrol.git.* = enabled
 tracopt.versioncontrol.svn.svn_fs.subversionconnector = enabled
 tracopt.versioncontrol.svn.svn_prop.subversionmergepropertydiffrenderer = enabled
 tracopt.versioncontrol.svn.svn_prop.subversionmergepropertyrenderer = enabled
@@ -100,6 +101,10 @@ mfpl.type = svn
 mfpl.url = https://support.mayfirst.org/browser
 mfpl.hidden = true

+puppet.dir = /srv/git/puppet
+puppet.description = MF/PL puppet configuration
+puppet.type = git
+
 .alias = mfpl

 [search]

It doesn't appear to need any resynchronization.

I've just pushed 352c837/puppet in an attempt to see if that works.

Last edited 5 years ago by https://id.mayfirst.org/dkg (previous) (diff)

comment:4 Changed 5 years ago by https://id.mayfirst.org/dkg

Hm, this doesn't seem to work yet. i think we need a post-commit hook for this git repo to push the changeset into place.

comment:5 Changed 5 years ago by https://id.mayfirst.org/dkg

In an attempt to allow explicit synchronization, i've added git-puppet as a postgres user account, and granted it select access to the following tables in the support db:

  • repository
  • system

I also changed the name of the primary repository from "mfpl" to "mfplsvn" (in the hopes that we can eventually migrate that to git, and that the name of the new git repository might be "mfpl" or something else).

At the moment, i'm still not sure how to get the git repositories to update automatically, though.

comment:6 Changed 5 years ago by https://id.mayfirst.org/dkg

  • Resolution set to fixed
  • Status changed from assigned to closed

OK, this is now done, though it's a bit more complex than i'd like it to be.

  • i've removed the cronjob for git-puppet@moses. commits are now pushed through.
  • on git.mayfirst.org, there is now an RSA key available for use by the gitosis user. This key is made available via an ssh-agent socket in cygnus:/var/run/gitosis-ssh-agent/socket2 (see my recent post to openssh-unix-dev about why this currently is "socket2" and requires socat instead of the simpler approach). The associated public key is known as gitosis@git.mayfirst.org in the OpenPGP Web of Trust, and i've certified it publicly.
  • on moses, i've added that identity to ~git-puppet/.monkeysphere/authorized_user_ids with a set of somewhat limited ssh constraints:
    0 git-puppet@moses:~$ cat ~/.monkeysphere/authorized_user_ids
    gitosis@git.mayfirst.org
     command="/usr/local/bin/gitserve",no-port-forwarding,no-agent-forwarding,no-X11-forwarding,no-pty
    0 git-puppet@moses:~$ cat /usr/local/bin/gitserve 
    #!/bin/sh
    exec git-shell -c "$SSH_ORIGINAL_COMMAND"
    0 git-puppet@moses:~$ 
    
  • I added a remote to mfpl/puppet.git on git.mayfirst.org that points to git-puppet@moses.mayfirst.org:/srv/git/puppet
  • I added post-receive and post-commit hooks to the mfpl/puppet.git repo on git.mayfirst.org that just push to smo:
    #!/bin/sh
    SSH_AUTH_SOCK=/var/run/gitosis-ssh-agent/socket2 git push smo
    
  • interestingly, it looks like i didn't need to set up any automatic repository syncing for trac itself. No hooks were necessary on moses, as long as the push completed correctly.

So now pushes to git.mayfirst.org's puppet repository propagate cleanly to this site and show up in the timeline.

Please login to add comments to this ticket.

Note: See TracTickets for help on using tickets.