Opened 6 years ago

Last modified 6 years ago

#6324 assigned Feature/Enhancement Request

Create MFPL specific ikiwiki auto setup scripts to work with hosting order file structure

Reported by: https://id.mayfirst.org/nat Owned by: https://id.mayfirst.org/nat
Priority: Low Component: Tech
Keywords: ikiwiki f2f Cc: tezcatl@…
Sensitive: no

Description

I'm still working on installing ikiwiki, and decided to run through the install process described on the setup page. In short, it suggests running this to set up a blog:

ikiwiki --setup /etc/ikiwiki/auto-blog.setup

This interactively asks you a couple of questions about the site, and creates all of the files for you. Upon resolution of #6313, that command should work for users on all moshes.

The problem is, that the default ikiwiki setup file puts files in places that don't work in our hosting order setup. The main problem is that the destination directory (which contains the files that live on the public internet), gets put in ~/public_html/sitename.

I think we should provide our own default setup files.

We should find out what hosting order the user associated with and automatically set the destination. Alternately we could ask the user what the hosting order/ destination dir is. I'm actually not sure what the best way to go about it is, so I'm curious to hear thoughts.

Here is the output of the current setup process.

0 nat@marx:~$ ikiwiki --setup /etc/ikiwiki/auto-blog.setup
What will the blog be named? blogname
What revision control system to use? git
What wiki user (or openid) will be admin? https://id.mayfirst.org/nat


Setting up blogname ...
Importing /home/members/nat/sites/nat.mayfirst.org/users/nat/blogname into git
Initialized empty shared Git repository in /home/members/nat/sites/nat.mayfirst.org/users/nat/blogname.git/
Initialized empty Git repository in /home/members/nat/sites/nat.mayfirst.org/users/nat/blogname/.git/
[master (root-commit) 4f04712] initial commit
 8 files changed, 49 insertions(+), 0 deletions(-)
 create mode 100644 .gitignore
 create mode 100644 archives.mdwn
 create mode 100644 comments.mdwn
 create mode 100644 index.mdwn
 create mode 100644 posts.mdwn
 create mode 100644 posts/first_post.mdwn
 create mode 100644 sidebar.mdwn
 create mode 100644 tags.mdwn
Counting objects: 11, done.
Compressing objects: 100% (9/9), done.
Writing objects: 100% (11/11), 1.55 KiB, done.
Total 11 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (11/11), done.
To /home/members/nat/sites/nat.mayfirst.org/users/nat/blogname.git
 * [new branch]      master -> master
Directory /home/members/nat/sites/nat.mayfirst.org/users/nat/blogname is now a clone of git repository /home/members/nat/sites/nat.mayfirst.org/users/nat/blogname.git


Creating wiki admin https://id.mayfirst.org/nat ...
Choose a password:
Confirm password:


successfully generated /home/members/nat/sites/nat.mayfirst.org/users/nat/public_html/blogname/ikiwiki.cgi
successfully generated /home/members/nat/sites/nat.mayfirst.org/users/nat/blogname.git/hooks/post-update
ikiwiki-update-wikilist: cannot write to /etc/ikiwiki/wikilist
** Failed to add you to the system wikilist file.
** (Probably ikiwiki-update-wikilist is not SUID root.)
** Your wiki will not be automatically updated when ikiwiki is upgraded.


Successfully set up blogname:
        url:         http://marx.mayfirst.org/~nat/blogname
        srcdir:      ~/blogname
        destdir:     ~/public_html/blogname
        repository:  ~/blogname.git
To modify settings, edit ~/blogname.setup and then run:
        ikiwiki -setup ~/blogname.setup
0 nat@marx:~$ 

So, there are are few things here that need changing.

  • /home/members/nat/sites/nat.mayfirst.org/users/nat/public_html/blogname/ikiwiki.cgi is not the path we want to set, we should change it to the cgi-bin for that hosting order.
  • destdir: ~/public_html/blogname, is also the wrong place under the hosting order file structure.
  • Finally, we should figure out how to add these files to the system wikilist file.

I think that the starting point is to make copies of /etc/ikiwiki/auto.setup and /etc/ikiwiki/auto-blog.setup, and hack those, and then put those changes in puppet.

--nat

Attachments (2)

mf-ikiwiki-auto.diff (1.5 KB) - added by https://id.mayfirst.org/nat 6 years ago.
mf-ikiwiki-auto-blog.diff (1.5 KB) - added by https://id.mayfirst.org/nat 6 years ago.

Download all attachments as: .zip

Change History (8)

Changed 6 years ago by https://id.mayfirst.org/nat

Changed 6 years ago by https://id.mayfirst.org/nat

comment:1 Changed 6 years ago by https://id.mayfirst.org/nat

  • Status changed from new to assigned

So I have taken a first stab at this.

What I did was copy the default auto setup scripts (from the squeeze version of ikiwiki), and modify them to fit some mfpl needs.

Namely, the default setup scripts dynamically determines the server's fqdn, and uses it to set the domain. The domain then gets set to something like marx.mayfirst.org/~nat/wikiname. The $domain variable, if set correctly helps us in setting defaults for the directory locations, assuming the default hosting order model (which is all we should support with these scripts imho).

I've attached the diffs of what I have changed from the defaults. I haven't tested this yet, but I think that given the correct question answers and a clean hosting order these changes should set up an ikiwiki site on a hosting order with files in the right places.

--nat

comment:2 Changed 6 years ago by https://id.mayfirst.org/nat

I broke ikiwiki out into a puppet module, in anticipation of these scripts and a squeeze-backport as well as the completion #6313, that module includes the two custom setup files.

In testing the changes to the mf-auto.setup scripts, I found that Ikiwiki is actually too well written to handle our use case. The issue is that Ikiwiki doesn't want to overwrite the web directory. In other words the setup script assumes that ~/ikiwiki.example.com/web (as well as any other directory that it creates or uses), doesn't exist. If it does, rather than overwriting what is in ~/ikiwiki.example.com/web, it tries to create ~/ikiwiki.example.com/2web. Since the user doesn't actually own that directory, they can't create that directory.

Anyway, I think this will ultimately require a patch to IkiWiki::Setup::Automator. Which I can hopefully find time to write soon. If anyone else wants to take this on, please do.

--nat

comment:3 Changed 6 years ago by https://id.mayfirst.org/tezcatl

  • Cc tezcatl@… added

comment:4 Changed 6 years ago by https://id.mayfirst.org/dkg

Looks to me like the web2web transition is handled on lines 44-55 of /usr/share/perl5/IkiWiki/Setup/Automator.pm, wrapped in a test for `$config{force_overwrite}':

     39 sub import (@) {
     40         my $this=shift;
     41         $config{setuptype}='Yaml';
     42         IkiWiki::Setup::merge({@_});
     43 
     44         if (! $config{force_overwrite}) {
     45                 # Avoid overwriting any existing files.
     46                 foreach my $key (qw{srcdir destdir repository dumpsetup}) {
     47                         next unless exists $config{$key};
     48                         my $add="";
     49                         my $dir=IkiWiki::dirname($config{$key})."/";
     50                         my $base=IkiWiki::basename($config{$key});
     51                         while (-e $dir.$add.$base) {
     52                                 $add=1 if ! $add;
     53                                 $add++;
     54                         }
     55                         $config{$key}=$dir.$add.$base;
     56                 }
     57         }

Is there a way you can set the force_overwrite option in the mfpl ikiwiki setup script? if so, you could avoid needing a patch for Automator.pm. If this setting is too scary on its own, you could do some kind of check before invoking the automator's import command with that set.

comment:5 Changed 6 years ago by https://id.mayfirst.org/nat

So I tried this, and it doesn't work. However when i asked in #ikiwiki (on oftc), joeyh (Ikiwiki's maintainer) said it should work. I'm still hitting permissions issues.

Here is the output of a failed setup run:

0 ikiwiki-test-nat@marx:~$ ikiwiki --setup /etc/ikiwiki/mf-auto.setup 
What will the wiki be named? test wiki
What revision control system to use? git
Which user (wiki account or openid) will be admin? https://id.mayfirst.org/nat
What is the domain name of the wiki? ikiwiki.test.nat.mayfirst.org


Setting up test wiki ...
Importing /home/members/nat/sites/ikiwiki.test.nat.mayfirst.org/users/ikiwiki-test-nat/testwiki into git
Initialized empty shared Git repository in /home/members/nat/sites/ikiwiki.test.nat.mayfirst.org/users/ikiwiki-test-nat/testwiki.git/
Initialized empty Git repository in /home/members/nat/sites/ikiwiki.test.nat.mayfirst.org/users/ikiwiki-test-nat/testwiki/.git/
[master (root-commit) 1a64181] initial commit
 Committer:  <ikiwiki-test-nat@marx.mayfirst.org>
Your name and email address were configured automatically based
on your username and hostname. Please check that they are accurate.
You can suppress this message by setting them explicitly:

    git config --global user.name "Your Name"
    git config --global user.email you@example.com

If the identity used for this commit is wrong, you can fix it with:

    git commit --amend --author='Your Name <you@example.com>'

 1 files changed, 2 insertions(+), 0 deletions(-)
 create mode 100644 .gitignore
Counting objects: 3, done.
Writing objects: 100% (3/3), 240 bytes, done.
Total 3 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (3/3), done.
To /home/members/nat/sites/ikiwiki.test.nat.mayfirst.org/users/ikiwiki-test-nat/testwiki.git
 * [new branch]      master -> master
Directory /home/members/nat/sites/ikiwiki.test.nat.mayfirst.org/users/ikiwiki-test-nat/testwiki is now a clone of git repository /home/members/nat/sites/ikiwiki.test.nat.mayfirst.org/users/ikiwiki-test-nat/testwiki.git
/etc/ikiwiki/mf-auto.setup: mkdir /home/members/nat/sites/ikiwiki.test.nat.mayfirst.org/users/ikiwiki-test-nat/ikiwiki.test.nat.mayfirst.org/web:  at /usr/share/perl5/IkiWiki/Setup/Automator.pm line 148.

usage: ikiwiki [options] source dest
       ikiwiki --setup configfile
255 ikiwiki-test-nat@marx:~$

Upon joeyh's suggestion I went and looked at branchable's Ikiwiki hosting setup, as well as copied the git repository for pointers from those auto-setup files. There are some good ideas in there; however I'm going idle this task until the completion of #6373, which involves upgrading to a newer version of Ikiwiki. The newer version has a different setup file format, which is why it makes sense to wait for the upgrade.

--nat

comment:6 Changed 6 years ago by https://id.mayfirst.org/nat

  • Keywords f2f added

Even with the upgraded version the force_overwrite portion of this is not working.

I am open to ideas about how to handle this.

--nat

Please login to add comments to this ticket.

Note: See TracTickets for help on using tickets.