Opened 10 years ago

Closed 10 years ago

#1883 closed Task/To do item (fixed)

Way to batch create users / email accounts?

Reported by: https://id.mayfirst.org/ryanbf Owned by: https://id.mayfirst.org/jamie
Priority: High Component: Tech
Keywords: eagle.mayfirst.org email quixote scripts Cc:
Sensitive: no

Description

Hi guys, we need to migrate all quixote email accounts to eagle. We have the email rsync'ed over, and will touch that up with a refreshed rsync later today. We are writing a script to go mbox>maildir using mb2md, and this seems to be straightforward.

What we don't have is the user accounts created...

Is the code from this thread still available? https://support.mayfirst.org/ticket/1134

We are trying to get this done today cause some folks will be on the road next week, and it would be nice to send them off switched over... so if we could get some special help this pm it would be awesome.

Change History (24)

comment:1 Changed 10 years ago by https://id.mayfirst.org/jamie

Yes - fortunately - we do have the import script!

If you can get us your users in the format described in ticket 1134 - assuming your passwords are md5 encrypted, we should be in good shape. If you can put this file in the root directory of eagle I can get it setup.

This weekend/next week, however, are really bad for me so I may not be able to get back to you in any kind of timely way.

Jamie

comment:2 Changed 10 years ago by https://id.mayfirst.org/jamie

That last comment was vague - I mean it might be 24 - 48 hours between getting your file before I'll be able to do anything with it.

comment:3 Changed 10 years ago by https://id.mayfirst.org/ryanbf

The file in the appropriate format is now on eagle at /home/members/quixote/mailconvert/users-to-create.txt

I believe I found the script:

(repo) https://support.mayfirst.org/browser/trunk/red/scripts/import-users-and-email-addresses

(locally) /usr/local/share/red/scripts/import-users-and-email-addresses

Is this something that I could run or would you suggest that you do this for us?

I will then create any aliases that can't be represented in your format manually through the control panel.

comment:4 Changed 10 years ago by https://id.mayfirst.org/jamie

The script is something I will have to run from our control panel server.

I just took a look at your file and the format looks fine.

However - I noticed a few potential problems. First - there are a few usernames that have already been created on the system by you (like ryanbf) and a few others that have already been created in different hosting orders (by me: quixote, cso, haitiborn).

And second - I think some of these users we will want to put in their respective hosting orders (haitireborn related users in the haitireborn hosting order, etc.).

To help you sort things out - I've given the Control panel user: quixote access to all the hosting orders. So - now if you login to the members control panel as quixote, you will see all your available hosting orders in the drop down menu.

I think our best bet would be for you to create a user file (just like the one you created) for each hosting order - each file would be named after the hosting order and contain the users that should be put in that hosting order.

By keeping the users with the site we will make things a lot easier down the line if we ever decide to move sites to a different server.

Does that seem right to you?

Jamie

comment:5 Changed 10 years ago by https://id.mayfirst.org/ryanbf

hm... these are email@… and we would love to keep them all together if we could. I see the problem if these users were already created with other hosting orders on the same server.

Could we recreate the other hosting orders users with different user/owner names, i.e. cso-hosting? I'd like to keep these email users together under the quixote banner as they should be in the quixote domain.

What do you think?

comment:6 Changed 10 years ago by https://id.mayfirst.org/ryanbf

email@…..... rather

comment:7 Changed 10 years ago by https://id.mayfirst.org/ryanbf

damn filter :D

comment:8 Changed 10 years ago by https://id.mayfirst.org/alfredo

You can do complete aliases on the emails. So that alfredo@… bounces to alfredo@… (or whatever address you want to). This is the least disruptive way. It's configurable in the email creation screen of the Control Panel.

Or

Destroy the other users and emails and recreate them on quixote but I don't think this is a good idea at all for many reasons. Better thing is the first one.

*But* this is not doable in a mass way, I don't think. You'd have to do it one at a time...unless someone has ideas about a script with which to do it.

Alfredo

comment:9 Changed 10 years ago by https://id.mayfirst.org/ryanbf

Alfredo, the point is that we want all quixote center email distributed in the same set of folders, this is superior in terms of ease of migration and in terms of backup on our end. It also seems to make sense as they are all the same domain. I'm not sure what the many reasons for splitting them up are, as you haven't listed them. I'm willing to listen if there is a there is a real showstopping issue with doing it our way.

comment:10 Changed 10 years ago by https://id.mayfirst.org/alfredo

You can do it any way you want to. I'm only pointing out the implications. To do what you want, you must destroy the users on the other sites (or create new users for those people) and that's not always something people want to do. However, by all means, wipe previous users out and create these new ones if that's not a problem for you folks. You will, of course, have to create additional new users for those users on the other sites that have admin privileges (since they "own" those sites).

At least that's how I see things. I *am* coming in towards the end of this issue discussion so I could be wrong but I think I'm understanding what's going on.

comment:11 Changed 10 years ago by https://id.mayfirst.org/ryanbf

Ok, We can make a list of which user should own which hosting order. These users will be distinct from the quixote email users which will all exist under the quixote aegis.

Can we reassign ownership to the new (non conflicting) users, or should we file a ticket to recreate those hosting orders where we need to reassign ownership?

Sounds like the latter would be easier given how your scripts work. Let me know.

Ryan

comment:12 Changed 10 years ago by https://id.mayfirst.org/jamie

Hi Ryan - I think you have not done any development on eagle - so deleting accounts won't cause any problems.

I would suggest that you:

  • Login as the quixote user in the members control panel and create a new user account under the quixote hosting order called quixote-admin or something like that - a username not currently in use on your existing site.
  • For each hosting order, add this user in the "hosting order access" section - so that this user has control panel access to all hosting orders.
  • Log out of the control panel as quixote and log back in as the user you just created. Make sure you have access to all hosting orders.
  • One by one, delete all your hosting order web sites and usernames that conflict with existing usernames (before you can delete a username, you have to delete the web config, ssh access, and hosting order access record related to it).

At this point, we should have no conflicting usernames. Next - give us a list of your usernames, which hosting order they should be under and which user name should be the primary account for which hosting order and we will import them all and set the properly permissions.

Does this sound like we're on the same page? Let me know if I'm misunderstanding it!

Jamie

comment:13 Changed 10 years ago by https://id.mayfirst.org/ryanbf

Sounds good to me... will give you guys a heads up on when this is prepared... should be in about an hour.

Ryan

comment:14 Changed 10 years ago by https://id.mayfirst.org/ryanbf

1) We actually have set up art-us.org as a live site on mayfirst as of Friday.. but we don't need to delete that, as username does not collide with an email account user.

2) Admin user: Hmmm... quixote at quixote.org is an email address I am willing to sacrifice as it was abandoned to spam in the last couple of years... So let's keep quixote as the main user rather than adding quixote-admin. We'll make sure that the quixote user is added as host-access everywhere (believe that Jamie already did this). Deleting quixote out of import file so as not to collide.

3) Other cleanup: Deleted out user amavis from our import file, which probably would not have been helpful.

comment:15 Changed 10 years ago by https://id.mayfirst.org/ryanbf

OK, here are the users that should be added as users and email addresses under quixote.org hosting order

/home/members/quixote/mailconvert/users-to-create.txt

Please also add users:

cso-owner as primary user for hosting order cso.quixote.org haiti-owner as primary user for hosting order haitireborn.org quest-owner as primary user for hosting order quest.quixote.org

Please let me know when the email address import has completed, as we are scheduling changeover for today (if possible of course.)

In future tickets to create hosting orders, we can specify this pattern.

comment:16 Changed 10 years ago by https://id.mayfirst.org/ryanbf

cso-owner as primary user for hosting order cso.quixote.org
haiti-owner as primary user for hosting order haitireborn.org
quest-owner as primary user for hosting order quest.quixote.org

comment:17 Changed 10 years ago by https://id.mayfirst.org/ryanbf

  • Priority changed from High to Urgent

Hi guys, any motion on this? We are really stalling out here waiting for this one step... it's hard to schedule email downtime with our users as it is...

comment:18 Changed 10 years ago by https://id.mayfirst.org/alfredo

  • Priority changed from Urgent to High
  • Type changed from Bug/Something is broken to Task/To do item

Jamie's away for the week so his response time is slower -- that's what he meant when he said he might not be responding for 24 to 48 hours. I just spoke with him though and he'll be on later today (couple of hours maybe) and he'll work on this then.

I'm taking this down to High from Urgent, Ryan. People sometimes think that escalation is a way of getting quicker attention. It's not. We attend to stuff the same no matter what the level.

It's just a way of defining what's happening and who should handle it. Urgent usually means that the system has broken down and that all the techies may need to jump in and save the ship. No question that there is an urgency for you but I think this is really closer to the "high" priority we had it at.

Also the type was listed as "bug" and I think this is close to a todo item. So I'm changing that while I'm at it.

So I'm gonna drop it to that.

Jamie will be back to you shortly.

Alfredo

changing priority to high

comment:19 Changed 10 years ago by https://id.mayfirst.org/ryanbf

Alfredo, thanks for altering the ticket. I agree with your changes. Thanks especially for responding generally. A couple of hours sounds totally reasonable.

I agree that this was closer to "High" priority, and we just switched it up to "Urgent" to try to get a timeframe negotiated after not hearing something for a while on a bottlenecked task.This is my last day, and I've been trying to wrap this up for a while, so I've been trying to cut through the ticket ping-pong.

It appears from browsing the tickets that people certainly do thing the Urgent priority gets them quicker service (as it should, but only in an emergency), so maybe continuing education is needed.

I agree that this is a todo item as well. Don't know why it was set as bug, guess that is the default. We'll learn.

Ryan

comment:20 Changed 10 years ago by https://id.mayfirst.org/jamie

  • Status changed from new to assigned

I'm working on this now - I'll report back in a half hour.

Jamie

comment:21 Changed 10 years ago by https://id.mayfirst.org/jamie

The initial import is done :).

There were a few errors. The script didn't handle more than one user sharing an email address. Our system can handle it - but not the import script. I can tweak those by hand.

More seriously - user accounts must be unique across all of MFPL (to facilitate moving members between servers) - and we had three conflicts with your usernames: billc, jobs, and toml.

The only way we'll be able to add these accounts is to give them new usernames that are not already taken by another MFPL member. They can still have whatever email addresses you want, however, the name they use when logging in will have to changed. Can you add those three by hand with new usernames?

Thanks,

Jamie

comment:22 Changed 10 years ago by https://id.mayfirst.org/jamie

Ok - just got the missing emails inserted.

The only remaining issue is the three usernames.

Jamie

comment:23 Changed 10 years ago by https://id.mayfirst.org/quixote

toml -> tomlo billc -> billca jobs -> qcjobs

comment:24 Changed 10 years ago by https://id.mayfirst.org/jamie

  • Resolution set to fixed
  • Status changed from assigned to closed

Thanks for closing the loop on this ticket - I'm closing it as fixed.

For future reference - a gotcha of this script is that all email addresses must be explicitly defined (not just the aliases - see discussion here: ticket:1947:2).

jamie

Please login to add comments to this ticket.

Note: See TracTickets for help on using tickets.