Opened 7 years ago

Last modified 6 years ago

#5475 assigned Feature/Enhancement Request

XMPP/Jabber server for members

Reported by: Stephen Mahood Owned by: Stephen Mahood
Priority: Medium Component: Tech
Keywords: jabber xmpp ejabberd otr Cc: support-team@…
Sensitive: no

Description (last modified by Daniel Kahn Gillmor)

This began as an investigation and resulted in a successful install of ejabberd on im.mayfirst.org.

We met the goals set out that is to have a thorough and secure authentication system for all existing members to have immediate access to a XMPP chat interface that connects to other xmpp servers from riseup.net, jabber.org, members.fsf.org and googletalk.

There is also a irc transport which means with your chat account you connect to any irc server that you wish. Further information can be found on the wiki

This ticket documents the process to successfully install and configure the site, including the mistakes that were made that were later resolved.

Happy chatting!

Change History (35)

comment:1 Changed 7 years ago by Ross

Owner: set to Stephen Mahood
Status: newassigned

Stephen,

This is great research. I've not had the time to parse it all, but what I'd like to do is assign this ticket to you and make it your responsibility to get this on the agenda for one of our support face to face meetings. You also might want to post the ticket or your findings to the support team list.

thanks for looking into this,

~/ross

comment:2 Changed 7 years ago by Ross

Keywords: jabber added

comment:3 Changed 7 years ago by Stephen Mahood

Cc: support-team@… added
Keywords: xmpp added

jabberd 2.x is in active development and the research follows below:

The coding is done in xml, which might be more common for configuration

The documentation of Jabberd2 is more thorough but the authentication information is not clear from the documentation.

However similar functionality exists in comparison to ejabberd

Sasl is supported and specifically the GnuSASL as documented in the jaberrd mailing list. From the GnuSASL site channel bindings are an option.

I have been reading through the mailing list and joined it to find out more details on if channel binding is possible and access to an existing mysql db for account creation.

According to the email list there is a new version 2.30 coming out, I sent an email asking about debian repository support and await a response.

Last edited 7 years ago by Stephen Mahood (previous) (diff)

comment:4 Changed 7 years ago by Jamie McClelland

I could have sworn I opened a ticket for this... but I can't seem to find it. Mea culpa.

I've started some experimentation on this a few months ago on paul.mayfirst.org (the same server that is running freeswitch, our phone system). I installed and started experimenting with ejabberd. I was particularly interested in the sasl authentication option of ejabberd since we have the scripts available to synchronize our entire user account database so that in theory we should be able to automatically allow any MFPL user to login via their existing user account.

Stephen: I'd be happy to grant you access to this server to continue experimentation.

jamie

comment:5 Changed 7 years ago by Stephen Mahood

Owner: changed from Stephen Mahood to Jamie McClelland

comment:6 Changed 7 years ago by Jamie McClelland

So far the work I have done on paul.mayfirst.org has been to:

  • install ejabberd
  • I created a DNS record for word.mayfirst.org to point to 209.234.253.18
  • I created a mfpl cert authority signed x509 certificate
  • I made the following changes to /etc/ejabberd/ejabberd.cfg
    0 paul:~# diff -u /root/ejabberd.cfg /etc/ejabberd/ejabberd.cfg 
    --- /root/ejabberd.cfg	2012-03-20 20:41:34.000000000 -0400
    +++ /etc/ejabberd/ejabberd.cfg	2012-01-20 04:22:03.000000000 -0500
    @@ -55,10 +55,10 @@
     %% Options which are set by Debconf and managed by ucf
     
     %% Admin user
    -{acl, admin, {user, "__USER__", "__HOSTNAME__"}}.
    +{acl, admin, {user, "", "localhost"}}.
     
     %% Hostname
    -{hosts, ["__HOSTNAME__"]}.
    +{hosts, ["localhost", "word.mayfirst.org"]}.
     
     %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
     
    @@ -117,7 +117,7 @@
     			{shaper, c2s_shaper},
     			{max_stanza_size, 65536},
                             %%zlib,
    -			starttls, {certfile, "/etc/ejabberd/ejabberd.pem"}
    +			starttls, {certfile, "/etc/ssl/private/word.mayfirst.org.pem"}
     		       ]},
     
       %%
    @@ -226,7 +226,7 @@
     %%
     %% s2s_certfile: Specify a certificate file.
     %%
    -{s2s_certfile, "/etc/ejabberd/ejabberd.pem"}.
    +{s2s_certfile, "/etc/ssl/private/word.mayfirst.org.pem"}.
     
     %%
     %% domain_certfile: Specify a different certificate for each served hostname.
    1 paul:~# 
    
  • I configured paul via puppet to include the syncusers puppet recipe which will run the syncusers script on a cron job to ensure that all users are installed on paul and available via the local instance of sasld.
  • Using the command line ejabberdctl I created a user account in the default ejabberd user system for myself. And it works.

My next planned steps were to:

  • re-configure ejabberd to use sasl so all mfpl users can login

comment:7 Changed 7 years ago by Jamie McClelland

Owner: changed from Jamie McClelland to Stephen Mahood

comment:8 Changed 7 years ago by Stephen Mahood

After joining the Jabberd2.x email list I was able to determine a couple of things. First about XMPP and second about Jabberd2.x

  1. Unfortunately it seems with sasl that XMPP does not support keybinding as seen here this came from the maintainer on the launchpad site via the email list as seen in this thread
  1. Jabberd2.x is well maintained on Launchpad/guthub but as gdl mentioned in IRC the debian package lacks a maintainer <wink wink>.

In my email correspondence with the jabberd2.x email list it on their upcoming release I asked here if it would be supported in the debian repos since the planned release is 2.30 which is a number of versions higher than the 2.8 in debian sid. The launchpad maintainer mentionded the need for a maintainer on the debian package, and one correspondent mentioned he maintains jabberd2 on a debian wheezy install. I asked that correspondent if he was interested in maintaining the debian package (he's looking into but said not to keep my hopes up).

Though bugs are handled on launchpad at the moment github is the main site for the project.

Question at this point: Though ejabberd is more widely used I am wondering what peoples thoughts are on using software that is written in a language not many know vs the potential of using Jabberd2.x which is written in a more common XML (the writer, me, will admit to not knowing either so incite would be nice :) )

Last edited 7 years ago by Stephen Mahood (previous) (diff)

comment:9 Changed 7 years ago by Jamie McClelland

Hi Stephen - thanks for all the research on this.

My 2 cents:

  • I'm not concerned because erlang is an uncommon language. I'd prefer a more standard language, but I think whether or not the package itself is widely used and supported is more important to me. And, in that regard, it seems like ejabberd is the most widely used.
  • I'm new to the concept of channel binding, but it seems important. Do other methods of authentication support it (e.g. Pam, or anything else that we could use)?
  • Having the ability for people to use their own email addresses as their XMPP login would be cool... but comes with other difficulties: would we allow un-encrypted connections? if not, how would we ensure we have proper x509 certificates or other ways to verify the host? Given these concerns and the fact that we could potentially grant everyone with a user account an automatic XMPP id of: <user_login>@word.mayfirst.org - I would opt for doing this easier step first and work on supporting virtual hosts at a later time.

comment:10 in reply to:  8 Changed 7 years ago by Daniel Kahn Gillmor

Replying to https://id.mayfirst.org/mahood:

Though ejabberd is more widely used I am wondering what peoples thoughts are on using software that is written in a language not many know vs the potential of using Jabberd2.x which is written in a more common XML

FWIW, XML is a data interchange format, not a programming language; i think jabberd2 is written in C, though it does appear to use some flavor of XML as its configuration language.

I have no objection to erlang; i'd be more concerned about decent attention from the maintainers of the software than about the choice of programming language, as long as there are free compilers/interpreters for the languages used (both erlang and C pass this test).

The lack of channel bindings for SASL is bad, but that seems baked into XMPP's use of SASL itself, if i'm reading your link right. Perhaps we could offer client-certificate support instead for users who care about avoiding replay attacks? client-side certificate authentication should properly offer channel binding since it happens within TLS itself.

comment:11 Changed 7 years ago by Stephen Mahood

Reply to Jamie The wide use of ejabberd attracts me to just thinking farther down the road if xml would be easier to hack on if we so choose too, but I am willing to balance functionality over hackability unless others want to chime in.

You and me both, dkg might be the best person to reference on channel binding itself, but from the [documentation of ejabberd PAM brings a different security concern forward as documented here

"To perform PAM authentication ejabberd uses external C-program called epam. By default, it is located in /var/lib/ejabberd/priv/bin/ directory. You have to set it root on execution in the case when your PAM module requires root privileges"

I am ok with getting the easier step first, but feel the integration to email is a feature to keep on the priority list. jamie you had mentioned the mail.mayfirst.org access to accounts, what does that mean in relation to virtual hosting and security?

Reply to dkg

Client Certificate support would that mean users would likely need a specific client to make use of the service?

Last edited 7 years ago by Stephen Mahood (previous) (diff)

comment:12 Changed 7 years ago by Stephen Mahood

In the continued research into the appropriate server software, I contacted some sysadmins on sites running different software for xmpp. In a non-scientific sense ''Prosody'' is more used and in large scale environments as well, however it is not the favored (me at least) GPL license but the MIT license instead which is acceptable too. It is also up to date in the debian repos.

One example of Prosody koumbit anarcat from the #riseup mentioned using it and provided memory usage details but pointed out that the number of users is low.

A friend (@Deepspawn) runs both a status instance and a jabber service and with 20 users on the jabber. He had tried a number of different free software variants of XMPP and offered opinions (from an IM chat) on ejabberd. "Ejabberd tends to use a lot of memory but doesn't grow that much with user load...ejabberd has the most completed and abandoned xmpp support...as in there's a lot supported but hasn't received much attention over the years" and "prosody is written on lua w[h]ich is not a popular language but is one with rapid growth and has proved to be suitable for large setups, the ram usage is low even on large setups but it has less features than ejabberd"

Prosody Research

  1. XMPP Extension support
  2. Encryption includes sasl
  3. Documentation is pretty clear and seems well maintained.

I feel after playing with the installed ejabberd this might be worth exploring. (might be more vegan friendly since it's not a elephant in relation to excess obsolete components in ejabberd).

Jamie...can I get access to the ejabberd experiment?

comment:13 Changed 7 years ago by Jamie McClelland

Yes, I'd be happy to give you root on paul to play around with ejabberd. Can you paste your public ssh key to this ticket and I'll put it on?

I think that giving people an xmpp user name that matches their email address will end up being very difficult to implement and confusing for many users (since we have users that have an email address that forwards to another address so there is not login info associated with it and therefore we wouldn't be able to authenticate them).

On the other hand, with either PAM or SASL integration, we can provide a user_account@talk.mayfirst.org xmpp account without very much hassle at all. I've already configured scripts on paul so that our entire user account database is synchronized to paul's /etc/passwd and /etc/shadow files. By configuring ejabberd to use these files via sasl, everyone MFPL member will automatically have a login using a password they already have access to. Everyone needs to know their user login (which is relatively known amongst people who use webmail or login to support and is readily available to everyone who has one). It also publicizes MFPL.

The PAM security issues are a concern - but I think we can get around that by using SASL instead.

As for the channel binding - is it possible to offer sasl authentication by default and configure client-certificate authentication to users who request it?

I think ejabberd is our best option - I think we can get it running with the least amount of work and since it's maintained in Debian, we can maintain it with little hassle.

If, after a few months of use, we decide we need something different, we can always migrate to a different server.

jamie

comment:14 in reply to:  13 Changed 7 years ago by Daniel Kahn Gillmor

Replying to https://id.mayfirst.org/jamie:

Can you paste your public ssh key to this ticket and I'll put it on?

There was no need for this; mahood was added via the monkeysphere.

comment:15 Changed 7 years ago by Stephen Mahood

Keywords: chidolista added

comment:16 Changed 7 years ago by Stephen Mahood

Testing on mcchesney in word.mayfirst.org Currently ross and marxistvegan have been working on the ejabberd install on mcchesney under the hosting order of word.mayfirst.org

At the moment the software is configured a created user can log in, but we have an issue in the communication. Specifically initiated message is read by the recipient but the recipient is unable to respond.

Decided to make the word.mayfirst.org have a dedicated IP address

comment:17 Changed 7 years ago by Ross

In order to allow external servers to communicate with our jabber client, we needed to modify the dns entries and add srv entries for word.mayfirst.org.

We did this by adding the following lines to /etc/tinydns/root/special/records on viewsic:

:_xmpp-server._tcp.word.mayfirst.org:33:\000\005\000\000\024\225\004word\010mayfirst\003org\000:86400
:_xmpp-client._tcp.word.mayfirst.org:33:\000\005\000\000\024\146\004word\010mayfirst\003org\000:86400

~/ross

comment:18 Changed 6 years ago by Stephen Mahood

though the service is functional there are issues with using otr and other encrypted features.

It has resulted in OTR being validated but upon second attempt to communicate the error of private session is being tried but never turns to be successful, resulting in no communicaton being able.

I want to see if we can complete this project soon so that we can also eventually work on using the chat feature in friends.mayfirst.org

comment:19 in reply to:  18 Changed 6 years ago by Daniel Kahn Gillmor

Replying to https://id.mayfirst.org/mahood:

though the service is functional there are issues with using otr and other encrypted features.

OTR is ostensibly messaging-layer agnostic. I don't think the configuration of the server is supposed to even be aware that OTR is being used. Are you sure this is a server configuration issue?

It has resulted in OTR being validated but upon second attempt to communicate the error of private session is being tried but never turns to be successful, resulting in no communicaton being able.

What clients are you testing your OTR connections with?

comment:20 Changed 6 years ago by Daniel Kahn Gillmor

Keywords: otr added

comment:21 Changed 6 years ago by Stephen Mahood

dkg,

valid questions...

Are you sure this is a server configuration issue?

I am wondering if it is, I am not certain but it seems plausible since I am able to gain the OTR settings with an @jabber.org account. Ross and I had trouble with this which we did not document as it was in passing.

What clients are you testing your OTR connections with?

At the moment I am using pidgin.

comment:22 Changed 6 years ago by Jamie McClelland

The system being functional is great news! Is their a wiki page with instructions for how I can setup an account? I would love to give it a whirl.

comment:23 Changed 6 years ago by Stephen Mahood

Now in testing phase im.mayfirst.org has instructions to test the demo.

The next step is to connect erland to authenticate with the existing membership database so that users can begin using the xmpp service with their existing accounts for openid and email.

Also to note the im.mayfirst.org server for xmpp has an ssl certificate :)

~marxistvegan

comment:24 Changed 6 years ago by Stephen Mahood

Keywords: ejabberd added; chidolista removed
Summary: XMPP/Jabber server for members -- ResearchXMPP/Jabber server for members

comment:25 Changed 6 years ago by Stephen Mahood

Because of an issue of /etc/init.d/ejabberd * was throwing errors, we had used the only other means to run the ejabberd service, that is we used root. This turned out to be big mistake as ejabberd user was unable to run the /etc/init.d/ejabberd as the script called to do.

This was because since the ejabberd command from root took ownership of key files needed to run the /etc/init.d/ejabberd script.

To resolve this the /etc/ejabberd and /var/lib/ejabberd directories of the current install were saved tarballed and saved in a local private directory for later use.

then I proceeded with purging the ejabberd install

apt-get purge ejabberd

followed by

apt-get install ejabberd

Proceeded to take a look at the /var/lib/ejabberd and the difference of ownership and permissions, which was clearly by default set for ejabberd:ejabberd which was in stark contrast to the previous configuration.

Copied 4 specific (/var/lib/ejabberd/.my.cnf, /etc/ejabberd/user-auth.py, /etc/ejabberd/ejabberd.cfg , /var/lib/ejabberd/auth-config.ini) files from the tarball for configuration and restarted the ejabberd services.

The services now run with the correct permissions and ownership of the /var/lib/ejabberd directory.

Any user that was used before today will need to re-add their buddies as the previous database was removed and a new one of buddy lists is created.

comment:26 Changed 6 years ago by Stephen Mahood

Owner: changed from Stephen Mahood to Ross

ross

can you update this ticket with the database magic? if not then lets close this :)

~mv

comment:27 Changed 6 years ago by Stephen Mahood

Description: modified (diff)

comment:28 Changed 6 years ago by Ross

We are using the same process for the ejabberd database as we are for the mfpl-openid module. That method can be viewed from this changeset. We should probably put some of the ejabberd work into a git repo, at least the user-auth.py file, and probably ejabberd.cfg.

~/ross

Last edited 6 years ago by Ross (previous) (diff)

comment:29 Changed 6 years ago by Daniel Kahn Gillmor

I note that ejabberd.cfg has permissions on mcchesney that suggest it is not intended for public consumption (maybe there are usually secrets stored there, like authentication credentials or something)?

I think that publishing the configuration is a healthy thing to do as long as someone has verified that the contents of the file are really safe to share publicly.

Last edited 6 years ago by Daniel Kahn Gillmor (previous) (diff)

comment:30 Changed 6 years ago by Stephen Mahood

Made changes to the ejabberd.cfg specifically commented out

 %% {mod_register, [  
                  %%
                  %% After successful registration, the user receives
                  %% a message with this subject and body.

%%                {welcome_message, {"Welcome!",
%%                                   "Welcome to a Jabber service powered by Debian. "
%%                                   "For information about Jabber visit "
%%                                   "http://im.mayfirst.org"}},
                  %% Replace it with 'none' if you don't want to send such message:                                                                 
                  %%{welcome_message, none},                                                                                                        
                                                                                                                                                    
                  %%                                                                                                                                
                  %% When a user registers, send a notification to                                                                                  
                  %% these Jabber accounts.                                                                                                         
                  %%                                                                                                                                
%%                {registration_watchers, ["admin1@example.org", "mv@im.mayfirst.org"]}                                                             
                                                                                                                                                    
                  %%{access, register}                                                                                                              
%%               ]},                                                                                                                                

As mod_register technically allows others to try to register an account, since we are using our own db for accounts this is irrelevant. Made aware of this from ejabberd@… chatroom discussion

~mv

comment:31 Changed 6 years ago by Ross

Resolution: fixed
Status: assignedclosed

I do believe this ejabberd ticket can be closed.

comment:32 Changed 6 years ago by Daniel Kahn Gillmor

Description: modified (diff)

comment:33 Changed 6 years ago by Ross

Resolution: fixed
Status: closedassigned

comment:34 Changed 6 years ago by Ross

Owner: changed from Ross to Daniel Kahn Gillmor

Apparently, this ticket needs something more to happen in order to be closed.

comment:35 Changed 6 years ago by Ross

Owner: changed from Daniel Kahn Gillmor to Stephen Mahood

Please login to add comments to this ticket.

Note: See TracTickets for help on using tickets.