Opened 12 years ago

Closed 7 years ago

#168 closed Bug/Something is broken (wontfix)

Fix openid drupal module to enable drupal users to be created without an email address

Reported by: alfredo Owned by: Jamie McClelland
Priority: High Component: Tech
Keywords: openid drupal blogs.mayfirst.org Cc:
Sensitive: no

Description

I understand that Jamie and Daniel logged into the blog site using OpenID and Drupal successfully created users for them with full data entered into the users table. This did not happen when I attempted to log in with "alfredo" (my OpenID username). Instead, the site returned an error:

Unable authenticate. OpenID Discovery failed.

I'm wondering if there's something missing in my OpenID login data or if there's something wrong with the Drupal. Can anyone shed light?

Change History (42)

comment:1 Changed 12 years ago by alfredo

Resolution: invalid
Status: newclosed

comment:2 Changed 12 years ago by Daniel Kahn Gillmor

Why is this invalid? It sounds like it could be a real problem.

comment:3 Changed 12 years ago by Jamie McClelland

I didn't notice that it was closed before mucking around and learned:

If you type your identity URL like this:

http://id.mayfirst.org/jamie

You will get the error Alfredo reports.

Entering properly like this:

https://id.mayfirst.org/jamie

Will work.

I also discovered that if your email address is not configured with our OpenID provider, you will get an error message about the email address being a required field and Drupal will quite nicely prompt you to enter the email address. However, if you do that, you get a new error saying that your username contains invalid characters. Sigh.

If you go back and enter your email address in the OpenID provider, then your account will be properly created.

comment:4 Changed 12 years ago by Daniel Kahn Gillmor

Resolution: invalid
Status: closedreopened
Summary: OpenID not creating new user on blogsmf siteOpenID occasionally not creating new user properly on blogsmf site

Thanks for the debugging, jamie.

Why does drupal require an e-mail address? Can we configure it to not do so? I'd love it if this could be an e-mail-free system for those of us who want to try to move away from SMTP.

Reopening ticket because this seems like a legitimate concern to me.

comment:5 Changed 12 years ago by Jamie McClelland

And lastly... if you go through the path I went through above with my alternate OpenID login, you get the message about validating your email address that Alfredo mentions elsewhere. You don't get that message if your email address was already configured in the OpenID setup.

comment:6 in reply to:  4 Changed 12 years ago by Jamie McClelland

Why does drupal require an e-mail address? Can we configure it to not do so? I'd love it if this could be an e-mail-free system for those of us who want to try to move away from SMTP.

I think the only logic is for resetting passwords, which is moot with openid. It might also be a step to confirm identities to something, also moot with OpenID.

comment:7 Changed 12 years ago by Jamie McClelland

The Alfredo mentions elsewhere link in my comment above should be:

Ticket 163, comment 1

comment:8 Changed 12 years ago by Daniel Kahn Gillmor

Replying to https://id.mayfirst.org/jamie:

I think the only logic is for resetting passwords, which is moot with openid. It might also be a step to confirm identities to something, also moot with OpenID.

Can we configure it not to require an e-mail address, then?

comment:9 Changed 12 years ago by alfredo

Drupal uses the email address for two reasons:

1 -- to confirm a registration and validate it -- which OpenID should make unnecessary but this module for Drupal appears to not integrate with OpenID deeply enough. Just does preliminary authentication and then let's Drupal's registration system take over.

2 -- to assure easy communication with authenticated users. While this isn't totally necessary in our system, there are many modules in Drupal that use this email record including Organic Groups (which we may eventually use), some of the email notification programs (which might be useful to us at some point given our other "communication with members" discussion) and advisories and stuff if we want those.

What we'd need to do to eliminate this is to write some code that installs the username and domain accurately into the Drupal tables -- i.e. a call to the appropriate red table and then an insert command into Drupal. I think it's much easier to keep the info.

Doing that, we could do what Daniel says.

But!!!...

My feeling is that the best way to handle all this is to register this person as a user when the user is first created. In other words, to rig it so that no one has to register to this site -- they use their login for OpenID and that's that.

Is that doable with OpenID/Drupal?

comment:10 in reply to:  9 Changed 12 years ago by Daniel Kahn Gillmor

Replying to https://id.mayfirst.org/alfredo:

Just does preliminary authentication and then let's Drupal's registration system take over.

This was not my experience. I simply logged in with my OpenID and proceeded to post content. Granted, i do have an e-mail address configured in my MFPL OpenID identity. It was a much clumsier process when i tried with my LiveJournal OpenID, which does not have an e-mail address associated with it. This leads me to believe that the issue is that drupal is requiring an e-mail address, which i'd rather not have it do.

I understand that we can't do e-mail notification if we don't have a valid e-mail address for the user, but then it should be up to the user to include a valid e-mail address.

I don't think the way to resolve this for MF/PL members (remember that other people have OpenID accounts too, and could use this system) would be to pre-populate the drupal tables. If we've got that data (default e-mail for the member), i suspect it'd make more sense to prepopulate the OpenID database itself with the default e-mail for the member, so that any OpenID consumer that requires an e-mail address can ask for it. (of course, people should be able to decline to give that information, also, even if they use an OpenID).

comment:11 Changed 12 years ago by alfredo

Why not pre-populate? Wouldn't that save the member the step of having to register on the Drupal site? And what's the reason not to do it? We need to have them in the user tables in order to have a drupal uid for just about everything so they must be in there one way or the other, right?

comment:12 Changed 12 years ago by Daniel Kahn Gillmor

Alfredo, what i'm saying in comment:10 is that i did not have to register on the blogsmf.mayfirst.org. I just used OpenID to log in; that's all there was to it. There was no need for pre-population for me (unless you or someone else actually did pre-populate the drupal users table with my account info, and i just didn't know about it). I'm not sure why that couldn't be the way it would be for everyone.

One reason not to pre-populate tables is data synchronization. It's one thing to set up something new when a new user is created. but how will we keep any data in sync? If an MFPL account is removed, should we remove the pre-populated account? If the details of an MFPL account are changed, should we remove or update the pre-populated account? These questions are valid for the non-pre-populated case where data is shared, too, but pre-populating means synchronizing significantly more data, including a lot of data that might never be touched (e.g. pre-populated drupal accounts that are never used).

comment:13 Changed 12 years ago by alfredo

I'll try and register a couple of other people and see what happens.

comment:14 Changed 12 years ago by alfredo

There is no question that OpenID does *not* automatically register people -- not in all cases anyway. I have attempted login for two people not yet registered on the blog site using OpenID login.

I am sent to the normal user/pass page (https://id.mayfirst.org/server/)

and then when I authenticate on that screen I am sent to another screen that tells me OpenID is tranferring my username and email and to approve that (all as it should be)

*then*

I'm sent to a drupal page that says:

  • E-mail address field is required.
  • OpenID registration failed for the reasons listed. You may register now, or if you already have an account you can log in now and add your OpenID under "My Account"

and has the normal registration form for the creation of a new account.

A drupal registration! :-)

And I'd like to avoid that. Because it's a deterrent for our members. It's discouraging. I don't want it to work this way.

If pre-population is not the answer, what is? I'd really like us to come up with a simpler alternative than the above one if possible.

comment:15 Changed 12 years ago by Daniel Kahn Gillmor

I agree that the above set of steps is excessive. Let's whittle them down.

I had already (several weeks ago) stored an e-mail address in my openid account at https://id.mayfirst.org/. Here's what happened for me when i tried to sign up for blogsmf:

  • On http://blogsmf.mayfirst.org/, I clicked on the "log in using openid" link. That replaced the two-field form with a single field for my OpenID.
  • I typed my openid into the field (https://id.mayfirst.org/dkg)
  • I clicked the "Log In" button
  • I was taken to our OpenID provider service (on https://id.mayfirst.org/) which prompted me to authenticate.
  • I authenticated to our OpenID provider, and was told that blogsmf.mayfirst.org wanted to know my nickname and my e-mail address. I clicked "Allow Once"
  • I was returned to http://blogsmf.mayfirst.org/, and was already automatically logged in, using my preferred nickname.

Simpler than what you described going through, above, but i'd like it to be even simpler than that. We could:

  • remove any non-OpenID login option, so the first step i took would disappear
  • not require an e-mail address, so the last step would be painless (a nickname is probably already known by our openid provider. if not, it should be generatable by the OpenID provider from the path). Not requiring an e-mail would mean that we wouldn't cause problems for OpenID users who don't have an e-mail associated with their account.

Can we do those things and see if your use case clears up also? Do we have dummy accounts to test with?

What will it take to get drupal to not require an e-mail address?

comment:16 Changed 12 years ago by Daniel Kahn Gillmor

Removing non-OpenID logins would also let us remove all the links below the User Login box. "Create New Account", "Request New Password", and "Log In Using OpenID" would all be gone, as would the "Password" text entry box.

less is more!

comment:17 Changed 12 years ago by Jamie McClelland

I had the same experience as dkg here. I don't think it's a problem with the OpenID code on Drupal. I think that the openid code is properly creating a drupal account provided:

  • You have your email address configured via open id OR
  • Drupal allows you to create a user account without an email address

I'm not sure dkg's solution is the best solution. It seems like anyone writing code for Drupal could safely assume that users have an email address. If we hack that out of Drupal (which would require hacking core drupal code), we risk building a broken system that is difficult to maintain.

Another option would be to require users who login via our OpenID Provider code to enter an email address if they haven't already done so. Our Drupal login is not the only OpenID consumer code that will want someone's email address - so it's not unreasonable to ask people to populate a single field.

comment:18 Changed 12 years ago by Daniel Kahn Gillmor

That's what i'm asking, though, jamie -- would it actually require hacking drupal core to remove that assumption? I haven't seen any documention that it is a requirement for the e-mail addresses to be valid (or even present). Do you have links?

i've just unchecked "Require e-mail verification when a visitor creates an account" on the user settings administration page. Maybe that's the first step?

comment:19 Changed 12 years ago by alfredo

Damn! I thought you'd solved if, Daniel, but no. I believe your configuration action makes an email account unnecessary if one is registering using the normal drupal registration system but, with the OpenID authentication, you return to that drupal page that insists you enter a valid email address.

I think the OpenID module is configured that way (assuming Drupal's "default" about valid email).

So I think the answer to Daniel's question about whether core code hacking is necessary is "No, it appears not to be." Drupal apparently can register someone without an email; hence the checkbox DKG unchecked. But I have a feeling that the Drupal OpenID module is hard-coded somehow to expect the email address.

And I'm also assuming that the presence of an email in you guys' OpenID accounts is the different in our experiences. I guess mine doesn't have that and the various accounts I'm using to test this (my family accounts, basically) don't have it either. That's the difference: Drupal's OpenID module wants the email account and OpenID can't provide it because it doesn't have it.

Well...we're starting to identify the source of the problem, at least. First steps toward a solution, no?

Wouldn't it be great if there were some documentation for that OpenID module -- even a useful readme? Nothing useful there. :(

comment:20 Changed 12 years ago by Daniel Kahn Gillmor

I've never seen the code you've used to get us this far, Alfredo. Can you link to the specific module we're using? When i google OpenID and Drupal, i get a lot of different stuff, some of which is clearly irrelevant (e.g. provider code, or modules/discussion for drupal6). Can you point me toward what we've got?

comment:21 Changed 12 years ago by Jamie McClelland

Wow - who knew about that check box? I stand corrected!

If the OpenID code is the only obstacle we're in pretty good shape.

Google is surprisingly unhelpful in finding the openid module. Based on taking a peek at the code on malcolm (where blogsmf.mayfirst.org is currently hosted) and doing a search on the drupal site itself, it appears as though the code being used is:

http://drupal.org/project/openid

In the course of searching, I came across:

http://walkah.net/blog/walkah/drupal-6-and-openid

Where I learned that OpenID consumer support is included in Drupal 6 core (yay!!)

A first glance through the code seems to indicate that the author intends us to use the code in the way dkg is describing - so I think it's a question of finding the bug.

comment:22 Changed 11 years ago by Jamie McClelland

Summary: OpenID occasionally not creating new user properly on blogsmf siteFix openid drupal module to enable drupal users to be created without an email address

Fixing title to be more accurate based on discussion.

comment:23 Changed 11 years ago by alfredo

I think we really either prepend the correct url to the login or give a very precise instruction about how to do it. Because it's really confusing. People aren't gonna assume they need to include a website address as part of their login. :-)

I have success with

https://id.mayfirst.org/myusername

But it took me several tries to remember to do that. :( I think we need to decide which choice to make (prepend or instruction)?

comment:24 Changed 11 years ago by Jamie McClelland

I think we should do both prepend and instructions.

comment:25 Changed 11 years ago by Daniel Kahn Gillmor

Here's what i think the ideal "prepend" strategy would be, in detail:

  • the OpenID login box should be initially pre-seeded with https://id.mayfirst.org/. Using javascript, when the focus enters the box, the cursor should be placed at the end of the URL
  • the login box should be wide enough (or the text small enough) to be able to see the entire URL clearly using "typical" font settings, whatever that means.
  • If a user decides to delete everything in the box, and just write a raw username (e.g. fubar), the code handling the form should treat it as if they had entered https://id.mayfirst.org/fubar.

As for instructions, i think a "What is OpenID?" link that takes the user to a special drupal page within the site with explanation (and another login box) would be a good option. I wouldn't want more than that cluttering up the login section.

comment:26 Changed 11 years ago by Jamie McClelland

Whatever fixes we make to the openid login for the blogs site should also be made to the new May First/People Link web site as well (see ticket #252).

comment:27 Changed 11 years ago by Daniel Kahn Gillmor

Keywords: openid drupal added

comment:28 Changed 11 years ago by alfredo

When Open ID creates a user record for someone, it seems to establish a username that is actually the open id url. Check out, in our new website, the users (you must log in as admin to do it) and you'll see me as my open id. I don't think this is what we want.

This is only when Open ID seeks to create a user; not when a user is already there created in some other way. Then it works fine.

The blog site acts a bit better but it's doing strange stuff. It inserts the login url as the name but it allows you to change the name by going to the user form. Our new main site doesn't do that.

All in all, however, the user is going to have to do a bit of work to make all this happen and have a good bit of knowledge about how the user, username and blogname work on Drupal. Most don't.

I'm concerned because this process is a lot for our members. Getting them to blog is going to take some effort and another obstacle to their participation is not what we need right now.

I'm not sure if it's wishful thinking but isn't there some way of making more of this process automatic? Are we absolutely sure that pre-popuation isn't the way to go. I just reread Daniel's arguments against it: all of which I agree with. But I also think pre-population and syncing (were it possible) would be a great help.

Barring that:

prepend Open ID forms with https url

Ensure that Open ID works the same on both sites -- I would say it's closer to correction on blogsmf now.

write very clear instructions on what, exactly, to do

I'll do the instructions.

comment:29 in reply to:  28 Changed 11 years ago by Jamie McClelland

Barring that:

I think this is where we should be now: figuring out what we want to have happen. Once we know that, we can try to get the openid module to do it as cleanly as possible.

In addition to your suggestions:

  • prepend Open ID forms with https url
  • Ensure that Open ID works the same on both sites -- I would say it's closer to correction on blogsmf now.
  • write very clear instructions on what, exactly, to do

I would add:

  • Remove (or at best make less prominent) the option to login with a username and password. It should only say: Login with OpenID (otherwise people will enter their usernames in the normal login box and they will wonder why it doesn't work).
  • Restrict to MFPL openids. This is actually a question - do we want to allow other users who have OpenID accounts to login? I would say that on both the blogs site and the main web site that all OpenID accounts can post comments, but only MFPL accounts can create blogs (on the blogs site). On the www site you need special permission to do anything other than comments (regardless of whether it's an MFPL open id or not).
  • Use short usernames rather than full openid usernames (see conflict scenario below)

I think we will want to test the following scenarios and make sure they all work:

  • MFPL members logs in with an OpenID account in which the email is not set in open id
  • MFPL members logs in with an OpenID account in which the email is set in open id
  • A MFPL member logs in with https://id.mayfirst.org/jamie and is given the username 'jamie'. Next, a non MFPL member logs in with https://livejournal.org/jamie and is presented a screen that says: the username jamie is already taken. What username would you like?

comment:30 Changed 11 years ago by Jamie McClelland

Keywords: www.dev.mayfirst.org blogsmf.mayfirst.org added

comment:31 Changed 11 years ago by Jamie McClelland

This issue is mostly completed.

Now:

  • Users can login with or without an email. If OpenID does not provide an email, then a fake @example.org is inserted for them. While Drupal allows you to not require email validation, there is no simple way that I can find to allow users to be created without an email address.

  • The user login form is changed so as not to allow old-style logins - only openid logins
  • The password reset and registration forms are disabled
  • The openid box is pre-seeded with the MFPL openid URL and there's a brief explanation.
  • OpenID extracts the most likely username portion of the OpenID and uses that as the drupal login, rather than trying to use the entire OpenID identity URL.

Still not done: I'm not sure what will happen if a user logs in with an OpenID that has a username already on the system.

comment:32 Changed 11 years ago by Daniel Kahn Gillmor

When i go to log in to the blogs site with my MF/PL OpenID, the OpenID provider still says "required" next to the "e-mail address" profile information, though it sounds like "optional" would be more accurate. Can we switch that?

comment:33 Changed 11 years ago by Daniel Kahn Gillmor

Since we have mod_auth_openid, we might also be interested in Drupal's webserver_auth module, or drupal-remoteuser, both of which rely on Apache's $REMOTE_USER environment variable, which can be set by mod_auth_openid.

comment:34 Changed 11 years ago by Jamie McClelland

Regarding webserver_auth an ddrupal-remoteuser, wow. That looks like a completely different approach that would allow us to avoid OpenID altogether. I wonder how they handle automatic account creation? I wonder if they rely on the admin setting set to allow registration? It certainly would be worth looking into to see if it's more elegant than what we are doing with openid. OTOH - the openid module is built-in to Drupal 6 core - so in the long run it might be better supported.

As for the email - drupal has no out of the box way to say that email is required. You can only say that email verification is required or not. I'm not sure if there is a clean way to change that.

comment:35 Changed 11 years ago by alfredo

Are we convinced that people who have OpenID login capability on the site itself (as opposed to blogs). I'm not sure I see the logic of that. In fact, I think we should consider removing OpenID from most of the pages if we even use it. Maybe a traditional login box on admin and user might be more effective since only admins are gonna log in, right?

comment:36 Changed 11 years ago by Jamie McClelland

For www.dev.mayfirst.org - I think your suggestion is a good one. Removing OpenID and the login box would be an improvement - both for the reason you mention (that site has very limited content so we if people are unable to add comments to it, it's not a great loss) and for a second reason: I think our members, when they are lost of confused (maybe the first time they try to login to webmail) go to our main web site and try to login to the first login box they see - then they get confused when they don't see their email. I think by eliminating the login box entirely on www.dev.mayfirst.org, we could help alleviate this problem.

I'd like to keep this ticket open - to address the possibility of changing our OpenID strategy on blogs.dev.mayfirst.org, but I would agree with not doing this for www.dev.mayfirst.org.

comment:37 Changed 11 years ago by alfredo

Resolution: fixed
Status: reopenedclosed

Login block removed -- forcing people to use /user page for login.

OpenID (which would permit logins by everyone with an OpenID) is now disabled.

closing ticket

comment:38 Changed 11 years ago by Daniel Kahn Gillmor

Resolution: fixed
Status: closedreopened

Alfredo, i think your most recent changes to www.dev.mayfirst.org sound like good ones. But i'm not sure that they address the main point of this ticket, which (according to the summary) is to change the OpenID module to let OpenID users create their accounts without requiring an e-mail address. Can you explain why your changes address this before closing the ticket?

comment:39 Changed 11 years ago by Jamie McClelland

Keywords: www.dev.mayfirst.org removed

I asked Alfredo to close this ticket as we tried to go through the www.dev.mayfirst.org tickets. I've just now removed www.dev.mayfirst.org from the key words, since this ticket is no longer relevant to www.dev.mayfirst.org with Alfredo's last change (which seems like a better idea).

And, I think the issue in the summary was actually resolved earlier.

Although, after reading your comment requesting that the message indicating that email address is required be changed to optional, I did some more research on this overall issue of requiring an email address and am having some doubts on our strategy. I think our strategy might not be the best one. Here are some strategies:

  • Hack OpenID (our current strategy): Add functionality to OpenID that achieves the goal of making it seem as though Drupal does not require an email address by auto-generating a fake email address. Try to add another hack to OpenID to resolve the required/optional issue noted above.
  • wontfix. If a user logs one with OpenID, and their email address is not provided by the OpenID provider, prompt the user to enter their email address.

dkg posted another comment proposing different approaches to OpenID, which we should examine, however, I suspect that the fundamental issue of not requiring an email address goes beyond how we do OpenID and is in fact a Drupal core issue. Therefore, I think taking a different approach to providing OpenID might be better discussed in a different ticket so this ticket can stay focused on the issue of requiring or not requiring an email address.

comment:40 Changed 11 years ago by Daniel Kahn Gillmor

Keywords: blogs.mayfirst.org added; blogsmf.mayfirst.org removed

comment:41 Changed 8 years ago by Jamie McClelland

Resolution: wontfix
Status: reopenedfeedback

I'm changing this to feedback/wontfix. I'm not sure it's worth the effort to fix. If someone else wants to take it on, I'll all for it.

comment:42 Changed 7 years ago by automatic

Status: feedbackclosed

No news is good news (we hope)! Given the lack of feedback, we think this ticket can be closed.

Please login to add comments to this ticket.

Note: See TracTickets for help on using tickets.