Opened 7 years ago

Closed 7 years ago

#5435 closed Bug/Something is broken (fixed)

How to get ip addresses for users in drupal with varnish

Reported by: Ross Owned by: Ross
Priority: Medium Component: Tech
Keywords: saharareporters drupal varnish Cc:
Sensitive: no

Description

Hi Jamie,

For some reason, the comment function that pulls in ip addresses from users making does not work correctly (consistently) for saharareporters.com. This is almost certainly caching related, but I cannot seem to figure out how/why it now fails.

The real conundrum is that some ip addresses do get stored correctly, just not most of them. The following output shows the ten most recent distinct ip addresses:

mysql> select distinct hostname from comments order by cid desc limit 10;
+-----------------+
| hostname        |
+-----------------+
| 209.234.253.250 |
| 216.66.23.43    |
| 210.170.35.21   |
| 178.223.221.47  |
| 109.230.216.225 |
| 109.230.216.60  |
| 115.249.252.235 |
| 110.84.58.63    |
| 98.126.95.98    |
| 96.31.86.184    |
+-----------------+
10 rows in set (0.03 sec)

mysql> 

And here's the most recent non-mfpl server IP Address.

mysql> select FROM_UNIXTIME(timestamp) from comments where hostname = "210.170.35.21";
+--------------------------+
| FROM_UNIXTIME(timestamp) |
+--------------------------+
| 2012-03-09 22:57:00      |
+--------------------------+
1 row in set (0.35 sec)

mysql> 

A couple of days ago. Any thoughts on how to mitigate this problem. It seems like it began around the time we switched off of debord for varnish around the beginning of February.

~/ross

Change History (7)

comment:1 Changed 7 years ago by Ross

Owner: set to Jamie McClelland
Status: newassigned

comment:2 Changed 7 years ago by Jamie McClelland

Owner: changed from Jamie McClelland to Ross

It's possible to access saharareporters.com using the standard way (which goes through varnish and will appear to apache to be coming from one of our varnish servers) or directly via apache (using apache.saharareporters.com and will appear to come from the IP address of the actual user).

I suspect that the recent IP addresses that are not MFPL IPs are coming via apache.saharareporters.com (but I can't really explain how or why that address is being used).

Somethings I'm not sure about: if varnish decides that your query should not be cached because you are logged in... does it pass through your IP address so apache knows your IP address? Or does it always seem to apache like it's the varnish IP? This might require some testing.

I'm assigning this back to you Ross because I don't really know the answer! It is possible that one of the sacrifices of using varnish is that we loose the IP info of the connecting user :(.

jamie

comment:3 Changed 7 years ago by Bart

A solution might be to set an X-FORWARDED-FOR header:

sub vcl_recv {
   set req.http.X-Forwarded-For = client.ip;
}

The IP address can then be accessed in Drupal via

$_SERVER["HTTP_X_FORWARDED_FOR"]

There seems to be a problem with Varnish 3.0 in combination with PHP 5.3, although this probably won't be an immediate showstopper as Squeeze contains Varnish 2.3.1.

comment:4 Changed 7 years ago by Ross

Resolution: fixed
Status: assignedclosed

I did manage to fix this, but I had to hack drupal core to do so :(. In bootstrap.inc I modified the ip_address() function to set the ip_address variable to $_SERVERHTTP_X_FORWARDED_FOR? by default.

Not the best solution, but the only one I could find that could produce the results I needed.

comment:5 Changed 7 years ago by Greg Lyle

If you haven't already done so, you should set the 'reverse_proxy' setting in settings.php to 'TRUE' for the given site, and list the ip addresses of the reverse prox{y,ies} in an array in the 'reverse_proxy_addresses' setting.

comment:6 Changed 7 years ago by Ross

Resolution: fixed
Status: closedassigned

Oh thanks greg!

comment:7 Changed 7 years ago by Ross

Keywords: drupal added; achebe.mayfirst.org bouazizi.mayfirst.org removed
Resolution: fixed
Status: assignedclosed

Apparently, though I'm not sure how at this point. Running a conditional check against the $_SERVER variable received from varnish can cause a significant number of cache requests to get passed by varnish. Again, I don't fully understand this problem, but the above hack which conditionally checks $_SERVERHTTP_X_FORWARDED_FOR?, created a flood of apache requests to the main server from the proxy servers.

This did not, however, kill all caching as varnishstat continued to show a 30-50% hit rate. Confusing.

However, the suggestion on comment:5 does solve the initial problem for which this ticket was opened.

Please login to add comments to this ticket.

Note: See TracTickets for help on using tickets.