Opened 11 years ago

Closed 11 years ago

#1021 closed Bug/Something is broken (fixed)

Need to limit length of display of aggregated items on mayfirst.org

Reported by: alfredo Owned by: alfredo
Priority: High Component: Tech
Keywords: www.mayfirst.org rss drupal Cc:
Sensitive: no

Description

Pulling my hair out trying to divine what I'm doing wrong in my code for blogs on MF site on fidel. This would be our front page on the live site but it's not yet live -- it's on fidel. :-) You'll see in a moment why that makes no difference.

The feed from Jamie's site shows, not the "description" (as the teaser is called) but the whole blog. After Jamie and I struggled with this and I struggled alone tonight...lo and behold!

http://current.workingdirectory.net/index.rss

Look at "Goals for a New Global Movement"

Entire blog is rendered as a teaser. This is *not* correct behavior, as least as far as Drupal's concerned. Let's take the rss page from our own blog site

http://blogs.mayfirst.org/blog/feed

Everything neatly packaged as a teaser! :-)

McClelland!!! Can you shed some light??? DKG??? have you thinking?

Because, my brothers, I know *nothing* about this rss stuff so I'm stumped.

Since we're almost ready to launch this very important site, I'll make this a high.

Change History (8)

comment:1 Changed 11 years ago by Daniel Kahn Gillmor

According to #680, the blogs site is already live. And the main http://mayfirst.org/ was launched several weeks ago, and is already presenting links and teasers to blogs on the home page. Which site did you mean is almost ready to launch?

I'm not sure if i understand what the problem behind this ticket is. Is the problem with http://blogs.mayfirst.org/, with http://mayfirst.org/, or with one of the external sites whose feeds we want to aggregate?

Looking at the source of each of the feeds mentioned in the description, the feed from Jamie's blog includes the entire post in the <description> element within each <item> element, while the feed from blogs.mayfirst.org only includes a short "teaser" snippet. FWIW, the feed from my own blog on d-a.org also includes the entire post in each <description> sub-element.

I don't think you can "fix" this by making Jamie's blog (and my blog) post "teaser" snippets instead of full posts, because that doesn't solve the general case. If MF/PL is interested in aggregating other people's blogs, we're cannot force everyone to emit a certain length <description> in their feed -- even if we did force them to do so once (during signup, say), the next day they could change the settings on their external blog, and go back to emitting full-length <description>s.

What we could do in this circumstance (i'm assuming you want to enforce a length limit on the teasers presented on mayfirst.org) is to accept the external feed, but truncate each item (and then properly close any open tags in the truncated version), and present the truncated version(s) on our web site.

If that's what you want to do, you should probably change the summary for this ticket to something like "Need to limit length of display of aggregated items on mayfirst.org", and add a www.mayfirst.org keyword.

But it's quite likely that i'm misunderstanding what you're trying to do: could you clarify the problem a bit?

comment:2 Changed 11 years ago by alfredo

No it's live. It was made live today actually. But we haven't made *the* announcement yet so people probably won't go there much quite yet. But we have to deal with that other issue before making the announcement, I think. I mean, deal with all three issues, no?

comment:3 Changed 11 years ago by alfredo

Keywords: www.mayfirst.org added
Summary: RSS from workingdirectory site feeds entire blog as "teaser"Need to limit length of display of aggregated items on mayfirst.org

On the other, more important issue, Daniel. I think you get it completely and I was afraid that what you correctly report was the case. Essentially, Drupal acts like it's being fed by other Drupal sites, I think and the Drupal sites tend to handle rss with teasers rather than full entries.

We are going to have to code into our display script (which by the way is used on the main mayfirst site as the home page in the box under the middle -- where the blogs are teased) the kind of truncating you described.

And, accordingly, I've made the changes you suggested.

Now I'll work tomorrow on make the code. :-)

Thank you, my brother!

comment:4 Changed 11 years ago by Daniel Kahn Gillmor

Owner: changed from Jamie McClelland to alfredo

How are you working on this, Alfredo? Are you committing your changes to SVN? How are you doing the aggregation? If you're using a stock drupal module, i'd be surprised if there wasn't a setting to limit overall post length. Otherwise, you'd open your site up to some really funky denial of service attacks.

comment:5 Changed 11 years ago by Jamie McClelland

I'm pretty amazed by the lack of teaser support in the aggregator module. I found a page describing how to hack the aggregator module to provide a teaser. Since the function being hacked starts with theme_ (theme_aggregator_page_item) it means it can be added to the theme template.php file without having to change the core aggregator module (just rename it to mytheme_aggregator_page_item).

Furthermore, the code change they suggest to create the teaser is:

$output .=  substr(aggregator_filter_xss($item->description), 0, 300) . '...';

However, given the node_teaser function, I think it would be better to let Drupal handle it with simply:

$output .= node_teaser(aggregator_filter_xss($item->description));

How are you working on this, Alfredo?

This code is being stored in the database not in a module :(.

comment:6 Changed 11 years ago by alfredo

Right now, I'm in the page itself:

blogsinner

because this was originally only a page call and that goes into the page by custom. Now it's ending up being a code snippet or even a mini-module. Pages are stored in the database and are available to anybody with admin privileges on the site itself.

However, I'm working on this page on fidel so you ain't seeing nothing yet. :-)

I think we'll make some progress on this today.

comment:7 in reply to:  6 Changed 11 years ago by Daniel Kahn Gillmor

Keywords: rss drupal added

Jamie, i suspect you're right about using node_teaser inside theme_aggregator_page_item. That looks the right tack to take. The only concern is an attack from someone who includes the "magic break tag" in their summaries, but puts it way way down at the end -- maybe that's a corner case we don't worry about until someone abuses it?

Alfredo: if you could put the code in question into svn, you could get more feedback from other people about what you're trying to do. Storing code in the database on a development machine makes collaboration more difficult.

comment:8 Changed 11 years ago by Jamie McClelland

Resolution: fixed
Status: newclosed

Ok - I think this is fixed - but please feel free to re-open if anyone disagrees with my approach.

I'm getting a better sense of why aggregator doesn't do teaser and why generating teasers on untrusted content (or displaying untrusted content at all) is iffy.

Last week Alfredo got it working kinda. The problem is that there was a blog in which a closing ul tag was cut off by the teaser function. Without the closing ul tag, the rest of the page was indented.

I ended up taking a heavy handed approach: stripping all html tags. It's so easy to write a blog with a tag that won't get closed (on purpose or not) when the description teaser is created. Italics, bold, h1 - there's not telling what could be in someone's blog that could make our home page look wonked out.

The strategy I took was two fold:

  • Override the function that handles the display of each aggregator feed item by creating a new function in our template.php file with the name: mfplnew_aggregator_page_item (mfplnew is the name of our theme). The function is an exact duplicate of the theme_aggregator_page_item function found in the aggregator module with one change. The line from the original that reads:
$output .= '<div class="feed-item-body">'. aggregator_filter_xss($item->description) ."</div>\n";

has been changed to:

$output .= '<div class="feed-item-body">'. strip_tags(aggregator_filter_xss(node_teaser($item->description))) ."</div>\n";
  • Modify the code in the inner_intro field of node 18 (the blogs node). As Alfredo states above - the use of php in this field is kinda a corner case. I'm not sure what the best strategy would be for moving it into a module. In any event, it now has the following php code:
<?php
// copied from aggregator_page_category() function which uses
// arg() function to determine cid, rather than taking it as a 
// function argument, which would allow us to re-use the function
// blah. 

$cid = 1;
$category = db_fetch_object(db_query('SELECT cid, title FROM {aggregator_category} WHERE cid = %d', $cid));

drupal_add_feed(url('aggregator/rss/'. $cid), variable_get('site_name', 'Drupal') . ' ' . t('aggregator - @title', array('@title' => $category->title)));

echo _aggregator_page_list('SELECT i.*, f.title AS ftitle, f.link AS flink FROM {aggregator_category_item} c LEFT JOIN {aggregator_item} i ON c.iid = i.iid LEFT JOIN {aggregator_feed} f ON i.fid = f.fid WHERE cid =' . $cid . ' ORDER BY timestamp DESC, iid DESC', 'view');
?>

Please login to add comments to this ticket.

Note: See TracTickets for help on using tickets.