Opened 3 months ago

Last modified 26 hours ago

#13671 assigned Bug/Something is broken

Nextcloud/Collabora limitation

Reported by: https://id.mayfirst.org/endisolation Owned by: https://id.mayfirst.org/jamie
Priority: Medium Component: Tech
Keywords: Cc:
Sensitive: no

Description

Our team has been using NextCloud more and more lately, and most users have preferred using the in-browser Collabora module for document editing. Today one person reported seeing the following message, and I was able to replicate it by opening up a handful of documents in different browser tabs.

This development build is limited to 
10 documents, and 20 connections - to avoid 
the impression that it is suitable for 
deployment in large enterprises. To find out 
more about deploying and scaling 
Collabora Online Development Edition 
(CODE) check out: 
https://www.collaboraoffice.com/code/#code-scalability

If individual users need to limit the number of tabs/windows they have open, that is probably workable. However, the one reporting the issue says it was showing the error even though it was the only nextcloud tab she had open. I tested it out and saw this message with only the 5th open document.

I'm concerned that the "share via link" feature may actually contribute to the limit. For example, we share a document and send the link to 50 people, will it become inaccessible once 20 of those people have it open in their browser?

Hopefully there's a simple fix... it's quite a job trying to convince the team to transition away from google docs!

Change History (22)

comment:1 Changed 3 months ago by https://id.mayfirst.org/jaimev

  • Owner set to https://id.mayfirst.org/jamie
  • Status changed from new to assigned

We've recently discovered that there is a limit to the number of people who can access pur current version of collabora siultaneously. Copying jamie here for more input.

comment:2 follow-up: Changed 3 months ago by https://id.mayfirst.org/jamie

Yes, we just learned of this limitation a week ago (see 13670#comment:3) and are pretty upset about it. We are still mulling over options.

I took this opportunity to continue that research. Our two main options are:

  • Try to run the collabora office suite without the limit. Another user has already built an image to do just that. This seems like the obvious choice but... I'm worried about maintainability. I'm not convinced the user who built this image will continue updating it or that the fix they made will continue working. I think this route could pose some maintenance problems going forward.

Fortunately, it appears you can have both onlyoffice and collabora available at the same time, so we may want to install onlyoffice and do a trial run.

comment:3 Changed 3 months ago by https://id.mayfirst.org/endisolation

Nice thank you for the followup!

Curiosity drove me to take a look at the source code to see how easy it would be to make the tweak, and in fact it looks like it's set at the configuration phase, with parameters --with-max-connections and --with-max-documents.

So, it seems that changing those limits would require compiling from source, but would NOT require any code change.

Now, I suppose needing to compile from source wouldn't be the *ideal* scenario for maintenance. However, once the scripts are in place to create the deployable package from source (docker image?), it seems it'd be relatively painless to stay up to date as new releases are tagged in their git repository.

Do you think that could be a workable scenario?

If so, i'd be happy to put a little time in poking around figuring out this process and creating those scripts... (just so happens one side-project i've been working on is finally coming to a conclusion this week... this could swoop in to replace it)

comment:4 Changed 3 months ago by https://id.mayfirst.org/jamie

That would be fantastic!

We already maintain a small variation in the collabora docker code (see https://support.mayfirst.org/wiki/nextcloud-admin#Collaboraonline). You can checkout our code with git clone git://git.mayfirst.org/mfpl/collabora-code.

Let us know what you come up with.

comment:5 Changed 4 weeks ago by https://id.mayfirst.org/jamie

Just a bump - endisolation - have you worked out a way to compile collabora without this limit? We are getting more and more members hitting this limit.

comment:6 Changed 3 weeks ago by https://id.mayfirst.org/endisolation

Oh dear, I can't believe that was 3 months ago. I'm sorry to have been silent this long. Unfortunately i haven't been able to put a ton of time into figuring it out; here's what i've got so far:

1) The nextcloud-collabora integration is its own component, who's source is here: https://github.com/nextcloud/richdocuments however, this merely connects nextcloud to a running instance of collabora.

2) The collabora source code is actually what we need to recompile, which is located here:

https://cgit.freedesktop.org/libreoffice/online/

The first steps to compilation are

./autogen.sh
./configure

As far as i've gotten is chasing down several dependencies identified (cryptically) in the configure phase.

Some prereqs thus far: (debian apt-get) libcap-dev libcppunit-dev libreofficekit-dev

The problem I've hit that I'm currently stuck on:

configure: error: header Poco/Net/WebSocket.h not found, perhaps you want to use --with-poco-includes

i've installed libpoco-dev, which has all sorts of Poco/Net headers, but alas, no WebSocket.h. Maybe it's an older version in the debian repo, and i'll need to compile a newer one... (my desktop is still on jessie)

Anyway, once we finally get a clean configure and compile, the next step is figuring out how to supply the --with-max-connections --with-max-documents (referenced in the configure script)

I'll wrestle with this a little more tonight...

comment:7 Changed 3 weeks ago by https://id.mayfirst.org/jamie

Great! Thanks for the report back. And yes, it looks like WebSocket.h is in the stretch version of libpoco-net.

Also, the official Collobara docker image uses ubuntu 16.04 - so compiling on an ubuntu 16.04 docker image might be another avenue (although running this on a debian stretch docker image would be much easier to maintain).

comment:8 Changed 3 weeks ago by https://id.mayfirst.org/endisolation

Genius idea putting the build environment into a docker container... i'm starting to catch up... i think...

https://github.com/ChiweenieDijon/collabora-build-environment

I ended up having to get the poco libs from source, since it's dependent on the JSON lib, which has idiotic licensing so debian excludes it from their repos... (it takes forever to compile)

Now, i've got configure succeeding, with the max-documents and max-connections changes.

However, the make is running into errors hitting the lokit stuff... i think version problems w/ libreoffice libraries. Probably will need to pull in the latest libreoffice kit from source as done with the poco libs.

comment:9 Changed 3 weeks ago by https://id.mayfirst.org/jamie

Excellent - progress!!

comment:10 Changed 3 weeks ago by https://id.mayfirst.org/endisolation

Successful compilation achieved!

The docker container in above repo *should* run to completion, although I haven't fully run the build. It may still require some tweaks, as it was an iterative process to go between running make and pulling in dependencies over and over again, and updating the script at each step.

This sucker builds from scratch the FULL poco library, as well as the full libre office core app. (TBD if its strictly required or if only a subset is needed - e.g. the libreoffice "kit") As such, it is a many-hour process (at least on my core2 duo desktop). I'll fire it up before going to bed tonight and see what happens.

Next step, i'd imagine, is pulling the necessary binaries out of this "build" container, and using those to build the proper collabora "deploy" container. Or is that even necessary you think? Maybe we should just morph this into the container that gets deployed? I'm new to this whole docker thing.

comment:11 Changed 8 days ago by https://id.mayfirst.org/jamie

OMG - sorry, this slipped by while I was traveling. Thanks so much!!

Did the build work out?

And, does the build process have any instructions on how to create debian packages?

I think we need to figure out a way to combine your build process above with the collabora docker image that we currently use. Specifically, the collobara docker image installs loolwsd code-brand collaboraoffice5.3-dict* collaboraofficebasis5.3 from the collabora ubuntu repository and then launches collabora. If we could figure out how to build those debian packages, I think we would be set - we would simply include the deb files in our image and dpkg -i them instead of adding the collabora repo.

comment:12 Changed 8 days ago by https://id.mayfirst.org/jamie

comment:13 Changed 8 days ago by https://id.mayfirst.org/jamie

However... I think building a proper and full debian package will probably work more reliably then trying to pick and choose some binaries to replace.

comment:14 in reply to: ↑ 2 Changed 8 days ago by https://id.mayfirst.org/jaimev

I just noticed a recent comment on this site https://apps.nextcloud.com/apps/onlyoffice suggests at the missing feature of sharing via link with ONLYOFFICE has been solved.

comment:15 Changed 8 days ago by https://id.mayfirst.org/endisolation

Yes, the build did work out! This docker project now successfully finishes the collabora build.

But yeah, i think it does make sense to simply add another script to the "builder docker" that implements Boniface's instructions you linked to and packages up a nice deb file.

Heck, even if we choose to jump ship from collabora in favor of onlyoffice, it's still probably worth the trouble to make it happen... it seems it would be beneficial to the free software movement to offer an automated way to remove an anti-feature from an open-source product.

comment:16 Changed 8 days ago by https://id.mayfirst.org/jamie

Oh wow - the missing link to collaboratively edit documents is really a big one, if that is fixed it does make onlyoffice a much more viable alternative, especially since it is advertised as a less resource intensive on the server.

After some digging, I seem to have found the issue which indeed seems to have been quietly closed and fixed. Also, it seems the problem was only affecting users that are not logged in, which is less serious that I originally thought.

A also agree about the value of the collabora work - even if we go with open office! endisolation - if you are able to complete that last step to build the debs that would be great. Then we can try both options and see which works out.

To be honest, I'm now more inclined to go with onlyoffice, but it's good for the world to have both options.

I think the next step with onlyoffice is to test locally and assuming it seems to work ok, either replace collabora or have a short period with both of them operating side by side.

I'm curious with onlyoffice to find out how well it converts odt documents.

comment:17 Changed 5 days ago by https://id.mayfirst.org/jamie

I'm installed onlyoffice on a dedicated guest and also scheduled a time to test it (I plan to also upgrade nextcloud to the next minor point release and pull in an update to the collabora nextcloud plugin).

Last edited 5 days ago by https://id.mayfirst.org/jamie (previous) (diff)

comment:18 Changed 29 hours ago by https://id.mayfirst.org/jamie

Onlyoffice is now available via our installation for testing.

By default, if you click on a document, it will still open in Collabora. However, the "..." menu now offers an option to open the document via Only Office. Can everyone on this ticket give it a shot?

It does introduce a small bit of confusion since it is installed side-by-side with collabora. The + button to create a new document now lists Document, Spreadsheet, and Presentation twice - one will create it via only office, the other via collabora.

I suggest that we give this a day or two of internal testing, then send an email to our membership letting them know that we plan to test it for the next few weeks.

If the tests are successful, I am leaning toward switching to Onlyoffice because it seems more responsive, it advertises itself as a lighter weight alternative to collabora, and it can be maintained in a normal virtual guest via apt rather than as a docker container.

I'm bothered by one import limitation: it always saves in microsoft foramt. You can still open and edit in libre office on your desktop, but I prefer to support the free open document format rather than the proprietary microsoft format.

comment:19 Changed 27 hours ago by https://id.mayfirst.org/jamie

Sadly, it looks like onlyoffice is not interested in changing their proprietary format defaults:

comment:20 Changed 27 hours ago by https://id.mayfirst.org/jaimev

Great work jamie. I opened a document and played around a bit, I don't really notice a difference but I am not a document power user so it hard for me to think of how to put it through the paces. I also found the onlyoffice interface a little overwhelming and invasive for my tiny screen but some members might appreciate the extra functionality.

It is confusing to see the options doubled up in the menu and I suspect that might cause some trouble for members. I don't imagine there's any easy way to edit how those menu options appear to clearly label some as "Only Office Document" etc.

The proprietary docx format goes against my politics as well although my radical vision for the future is the renouncement of all binary document formats in favor of simple markup formats like markdown so it is hard for me to get excited about this one. I suspect many of our members already have to convert documents between docx and otf on a regular basis as it is. Is this the hill we are prepared to die on?

If OnlyOffice actually performs 10x better then Collabora then it would be hard to argue against it for now but if it is only slightly better then I wonder if it is worth it. Couldn't narrow performance differences be negated on one side or the other after a significant version upgrade?

https://en.wikipedia.org/wiki/OpenDocument

comment:21 Changed 27 hours ago by https://id.mayfirst.org/jamie

This article is written by onlyoffice so is not exactly unbiased, but it does describe in detail how onlyoffice and collabora are architecturally different in critical ways that means onlyoffice will always work more efficiently then collabora:

https://medium.com/onlyoffice/onlyoffice-vs-collabora-a-critical-comparison-18a5ba0dee62

It really helps explain why collabora sets the default limits - even if we successfully remove the limits in collabora - how much RAM would we need to allocate to the collabora-running docker image to ensure all of our members who need to edit a document together would be able to?

comment:22 Changed 26 hours ago by https://id.mayfirst.org/jaimev

Ah, those differences explain the extra wait and load on my local machine when initially loading documents through OnlyOffice. My browser is doing more of the up front work. The scenarios they propose of users opening several documents at once and leaving them open are probably spot on. Yeah I can see we would probably eventually need to dedicate a significant amount of computing resources to keep Collabora running beyond the set user limit.

Please login to add comments to this ticket.

Note: See TracTickets for help on using tickets.