Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Eclipse Projects » Eclipse Platform » Large Text Files Editor(Editor for handling large text files (>1GIG))
Large Text Files Editor [message #508377] Mon, 18 January 2010 10:36 Go to next message
Noam Behar is currently offline Noam Behar
Messages: 12
Registered: July 2009
Junior Member
Hello,
It is known that eclipse default text editor lack of support to read huge files.
It reads the entire file into a string buffer and then creates a document out of it.

My question is if there is a text editor in Eclipse that support paging of file? meaning -
a user opens a huge file, but only the part that he currently sees is loaded into the memory, and he doesn't feel that only part of the file is loaded.

If not, I would like some tips for where to get started in writing such editor my self. I have good knowledge of PDE and wrote a few plugins, but this problem seems to be more complicated.
Moreover, the files I want to be able to open are not in the local workspace, but outside of it, so I am using IFileStore as a resource, FileStoreEditorInput as an IEditorInput and TextFileDocumentProvider as provider

Thanks a lot !

[Updated on: Mon, 18 January 2010 10:39]

Report message to a moderator

Re: Large Text Files Editor [message #509016 is a reply to message #508377] Thu, 21 January 2010 02:45 Go to previous messageGo to next message
Noam Behar is currently offline Noam Behar
Messages: 12
Registered: July 2009
Junior Member
Anyone?

Thanks
Re: Large Text Files Editor [message #509106 is a reply to message #509016] Thu, 21 January 2010 06:56 Go to previous messageGo to next message
Ed Merks is currently offline Ed Merks
Messages: 26000
Registered: July 2009
Senior Member
Your question is a very complex one. There isn't a paging editor in
Eclipse and if you wanted to build one from what's there, you'd need to
feed subsets of the file into it. Of course upon saving, you'd need to
update the right subset of the large target file. It's unlikely anyone
is going to explain the complex details of how to implement that so
better you start writing it yourself and then ask more specific
questions when you hit roadblocks.

bnoam@il.ibm.com wrote:
> Anyone?
>
> Thanks
Re: Large Text Files Editor [message #509294 is a reply to message #509106] Thu, 21 January 2010 16:15 Go to previous messageGo to next message
Noam Behar is currently offline Noam Behar
Messages: 12
Registered: July 2009
Junior Member
Hi,

Thanks for replying. Now I know that there is no such editor.

Lets assume I want it only for reading purposes (for example big system log files).

I started implementing, but as long as I keep digging, I realize that probably one of the major design decisions of eclipse, was that a file can be loaded into a String.
The entire API is based on that fact. From the high level of SourceViewer, to the low level of ILineTrucker.

For exmaple, the VerticalNumberRuler, relies on the fact that the file is loaded to a String, and then it calculates the number of lines in the document, and assigns enough space for the digit to fit in the column.
So when scrolling down, I need to load dynamically new content to the document, but then it still "thinks" that it displays rows starting at index 0, and I need to make it think that it displays higher lines.
So I took care of it. But then the VerticalNumberRuler assigned less space for characters, because it just counts number of lines in the document, in the assumption that they start at line 0. and lines 0-10 requires less space for digits than lines 100-110.
How would you handle this case?

Another thing among many- the search mechanism just searches in the current Document that is loaded, and not in the entire file. for making it search in the paged file, this mechanism also needs to be changed.

So I am pretty lost here...Since String is not extensible, it looks like the solution is to write an entire new implementation, in parallel to the current one, for the EditorPart. and that's probably a work for the Eclipse developers.
On the other hand I still try to think if I can just place a few "hacks" in key locations in order to solve it.
Help me !
Re: Large Text Files Editor [message #509410 is a reply to message #509294] Fri, 22 January 2010 06:47 Go to previous messageGo to next message
Ed Merks is currently offline Ed Merks
Messages: 26000
Registered: July 2009
Senior Member
Comments below.

bnoam@il.ibm.com wrote:
> Hi,
>
> Thanks for replying. Now I know that there is no such editor.
>
> Lets assume I want it only for reading purposes (for example big
> system log files).
Perhaps a text widget isn't a good design for that. Some type of lazy
list/tree view might be better.
>
> I started implementing, but as long as I keep digging, I realize that
> probably one of the major design decisions of eclipse, was that a file
> can be loaded into a String. The entire API is based on that fact.
> From the high level of SourceViewer, to the low level of ILineTrucker.
Yes, it pretty much expects to load the whole thing.
> For exmaple, the VerticalNumberRuler, relies on the fact that the file
> is loaded to a String, and then it calculates the number of lines in
> the document, and assigns enough space for the digit to fit in the
> column.
> So when scrolling down, I need to load dynamically new content to the
> document, but then it still "thinks" that it displays rows starting at
> index 0, and I need to make it think that it displays higher lines.
> So I took care of it. But then the VerticalNumberRuler assigned less
> space for characters, because it just counts number of lines in the
> document, in the assumption that they start at line 0. and lines 0-10
> requires less space for digits than lines 100-110.
> How would you handle this case?
I'm doubting you can...
>
> Another thing among many- the search mechanism just searches in the
> current Document that is loaded, and not in the entire file. for
> making it search in the paged file, this mechanism also needs to be
> changed.
Starts to sound like an uphill battle...
>
> So I am pretty lost here...Since String is not extensible, it looks
> like the solution is to write an entire new implementation, in
> parallel to the current one, for the EditorPart. and that's probably a
> work for the Eclipse developers.
> On the other hand I still try to think if I can just place a few
> "hacks" in key locations in order to solve it.
I doubt it.
> Help me !
If there were obvious things to point out I'm sure someone would do
that, but these are hard problems that I doubt anyone else has solved...
Re: Large Text Files Editor [message #509423 is a reply to message #509410] Fri, 22 January 2010 07:32 Go to previous messageGo to next message
Noam Behar is currently offline Noam Behar
Messages: 12
Registered: July 2009
Junior Member
What do you mean about the lazy list/tree view? Isn't the Editor Part supposed to get only a TextViewer?

I'll write in detail what I try to achieve, and please tell me if you think about a better design.

What I currently have now is a search engine plugin that I implemented, that makes specific search queries on the opened file.

The problem is that I can't run this engine on huge files, because I simply can't open them in the editor.

So what I basically need is to open a huge file, in read only mode (but it still needs to feel like it's opened in an editor), allow the user to navigate through the file, and active my search engine on it.

Noam


Re: Large Text Files Editor [message #509595 is a reply to message #509423] Sat, 23 January 2010 08:10 Go to previous messageGo to next message
Ed Merks is currently offline Ed Merks
Messages: 26000
Registered: July 2009
Senior Member
I know there's an ILazyTreeContentProvider and ILazyContentProvider
which are supposed to help with creating views on huge instances but
there's nothing equivalent for huge text views.

bnoam@il.ibm.com wrote:
> What do you mean about the lazy list/tree view? Isn't the Editor Part
> supposed to get only a TextViewer?
>
> I'll write in detail what I try to achieve, and please tell me if you
> think about a better design.
>
> What I currently have now is a search engine plugin that I
> implemented, that makes specific search queries on the opened file.
> The problem is that I can't run this engine on huge files, because I
> simply can't open them in the editor.
>
> So what I basically need is to open a huge file, in read only mode
> (but it still needs to feel like it's opened in an editor), allow the
> user to navigate through the file, and active my search engine on it.
>
> Noam
>
>
>
Re: Large Text Files Editor [message #509615 is a reply to message #509595] Sat, 23 January 2010 11:35 Go to previous message
Noam Behar is currently offline Noam Behar
Messages: 12
Registered: July 2009
Junior Member
ok, thanks anyway. Guess I'm gonna try to invent the wheel from scratch.

If anyone else wants to share his thoughts, please do.

Noam
Previous Topic:automatic Export-Package versioning?
Next Topic:Printer BUG, i need help
Goto Forum:
  


Current Time: Fri Aug 22 13:48:34 EDT 2014

Powered by FUDForum. Page generated in 0.02164 seconds