When the dreaded “Field is too large (32K)” Lotus Notes error strikes…

There are some problems in life—and in Lotus Notes—that defy easy solutions. And one such problem is the “Field is too large (32K)” error, which affects documents whenever the size of the Summary data for a field grows beyond the limit set in Notes.

*The Lotus Notes Summary data limit is 32K for a single field or 64k for an entire document.

Let’s get one thing out of the way: when you see the 32K error, the affected document is in deep trouble. The document may be listed in views but you cannot open it by double-clicking nor can you access its data through code. And in our personal experience, Compact and Fixup tasks weren’t able to resolve the issue either (that said, we’ve read about other people reporting success using compact commands) and neither were stamp commands. If you create a new replica, the errors may go away but the documents affected with the 32K error won’t be copied, so that might be considered a heavy-handed approach in some instances.

While the data contained inside these documents are now difficult to retrieve, you’ll have to contend with the fact that the broken docs are still hanging around in your database. They still take up disk space and they can still cause you major headaches by crashing agents and breaking any code that attempts to open them. And then there’s the small matter of the frustration that end-users feel when they attempt to open a doc and get a 32K error. And they get to feel that frustration all over again when you tell them that the affected documents likely can’t be fixed.

One bit of good news when you’re faced with 32k error documents: they’re still readable in views* as long as the field that exceeded the 32K limit is not being displayed in any of the columns.

This means that you can create a new view with columns to list any summarized items that you’d like to look at. From there you can export the data from the view. Going through this process will also help you discover which item exceeded the 32K limit. (In our case, we had to do some trial-and-error, but we finally figured out that including one particular Text List field would cause the document to appear blank in the view–thus it was the culprit). Of course, you’re still out of luck when it comes to non-summarized fields (like Rich Text) which can’t be displayed in views but at least you can recover something.

*At least they’re readable in views in our testing—your mileage may vary. Please note that we were only able to test documents where a single field exceeded the 32K limit–so the behavior of documents where aggregate summary data exceed 64k may or may not be different (please comment if you have any information regarding this).

This Summary information at the heart of the32K error problem is used for showing field values in views, folders or search results. Summary information is created whenever a field has the Summary flag enabled. By default, whenever Text, Text List, Number, Number List, Date, Date List, Names, Authors or Readers field is created in a document by a Form, or by LotusScript or Java, the new item will have the Summary flag enabled. (Note that Rich Text fields should never have the Summary flag enabled).

If you don’t need to display a given field’s content in a view or in search results, you may want to disable the Summary flag either through LotusScript (see your Lotus Notes Help for more on this) or by using Ytria’s scanEZ tool (which will let you toggle field flags for one or many documents). This can help you avoid the Field is too large (32K) error from occurring in the future because Summary data will no longer be generated by the fields in question.

UPDATE: Please check out the comments at the bottom of this article for an alternate (and likely faster) way to find docs with the 32k error -PW

Another frustrating aspect of this “Field is too large (32K)” error issue, is that it can be quite tricky to track down all the documents in the database that have the problem. You could go and try double-clicking every document in every view to find out which ones trigger the 32K error. But if you’re in any sort of a hurry, Ytria’s scanEZ tool offers a much faster way to find these docs through its Search by Note ID Range feature.

Search by Note ID Range lets you enter a numeric range of Note IDs in either hexadecimal or decimal units, then attempt to retrieve all the notes that match IDs inside the range. The larger the range that you choose, the longer the scanning process will take—the largest search you can perform (h1 to hFFFFFFFF) will attempt to search through 4,294,967,295 Note IDs. That might take a while. (The last time we tried a full-range scan, it took us a full weekend, and that was on a local NSF file). But we have some good news for scanEZ users: We’re currently developing a much faster alternative method for Note ID range searches—stay tuned!

Since we usually don’t have time for a full scan, we’ve found that the following method usually does this trick:

Just create a new document in the database and copy its NoteID to the clipboard. You can then enter a Note ID range search with a lower limit of one (h1) and an upper limit value of the NoteID that you just copied copied to the clipboard. Since Note IDs are created sequentially, this new Note ID should theoretically represent the upper limit for your database.

Once the Search by Note ID Range is complete, any 32K error document should be categorized under ‘Errors’ and marked with a ‘bomb’ icon in your search results collection (search results in scanEZ are placed in virtual folders labeled ‘My Selection’). By clicking on the individual documents with 32K errors you’re still not going to see their data, but you do at least get their NoteID, UNID and date information (and you may copy this information to the clipboard in scanEZ).

Once you’ve tracked all the documents with 32K errors, you may want to delete them. To do this in scanEZ, you just need to right-click the document in question and choose Delete. If you want to get rid of them in one fell swoop just highlight the Error category then right-click> Delete all documents in category.

If the documents in question are only afflicted with this 32K error on one database replica but are still OK on others, you may want to use the ‘Do not create deletion stub‘ option when deleting the docs in scanEZ. By doing this, the next time you replicate, these documents will be regenerated from replicas. (Just to clarify, if a deletion stub is created, the deleted 32K error document will be removed from all the other replicas upon replication).