Concurrent Text Extraction

Document-level concurrency is the straightforward case: each goroutine opens its own file, builds its own reader and never shares anything with the others. It applies to every operation in the library, not just extraction, and it is what you want when the work is a batch of files rather than one large document.

Doing it

var wg sync.WaitGroup
for _, path := range paths {
    wg.Add(1)
    go func(path string) {
        defer wg.Done()

        reader, f, err := model.NewPdfReaderFromFile(path, nil)
        if err != nil {
            return
        }
        defer f.Close()

        // Extract from reader, then send the result on a channel.
    }(path)
}
wg.Wait()

model.NewPdfReaderFromFile returns the opened *os.File as its second value and hands ownership to you; close it once you are done with the reader. The example discards it, which is fine for a short-lived program but not for a long-running service.

The example uses a buffered channel sized to the number of documents rather than a WaitGroup, which lets each goroutine hand off its result and exit without waiting for a receiver.

Limitations

Nothing is shared here, so the constraints are the ordinary ones. Each reader loads a full document structure into memory, which puts a ceiling on how many documents you can have in flight at once; on large files, cap the number of goroutines rather than launching one per input. The example launches one per document because it is meant to be run over a handful of files.

Results come back in completion order, not input order. The example works around that by keying each channel value with the source path.

Run the example

extractSingleDoc opens one file and concatenates the text of its pages; concurrentExtraction launches one goroutine per input document and writes each result to a channel. Every input gets a .txt file of the same base name in the output directory, and the total wall-clock time is printed at the end.

git clone https://github.com/unidoc/unipdf-examples.git
cd unipdf-examples/concurrent-processing
go run concurrent_extraction.go input1.pdf input2.pdf output_dir

If this is your first time using UniPDF, follow the getting started guide to create an API key and set up your development environment.

To parallelize a single large document instead, see page-level extraction.