fix(dedup): accept single-element input_keys in deduplicate filters

[raams95]

Summary

The deduplicate filters accept either input_key or input_keys. When input_keys is a list with a single column (for example input_keys=["text"]), run() takes the single-column branch, but that branch reads sample[self.input_key]. self.input_key is None in that case, so the operator crashes with KeyError: None on the first row.

storage = ...  # dataframe with a "text" column
HashDeduplicateFilter().run(storage, input_keys=["text"])
# KeyError: None

The same line exists in six operators:

• HashDeduplicateFilter
• MinHashDeduplicateFilter
• NgramHashDeduplicateFilter
• SimHashDeduplicateFilter
• SemDeduplicateFilter
• CCNetDeduplicateFilter

Fix

When input_keys is given, the single-column branch now reads input_keys[0]. This makes input_keys=["text"] behave the same as input_key="text". Passing input_key or a list of several keys works exactly as it did before.

Tests

I added test/cpu_only/test_dedup_single_input_keys.py. For each CPU-only dedup filter, it checks that input_keys=["text"] gives the same filtered dataframe as input_key="text".

• Before the fix: 5 failed (KeyError: None)
• After the fix: 5 passed. The existing test/cpu_only/test_minhash_short_texts.py also still passes.

pytest test/cpu_only/test_dedup_single_input_keys.py test/cpu_only/test_minhash_short_texts.py -q
13 passed

SemDeduplicateFilter gets the same one-line change. The new test doesn't cover it because it needs torch/transformers and a BERT model.

Fixes #552

[SunnyHaze]

LGTM, thanks!