Place catfish pieces into a mixing bowl and sprinkle with thyme leaves; toss to coat. Mix in mustard with your hands. Cover and refrigerate for 8 hours to overnight.

Heat oil in a deep-fryer or large saucepan to 350 degrees F (175 degrees C).

Whisk together flour, salt, black pepper, and baking soda in a large bowl. Dip catfish pieces, one at a time, into flour mixture and shake off excess.

Working in batches, fry catfish nuggets in hot oil until no longer translucent in the center and outside is golden brown, 3 to 4 minutes. Drain on a paper towel-lined plate. Let oil come back up to temperature between batches.