Preheat the oven to 350 degrees F (175 degrees C). Grease two 9-inch round cake pans.

Combine flour, salt, and baking soda together in a medium bowl.

Beat brown sugar, oil, egg, and vanilla with an electric mixer in a large bowl until smooth. Add flour mixture and milk, alternately, until combined. Stir in rhubarb and 1/2 cup almonds until incorporated. Pour into the prepared pans.

Mix white sugar and melted butter together with a fork in a small bowl; stir in 1/4 cup almonds, then sprinkle over batter.

Bake in the preheated oven until a toothpick inserted into the center comes out clean, 30 to 35 minutes.