Heat olive oil in a skillet over medium heat. Stir in jalapenos; cook until tender, about 5 minutes. Mix in garlic, and adobo sauce; bring to a simmer. Stir the raspberries into the sauce; cook until soft, about 3 minutes. Stir in the vinegar, salt, brown sugar, and white sugar. Mix well. Simmer until thickened and reduced by half, about 15 minutes. Transfer sauce to a heatproof bowl; allow to cool to room temperature before serving, about 20 minutes.