I want my players to be able to kill a Mimic and get the loot from the Mimic's corpse.

The Mimic has an acidic bite, and is a creature, not actually a container.

Can a Mimic store items?

Probably not

Because the mimic has the always on property of Adhesive when in object form:

The mimic adheres to anything that touches it.

It seems unlikely that someone was able to get something inside of it. Once it moves, it's no longer the object and is instead it's true form. And I'd saying that it opening is it moving (because it's moving.)

However, should whomever is using it want to store something inside it (cruel!), I'm not sure that would be totally against the rules.

Storing items inside creatures

I guess technically it can 'contain' the item - but the players are going to have to 'dig' for it.

The mimic is only an object when it's using it's shapechanger ability (my emphasis):

The mimic can use its action to polymorph into an object or back into its true, amorphous form. Its statistics are the same in each form. Any equipment it is wearing or carrying isn't transformed. It reverts to its true form if it dies.

If they've killed it, then it's no longer acting as the 'container', but I don't see why it can't still have something inside it (heck, any monster could have items implanted, I guess.)

It's fairly common in published adventures for e.g. a roper to have loot in its gizzard. It occurs multiple times in Tales from the Yawning Portal.

I think the monster's manual even suggests treasure winds up there.

There's no reason your mimic couldn't have something similar, though I suppose it's not really explicitly spelled out as a common mimic characteristic.

As a container, no.

When the mimic dies, it reverts into it's true shape, which is not a container which would normally contain loot. It could have eaten something like an enchanted ring by mistake or something while feasting on the poor adventurer before you, but otherwise, I think it would be on the outside of the Mimic due to its Adhesive property. Now, other monsters do have the ability to have loot inside them, as @kyle sexton has said, like the Roper, which is actually stated in the MM

A roper can digest anything it eats with the
exception of platinum, gemstones, and magic
items, which can sometimes be retrieved from the
creature's gizzard after death. A roper's digestive juices
are also valuable, fetching a high price from alchemists
who use them as a solvent.

Then you have the Purple Worm which eats through the earth, and can have gems and the like inside its stomach, which is also referenced in the MM.

When a purple worm burrows
through the ground, it consumes earth and rock, which
it breaks down and constantly excretes. Precious metals
and gems can thus be found within the bodies of purple
worms, which are targeted by particularly brave and
foolhardy treasure hunters.

So... For the mimic, I would think not, but as DM, you could rule it so. Other monsters consume things that they cannot digest and you can loot them for it, so why not the mimic?