WPF, cargando imágenes en bucle (BitmapImage)

I have a problem with BitmapImage class. I know there are a few similar questions but I could not find answer for that particular issue.

Basically when I load a few times a large picture it throws silent exception. I tried to create a simple application which presents the issue.

The real scenario is of course different, this is just to reproduce it. Let's say somewhere in button click event we have:

        var openFileDialog=new OpenFileDialog();

        if (openFileDialog.ShowDialog()==true)
        {
            BitmapImage bitmapImage = null;

            for (int i = 0; i < 50; i++)
            {
                if (bitmapImage != null)
                {
                    bitmapImage.StreamSource.Dispose();
                    bitmapImage.DecodeFailed -= bitmap_DecodeFailed;
                    canvas.Bitmap = null;


                }

                bitmapImage = Load(openFileDialog.FileName);
                Thread.Sleep(100);                                    
                Debug.WriteLine(GC.GetTotalMemory(false));
            }

            canvas.Bitmap = bitmapImage;
            canvas.InvalidateVisual();
        }

Here in the loop we load 50 times the same pictures (one by one). It simulates the scenario where user can change a picture. It should replace previous one, and reload next one. I don't understand why it fails after 4 iterations. If there was enough memory to load it first time and second time, why there is not enough to reload it later?

This is how I load images:

    public BitmapImage Load(string path)
    {
        var memoryStream = new MemoryStream();

        using (var reader = new BinaryReader(File.Open(path, FileMode.Open)))
        {
            byte[] allData = reader.ReadBytes((int) reader.BaseStream.Length);
            memoryStream.Write(allData, 0, allData.Length);
        }

        memoryStream.Position = 0;


        var bitmap = new BitmapImage();
        bitmap.DecodeFailed += bitmap_DecodeFailed;

        bitmap.BeginInit();
        bitmap.StreamSource = memoryStream;
        bitmap.CacheOption = BitmapCacheOption.OnLoad;
        bitmap.EndInit();

        return bitmap;

    }

    void bitmap_DecodeFailed(object sender, ExceptionEventArgs e)
    {
        MessageBox.Show(e.ErrorException.ToString());
        Environment.FailFast(e.ErrorException.ToString());
    }

It fails (DecodedFailed) after a few iterations (it doest crashes, it just raise the event). The exception is:

System.OutOfMemoryException: Insufficient memory to continue the execution of the program. at System.Windows.Media.Imaging.BitmapSource.CreateCachedBitmap(BitmapFrame frame, BitmapSourceSafeMILHandle wicSource, BitmapCreateOptions createOptions, BitmapCacheOption cacheOption, BitmapPalette palette) at System.Windows.Media.Imaging.CachedBitmap.FinalizeCreation() at System.Windows.Media.Imaging.CachedBitmap.EndInit() at System.Windows.Media.Imaging.CachedBitmap..ctor(BitmapSource source, BitmapCreateOptions createOptions, BitmapCacheOption cacheOption) at System.Windows.Media.Imaging.BitmapImage.FinalizeCreation()

I tried different things: - changing caching mode - disposing stream. - freeze - it happens also if I don't attach any events (I know there is well-known memory leak for BitmapImage's events)

Please note that there are no any bindings. I draw a picture on canvas and that's it. I tested the code using http://followhislight.files.wordpress.com/2013/08/map_of_the_world_1998.jpg.

It's not always after 4 iterations. It all depends on available memory and picture size.