VSTO, Memory Leaks & Reference Management

Developing Add-ins for Office calls for some careful handling of references, particularly with references to objects that have an underlying COM reference. 

My previous blog entry Listening to Calendar Events with Outlook 2007 and VSTO talked about listening to Appointment events, but my code didn't handle references very well. This left some references to Outlook objects hanging and Outlook was showing values that were old.

I've since update this with some better handling of these references. In particular, the explicit releasing of COM objects via the Marshal.ReleaseComObject() method.

More Reading:

• Systematically Releasing Objects


• OOM.NET: Part 2 - Outlook Item Leaks

Visual Studio Using Huge Amounts of Memory

Lately I have been working on a solution with 33 projects on Visual Studio 2008.

Yeah, I know. That's a lot.

A lot of them were test projects and it is SCSF-based, so that is okay, but Visual Studio was chewing up loads of memory. I mean serious amounts, towards the 2GB limit. At this point, my hard disk would thrash at the smallest use of the IDE and be unusable for minutes on end.

Someone put me on to a tool from JetBrains, the guys that did ReSharper - it is supposed to change the way memory is allocated. One of the guys on our project who is using this tool is now using a lot less memory, but whilst mine is better than it was, it still aint that great.

So it may help.

Another option is to add /3GB /USERVA=3030 to the boot.ini file. Visual Studio is already large address aware (I think), so it should allow more VM use for the devenv.exe process.

Update:

One of the biggest causes of this was DevExpress Refactor! Pro. I'm normally a big fan of this add-in, but uninstalling it saved me enormous amounts of memory. 

Including Setup Projects in Team Build

Setup (Deployment) projects are not supported by Team Build out of the box. You have to add
additional steps to the TFSBuild.proj file to build the setup project and copy the files to the output directory.

The following MSDN article describes this:

Walkthrough: Configuring Team Build to Build a Visual Studio Setup Project

Microsoft Office Developer Show Episode 4 (Harry Miller)

Its not everyday you get mentioned in an MSDN video blog:

http://blogs.msdn.com/vsto/archive/2008/06/11/microsoft-office-developer-show-episode-4-harry-miller.aspx

PowerCommands for Visual Studio 2008

This extremely handy set of extensions for Visual Studio 2008 provides extra functionality to various parts of the IDE.

PowerCommands for Visual Studio 2008

Personally, I think these are the sort of things that should have been provided out of the box.

ObjectContainerDataSource, FormViews and LLBLGen Entities

I've always run into issues with databinding in ASP.NET 2. It is nice and straightfoward with basic examples, but there always seems to be some scenario that causes an event to fire at an unexpected time, stuffing things up.

I am currently using the Web Client Software Factory (WCSF) by Microsoft patterns & practices and one of the little gems in this is the ObjectContainerDataSource. This datasource implements data binding in a way that easily integrates with the MVP pattern. That is, the Select/Update/Delete events can be passed on and handled by the Presenter. If you are doing web development and are not aware of the WCSF, it is worth checking out.

This all seems fine, until I wanted to bind to an LLBLGen entity (LLBLGen is an easy-to-use and inexpensive O/R mapper). The LLBLGen entity had a Timestamp field on it that was used for optimistic concurrency - it was read-only and the optimistic concurrency checking would ensure that the Timestamp field had not changed on the database since the entity was fetched.

On an Update operation I found the ObjectContainerDataSource was creating a new instance of my entity and updating only the properties that were databound in the FormView on my page. No primary keys, no foreign keys, no timestamp fields; by the time I was trying to handle the Update event in the Presenter, I had an instance that supposedly new, with no relationships. Argh!

I found some related entries in the WCSF Discussions, but no one had a good solution that suited me and my LLBLGen entity.

One of the good things about the WCSF is that you get code, and the ObjectContainerDataSource is provided in its own project. I've now made some modifications to it that works the way I believe it should. Basically, instead of creating a new instance of the type you are binding to and updating the bound properties on that, the current instance (that is cached on the ObjectContainerDataSource after the Selecting event is called) is used and its properties are updated with the new databound values. This gives you the state of the entity as it was when you fetched it with the exception of the changes made through databinding. This is the behaviour I would have expected initially.

Note: to get access to the WCSF source, you'll need to install the source separately after you have installed the WCSF. Default folder is:

C:\Program Files\Microsoft Web Client Factory\Source Code\

So, the new ExecuteUpdate method in ObjectContainerDataSourceView.cs looks like this:

protected override int ExecuteUpdate(IDictionary keys, IDictionary values, IDictionary oldValues)
{
Guard.CollectionNotNullNorEmpty(keys, String.Format(CultureInfo.CurrentCulture,
  Properties.Resources.NoKeysSpecified), "keys");
Guard.ArgumentNotNull(values, "values");

ObjectContainerDataSourceUpdatingEventArgs updatingEventArgs =
    new ObjectContainerDataSourceUpdatingEventArgs(DictionaryHelper.GetReadOnlyDictionary(keys),
    values, oldValues);
OnUpdating(updatingEventArgs);
if (updatingEventArgs.Cancel)
    return 0;
//
// -------------------------------------------------------------
// This section has been modified to keep the old instance and
// update the appropriate fields on it that changed through
// databinding. If no old instance found, a new instance is
// created.
//
int rowsAffected;
object newInstance;
object oldInstance = FindInstance(keys);
if (oldInstance != null)
{
  TypeDescriptionHelper.BuildInstance(values, oldInstance);
  rowsAffected = 1;
  newInstance = oldInstance;
}
else
{
  newInstance = CreateInstance();
  TypeDescriptionHelper.BuildInstance(keys, newInstance);
  TypeDescriptionHelper.BuildInstance(values, newInstance);
  rowsAffected = 0;
}
//
// -------------------------------------------------------------
//
OnDataSourceViewChanged(EventArgs.Empty);

ObjectContainerDataSourceStatusEventArgs updatedEventArgs =
  new ObjectContainerDataSourceStatusEventArgs(newInstance, rowsAffected);
OnUpdated(updatedEventArgs);

return rowsAffected;
}

The Delete operation is affected in a similar manner, so it has also been updated:

protected override int ExecuteDelete(IDictionary keys, IDictionary oldValues)
{
Guard.CollectionNotNullNorEmpty(keys, String.Format(CultureInfo.CurrentCulture,
  Properties.Resources.NoKeysSpecified), "keys");

ObjectContainerDataSourceDeletingEventArgs deletingEventArgs =
    new ObjectContainerDataSourceDeletingEventArgs(DictionaryHelper.GetReadOnlyDictionary(keys),
    oldValues);
OnDeleting(deletingEventArgs);
if (deletingEventArgs.Cancel)
    return 0;

int rowsAffected;
object instance = FindInstance(keys);
if (instance == null)
{
    rowsAffected = 0;
  //
  // -------------------------------------------------------------
  // This section has been modified to only create a new instance
  // if an old one cannot be found. Moved up from below this if()
  // statement.
  //
  instance = CreateInstance();
  //
  // -------------------------------------------------------------
  //
}
else
{
    Data.Remove(instance);
    rowsAffected = 1;
}
TypeDescriptionHelper.BuildInstance(oldValues, instance);
TypeDescriptionHelper.BuildInstance(keys, instance);
OnDataSourceViewChanged(EventArgs.Empty);

ObjectContainerDataSourceStatusEventArgs deletedEventArgs =
  new ObjectContainerDataSourceStatusEventArgs(instance, rowsAffected);
OnDeleted(deletedEventArgs);

return rowsAffected;
}

These changes should work with any class, not just LLBLGen entities. For me, retaining the Timestamp value was the most important (for optimistic concurrency as I mentioned earlier), and there is no way to set that externally. I am going to go out on a limb here and say that this probably applies to most O/R mappers.

I'd be glad to here from anyone who has had similar issues or has found a better solution.

Interactive Application Architecture Patterns: MVC, MVP & PAC

Found a great article on the comparisons between Model-View-Controller (MVC), Model-View-Presenter (MVP), and Presentation-Abstraction-Control (PAC):

Interactive Application Architecture Patterns by Derek Greer

Particularly useful (for me) is the descriptions of the pattern variations used by the Microsoft Patterns & Practices software factories (i.e.: Smart Client Software Factory and Web Client Software Factory).

Outlook 2007: How to determine if you are the owner of an Appointment

There doesn't appear to be an easy way to find out if the current user is the owner of an Appointment. Maybe I've something extremely simple, but I just couldn't find that property called IsOwner.

I basically want to prevent functionality from being available if the user has opened an Appointment that isn't in one of their own calendars (ie: readonly and from someone elses calendar). After looking through the AppointmentItem and MAPIFolder properties, I finally found something that was useful: MAPIFolder.Store. This property returns a Store object except when the Folder is a shared folder (ie: someone elses). Cool.

So, I've now created my IsOwner method:

    public static bool IsOwner(AppointmentItem anAppointment)
{
    bool result = false;

    if (anAppointment.Parent is MAPIFolder)
    {
        //
        // A Store of null indicates a shared folder (ie: not the current user's).
        //
        result = (((MAPIFolder)anAppointment.Parent).Store != null);
    }

    return result;
}

This, of course, makes the assumption that if the AppointmentItem is not in a shared folder, it is in a folder that is owner by the current user. I'll need to confirm this, but for now it allows me to move forward.

Listening to Calendar Events with Outlook 2007 and VSTO

I've recently being trying to listen to events from a user's calendars so that I can track when appointments are added, removed and modified, and ran into a couple of problems.

Problem 1: Events intermittently stop firing

I was listening to events on the MAPIFolder object and the Calendar's Items collection and they would work for a while and at some point would stop firing.

I eventually traced this down to what I believe is a COM reference counting issue. I wasn't storing a reference to these objects, so I guess there was no implicit AddRef by the interop library. As such, there was no need for the runtime to keep the object around.

Hanging on to these references appears to be resolving this issue, so I stand by my assumptions.

Problem 2: How do we know what has been deleted?

The events I was hooking up to were on the ItemsEvents_Event interface off the MAPIFolder.Items property. There are thee events on this interface:

• ItemAdd


• ItemChange


• ItemRemove

The first two have a single parameter which will be the AppointmentItem that is either being added or changed. Great! The last one doesn't have any parameters - all it tells you is that an item has been removed, but doesn't give you any indication what has been deleted. Besides that, its actually too late anyway, the item has already been deleted (you cannot cancel it).

So, I am now hooking up to the BeforeItemMove event on the MAPIFolderEvents_12_Event interface off the MAPIFolder object. This gives me a reference to the AppointmentItem being moved, the folder in which it is being moved to, and also allows me to cancel it.

To detect that it is being deleted, just check the folder it is being moved to is null (hard-delete such as Shift-Del) or the deleted items folder (soft-delete).

Solution

I have created a class that will monitor a user's default calendar and fires three events:

• AppointmentAdded - gives you the AppointmentItem instance added.


• AppointmentModified - gives you the AppointmentItem instance changed.


• AppointmentDeleting - gives you the AppointmentItem that is about to be deleted and allows you to cancel the operation if needed.

Sample usage:

The following code is for use within the ThisAddIn class. It outputs to console when an item is added, modified, or about to be deleted. It also displays a dialog box confirming an item should be deleted.

private void ThisAddIn_Startup(object sender, System.EventArgs e)
{
 CalendarMonitor monitor = new CalendarMonitor(this.Application.ActiveExplorer());
 monitor.AppointmentAdded +=
     new EventHandler<EventArgs<AppointmentItem>>(monitor_AppointmentAdded);
 monitor.AppointmentModified +=
     new EventHandler<EventArgs<AppointmentItem>>(monitor_AppointmentModified);
 monitor.AppointmentDeleting +=
     new EventHandler<CancelEventArgs<AppointmentItem>>(monitor_AppointmentDeleting);
}

private void monitor_AppointmentAdded(object sender, EventArgs<AppointmentItem> e)
{
 Debug.Print("Appointment Added: {0}", e.Value.GlobalAppointmentID);
}

private void monitor_AppointmentModified(object sender, EventArgs<AppointmentItem> e)
{
 Debug.Print("Appointment Modified: {0}", e.Value.GlobalAppointmentID);
}

private void monitor_AppointmentDeleting(object sender, CancelEventArgs<AppointmentItem> e)
{
 Debug.Print("Appointment Deleting: {0}", e.Value.GlobalAppointmentID);
 DialogResult dr = MessageBox.Show("Delete appointment?", "Confirm",
 MessageBoxButtons.YesNo, MessageBoxIcon.Question);
 if (dr == DialogResult.No)
 {
     e.Cancel = true;
 }
}

There are two other classes used: EventArgs and CancelEventArgs. These generics are used by the events to pass strongly typed appointments.

EventArgs.cs:

using System;
using System.Collections.Generic;

namespace OutlookAddIn2
{
 public class EventArgs<T> : EventArgs
 {
     private T _value;

     public EventArgs(T aValue)
     {
         _value = aValue;
     }

     public T Value
     {
         get { return _value; }
         set { _value = value; }
     }
 }
}

CancelEventArgs.cs:

using System;
using System.Collections.Generic;

namespace OutlookAddIn2
{
 public class CancelEventArgs<T> : EventArgs<T>
 {
     private bool _cancel;

     public CancelEventArgs(T aValue)
         : base(aValue)
     {
     }

     public bool Cancel
     {
         get { return _cancel; }
         set { _cancel = value; }
     }
 }
}

CalendarMonitor.cs:

using System;
using System.Collections.Generic;
using Microsoft.Office.Interop.Outlook;
using System.Runtime.InteropServices;
using System.Diagnostics;

namespace OutlookAddIn2
{
  public class CalendarMonitor
  {
    private Explorer _explorer;
    private List<string> _folderPaths;
    private List<MAPIFolder> _calendarFolders;
    private List<Items> _calendarItems;
    private MAPIFolder _deletedItemsFolder;

    public event EventHandler<EventArgs<AppointmentItem>> AppointmentAdded;
    public event EventHandler<EventArgs<AppointmentItem>> AppointmentModified;
    public event EventHandler<CancelEventArgs<AppointmentItem>> AppointmentDeleting;

    public CalendarMonitor(Explorer anExplorer)
    {
      _folderPaths = new List<string>();
      _calendarFolders = new List<MAPIFolder>();
      _calendarItems = new List<Items>();

      _explorer = anExplorer;
      _explorer.BeforeFolderSwitch += 
        new ExplorerEvents_10_BeforeFolderSwitchEventHandler(Explorer_BeforeFolderSwitch);

      NameSpace session = _explorer.Session;
      try
      {
        _deletedItemsFolder = session.GetDefaultFolder(OlDefaultFolders.olFolderDeletedItems);
        HookupDefaultCalendarEvents(session);
      }
      finally
      {
        Marshal.ReleaseComObject(session);
        session = null;
      }
    }

    private void HookupDefaultCalendarEvents(NameSpace aSession)
    {
      MAPIFolder folder = aSession.GetDefaultFolder(OlDefaultFolders.olFolderCalendar);
      if (folder != null)
      {
        try
        {
          HookupCalendarEvents(folder);
        }
        finally
        {
          Marshal.ReleaseComObject(folder);
          folder = null;
        }
      }
    }

    private void Explorer_BeforeFolderSwitch(object aNewFolder, ref bool Cancel)
    {
      MAPIFolder folder = (aNewFolder as MAPIFolder);
      //
      // Hookup events to any other Calendar folder opened.
      //
      if (folder != null)
      {
        try
        {
          if (folder.DefaultItemType == OlItemType.olAppointmentItem)
          {
            HookupCalendarEvents(folder);
          }
        }
        finally
        {
          Marshal.ReleaseComObject(folder);
          folder = null;
        }
      }
    }

    private void HookupCalendarEvents(MAPIFolder aCalendarFolder)
    {
      if (aCalendarFolder.DefaultItemType != OlItemType.olAppointmentItem)
      {
        throw new ArgumentException("The MAPIFolder must use " +
          "AppointmentItems as the default type.");
      }
      //
      // Ignore other user's calendars.
      //
      if ((_folderPaths.Contains(aCalendarFolder.FolderPath) == false) 
        && (IsUsersCalendar(aCalendarFolder)))
      {
        Items items = aCalendarFolder.Items;
        //
        // Store folder path to prevent double ups on our listeners.
        //
        _folderPaths.Add(aCalendarFolder.FolderPath);
        //
        // Store a reference to the folder and to the items collection so that it remains alive for
        // as long as we want. This keeps the ref count up on the underlying COM object and prevents
        // it from being intermittently released (then the events don't get fired).
        //
        _calendarFolders.Add(aCalendarFolder);
        _calendarItems.Add(items);
        //
        // Add listeners for the events we need.
        //
        ((MAPIFolderEvents_12_Event)aCalendarFolder).BeforeItemMove += 
          new MAPIFolderEvents_12_BeforeItemMoveEventHandler(Calendar_BeforeItemMove);
        items.ItemChange += new ItemsEvents_ItemChangeEventHandler(CalendarItems_ItemChange);
        items.ItemAdd += new ItemsEvents_ItemAddEventHandler(CalendarItems_ItemAdd);
      }
    }

    private void CalendarItems_ItemAdd(object anItem)
    {
      AppointmentItem appointment = (anItem as AppointmentItem);
      if (appointment != null)
      {
        try
        {
          if (this.AppointmentAdded != null)
          {
            this.AppointmentAdded(this, new EventArgs<AppointmentItem>(appointment));
          }
        }
        finally
        {
          Marshal.ReleaseComObject(appointment);
          appointment = null;
        }
      }
    }

    private void CalendarItems_ItemChange(object anItem)
    {
      AppointmentItem appointment = (anItem as AppointmentItem);
      if (appointment != null)
      {
        try
        {
          if (this.AppointmentModified != null)
          {
            this.AppointmentModified(this, new EventArgs<AppointmentItem>(appointment));
          }
        }
        finally
        {
          Marshal.ReleaseComObject(appointment);
          appointment = null;
        }
      }
    }

    private void Calendar_BeforeItemMove(object anItem, MAPIFolder aMoveToFolder, ref bool Cancel)
    {
      if ((aMoveToFolder == null) || (IsDeletedItemsFolder(aMoveToFolder)))
      {
        AppointmentItem appointment = (anItem as AppointmentItem);
        if (appointment != null)
        {
          try
          {
            if (this.AppointmentDeleting != null)
            {
              //
              // Listeners to the AppointmentDeleting event can cancel the move operation if moving
              // to the deleted items folder.
              //
              CancelEventArgs<AppointmentItem> args = new CancelEventArgs<AppointmentItem>(appointment);
              this.AppointmentDeleting(this, args);
              Cancel = args.Cancel;
            }
          }
          finally
          {
            Marshal.ReleaseComObject(appointment);
            appointment = null;
          }
        }
      }
    }

    private bool IsUsersCalendar(MAPIFolder aFolder)
    {
      //
      // This is based purely on my observations so far - a better way?
      //
      return (aFolder.Store != null);
    }

    private bool IsDeletedItemsFolder(MAPIFolder aFolder)
    {
      return (aFolder.EntryID == _deletedItemsFolder.EntryID);
    }
  }
}

Maintaining Custom Toolbar Positions in Outlook 2007

NOTE: This applies to Visual Studio 2008 and Outlook 2007.

There are a few articles floating around have examples on saving toolbar position information at the end of a session and restoring that position the next time Outlook is loaded. One at MSDN applies to Visual Studio 2008 and .NET Framework 3.5:

How to: Maintain Position Information for Custom Toolbars between Outlook Sessions

I tried this code with Outlook 2007 and it throws an exception on shutdown when it tries to save the toolbar position information. Reading the article again, I noticed that it says it applies to Outlook 2003.

I just assumed that it would work with Outlook 2007, and why would they release an article for Visual Studio 2008 that works with an old version of Outlook?

The exception that is being thrown when the CommandBar instance is accessed is:

System.Runtime.InteropServices.COMException: Exception from HRESULT: 0x800A01A8

Look up this HRESULT value and you can see that it means "Object Required".

To me, this means that the CommandBar has already been cleaned up and is no longer accessible. ThisAddIn_Shutdown is supposedly the first user code to be run before a shutdown occurs, so what do you do?

I have settled on listening to the Close event of the main Explorer window (the one the CommandBar is added to). This is called before ThisAddIn_Shutdown and the CommandBar is still accessible.

Example:

  ...

  private void ThisAddIn_Startup(object sender, System.EventArgs e)
  {
      Outlook.Explorer explorer = this.Application.ActiveExplorer();
    
      if (explorer != null)
      {
          ((ExplorerEvents_10_Event)explorer).Close += new ExplorerEvents_10_CloseEventHandler(ThisAddIn_Close);

          commandBar = explorer.CommandBars.Add(TOOLBARNAME,
              Office.MsoBarPosition.msoBarFloating, false, true);
        
          ...
      }
  }

  private void ThisAddIn_Close()
  {
      SaveCommandBarSettings();
  }

  ...