63

Closed

System.NullReferenceException: Object reference not set to an instance of an object.

description

From time to time, my clients gets this error.

System.NullReferenceException: Object reference not set to an instance of an object.
at System.Collections.Generic.Dictionary`2.Insert(TKey key, TValue value, Boolean add)
at AjaxControlToolkit.ToolkitScriptManager.GetScriptCombineAttributes(Assembly assembly)
at AjaxControlToolkit.ToolkitScriptManager.IsScriptCombinable(ScriptEntry scriptEntry)
at AjaxControlToolkit.ToolkitScriptManager.OnResolveScriptReference(ScriptReferenceEventArgs e)
at System.Web.UI.ScriptManager.RegisterScripts()
at System.Web.UI.ScriptManager.OnPagePreRenderComplete(Object sender, EventArgs e)
at System.Web.UI.Page.OnPreRenderComplete(EventArgs e)
at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)

file attachments

Closed Jan 23, 2013 at 7:36 PM by Superexpert
Fixed with January 2013 Release of the Ajax Control Toolkit -- Pull Request merged.

comments

smartgeek wrote Jun 1, 2010 at 9:05 AM

I got several random exceptions like this in production (IIS 6), and only one in debug (IIS 5.1).
We use the last AjaxControlToolkit release (file version 3.5.40412.1).

According to ToolkitScriptManager.cs, this exception is thrown here :
line 81: _scriptCombineAttributeCache[assembly] = attributes;

But "assembly" was correctly assigned. It points to our ComponentArt.Web.UI assembly (2007.2 release).
The morning, our main website locked up, after this exception was thrown. We don't know if that's related to this, but that's the first time it lock up.

And sorry for my bad english.

ivanc wrote Jul 16, 2010 at 9:44 PM

We are receiving this exception in production.
after initial exception the server goes into a state when the IIS workign process is at 100% CPU.
we did a crash on the production server and the CLR stack looks like this: (I pasted just stack form two threads but all but one threads were in the same state of execution)

OS Thread Id: 0x140c (60)
Child-SP RetAddr Call Site
000000000096d7c0 000006427830c174 System.Collections.Generic.Dictionary2[[System.__Canon, mscorlib],[System.__Canon, mscorlib]].FindEntry(System.__Canon)
000000000096d830 00000642804997ef System.Collections.Generic.Dictionary
2[[System.__Canon, mscorlib],[System.__Canon, mscorlib]].TryGetValue(System.__Canon, System.__Canon ByRef)
000000000096d870 00000642804990c2 AjaxControlToolkit.ToolkitScriptManager.GetScriptCombineAttributes(System.Reflection.Assembly)
000000000096d8c0 000006428049703c AjaxControlToolkit.ToolkitScriptManager.IsScriptCombinable(ScriptEntry)
000000000096d960 00000642ad9ab61f AjaxControlToolkit.ToolkitScriptManager.OnResolveScriptReference(System.Web.UI.ScriptReferenceEventArgs)
000000000096da70 00000642ad9ad5a8 System.Web.UI.ScriptManager.RegisterScripts()
000000000096db00 00000642bd39ecee System.Web.UI.ScriptManager.OnPagePreRenderComplete(System.Object, System.EventArgs)
000000000096db90 000006428048e6c8 System.Web.UI.Page.OnPreRenderComplete(System.EventArgs)
000000000096dbd0 00000642bc919d88 SportsWear.BasePage.OnPreRenderComplete(System.EventArgs)
000000000096dc40 00000642bc918db0 System.Web.UI.Page.ProcessRequestMain(Boolean, Boolean)
000000000096dd10 00000642bc918cdb System.Web.UI.Page.ProcessRequest(Boolean, Boolean)
000000000096dd70 00000642bc918c70 System.Web.UI.Page.ProcessRequest()
000000000096ddd0 000006428032b554 System.Web.UI.Page.ProcessRequest(System.Web.HttpContext)
000000000096de30 00000642bc920117 ASP.frontpage_aspx.ProcessRequest(System.Web.HttpContext)
000000000096de60 00000642bc8e449b System.Web.HttpApplication+CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
000000000096df10 00000642bc8f2215 System.Web.HttpApplication.ExecuteStep(IExecutionStep, Boolean ByRef)
000000000096dfb0 00000642bc8e3553 System.Web.HttpApplication+ApplicationStepManager.ResumeSteps(System.Exception)
000000000096e060 00000642bc8e7874 System.Web.HttpApplication.System.Web.IHttpAsyncHandler.BeginProcessRequest(System.Web.HttpContext, System.AsyncCallback, System.Object)
000000000096e0c0 00000642bc8e745c System.Web.HttpRuntime.ProcessRequestInternal(System.Web.HttpWorkerRequest)
000000000096e150 00000642bc8e608c System.Web.HttpRuntime.ProcessRequestNoDemand(System.Web.HttpWorkerRequest)
000000000096e190 000006427f602012 System.Web.Hosting.ISAPIRuntime.ProcessRequest(IntPtr, Int32)
OS Thread Id: 0x144c (61)
Child-SP RetAddr Call Site
000000000d5bd680 000006427830c174 System.Collections.Generic.Dictionary2[[System.__Canon, mscorlib],[System.__Canon, mscorlib]].FindEntry(System.__Canon)
000000000d5bd6f0 00000642804997ef System.Collections.Generic.Dictionary
2[[System.__Canon, mscorlib],[System.__Canon, mscorlib]].TryGetValue(System.__Canon, System.__Canon ByRef)
000000000d5bd730 00000642804990c2 AjaxControlToolkit.ToolkitScriptManager.GetScriptCombineAttributes(System.Reflection.Assembly)
000000000d5bd780 000006428049703c AjaxControlToolkit.ToolkitScriptManager.IsScriptCombinable(ScriptEntry)
000000000d5bd820 00000642ad9ab61f AjaxControlToolkit.ToolkitScriptManager.OnResolveScriptReference(System.Web.UI.ScriptReferenceEventArgs)
000000000d5bd930 00000642ad9ad5a8 System.Web.UI.ScriptManager.RegisterScripts()
000000000d5bd9c0 000006428003a119 System.Web.UI.ScriptManager.OnPagePreRenderComplete(System.Object, System.EventArgs)
000000000d5bdb10 000006428048e6c8 System.Web.UI.Page.OnPreRenderComplete(System.EventArgs)
000000000d5bdb50 00000642bc919d88 SportsWear.BasePage.OnPreRenderComplete(System.EventArgs)
000000000d5bdbc0 00000642bc918db0 System.Web.UI.Page.ProcessRequestMain(Boolean, Boolean)
000000000d5bdc90 00000642bc918cdb System.Web.UI.Page.ProcessRequest(Boolean, Boolean)
000000000d5bdcf0 00000642bc918c70 System.Web.UI.Page.ProcessRequest()
000000000d5bdd50 00000642807ba1a4 System.Web.UI.Page.ProcessRequest(System.Web.HttpContext)
000000000d5bddb0 00000642bc920117 ASP.product_aspx.ProcessRequest(System.Web.HttpContext)
000000000d5bdde0 00000642bc8e449b System.Web.HttpApplication+CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
000000000d5bde90 00000642bc8f2215 System.Web.HttpApplication.ExecuteStep(IExecutionStep, Boolean ByRef)
000000000d5bdf30 00000642bc8e3553 System.Web.HttpApplication+ApplicationStepManager.ResumeSteps(System.Exception)
000000000d5bdfe0 00000642bc8e7874 System.Web.HttpApplication.System.Web.IHttpAsyncHandler.BeginProcessRequest(System.Web.HttpContext, System.AsyncCallback, System.Object)
000000000d5be040 00000642bc8e745c System.Web.HttpRuntime.ProcessRequestInternal(System.Web.HttpWorkerRequest)
000000000d5be0d0 00000642bc8e608c System.Web.HttpRuntime.ProcessRequestNoDemand(System.Web.HttpWorkerRequest)
000000000d5be110 000006427f602012 System.Web.Hosting.ISAPIRuntime.ProcessRequest(IntPtr, Int32)
OS Thread Id: 0xfb8 (62)

ivanc wrote Jul 16, 2010 at 9:47 PM

Just an addition to the previous post:
after IIS reset this issue goes away and may randomly show up on one of the servers. The servers are x64 IIS 6.0 on Windows 2003 Standard server.

CodeManPlex wrote Jul 20, 2010 at 5:25 PM

I also get this error from time to time.

r_k wrote Oct 27, 2010 at 2:29 PM

We recently upgraded the AjaxControlToolkit in our production system to version 3.5.40412.0 from version 3.0.20229.0. We were using the previous version for at least 2 years and never saw this issue crop up. The new version has been in place now a couple of weeks and now we see this issue come up.

We're not currently load balancing so this happens regardless of that or not.
The exception and stacktrace:

Type: System.NullReferenceException
Source: mscorlib
Message: Object reference not set to an instance of an object.
Stack Trace: at System.Collections.Generic.Dictionary2.Insert(TKey key, TValue value, Boolean add)
at System.Collections.Generic.Dictionary
2.set_Item(TKey key, TValue value)
at AjaxControlToolkit.ToolkitScriptManager.GetWebResourceAttributes(Assembly assembly)
at AjaxControlToolkit.ToolkitScriptManager.IsScriptCombinable(ScriptEntry scriptEntry)
at AjaxControlToolkit.ToolkitScriptManager.OnResolveScriptReference(ScriptReferenceEventArgs e)
at System.Web.UI.ScriptManager.RegisterScripts()
at System.Web.UI.ScriptManager.OnPagePreRenderComplete(Object sender, EventArgs e)
at System.Web.UI.Page.OnPreRenderComplete(EventArgs e)
at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)

cscortes wrote Nov 9, 2010 at 8:00 PM

Yep, we have this same issue and we have seen the website lockup. It takes the 8 cpu's to 100% and just stays there until the application pool is reset. It is upsetting our client because it is critical to have our website up for him. I am surprised that this has a "low" priority, it is causing a nightmare situation for us since we can't find out why it fails (I guess, until now) and have to scramble when the client calls us since he has people screaming at him.

JohnScott wrote Nov 15, 2010 at 10:55 PM

We are also seeing this - we end up restarting the app pool several times a week. It would be very nice to have this fixed

Cheers
John

Epic Games, Inc.

Eric123 wrote Dec 9, 2010 at 3:29 PM

My users are experiencing the same problem. They are on Windows Server 2008. Fortunately the problem goes away when they reload, but it sure looks bad. The users really like the functionality the controls provide, so I don't want to remove them.

user423430 wrote Dec 13, 2010 at 7:59 PM

per http://stackoverflow.com/q/2833444/423430 circa May 14 2010:

"the GetScriptCombineAttributesmethod access a static dictionary, which is not protected against concurrent access"

machj wrote Mar 1, 2011 at 8:59 AM

There is still bug in the AjaxControlToolkit? I have latest version (3.5.40412.2). Could be the problem in typing error in "Dictionary2" instead of "Dictionary" ?

Object reference not set to an instance of an object.
in System.Collections.Generic.Dictionary
2.Insert(TKey key, TValue value, Boolean add)
in AjaxControlToolkit.ToolkitScriptManager.GetScriptCombineAttributes(Assembly assembly) in d:\hg\act\Server\AjaxControlToolkit\ToolkitScriptManager\ToolkitScriptManager.cs:řádek 81
in AjaxControlToolkit.ToolkitScriptManager.IsScriptCombinable(ScriptEntry scriptEntry) in d:\hg\act\Server\AjaxControlToolkit\ToolkitScriptManager\ToolkitScriptManager.cs:řádek 481
in AjaxControlToolkit.ToolkitScriptManager.OnResolveScriptReference(ScriptReferenceEventArgs e) in d:\hg\act\Server\AjaxControlToolkit\ToolkitScriptManager\ToolkitScriptManager.cs:řádek 219
in System.Web.UI.ScriptManager.RegisterScripts()
in System.Web.UI.ScriptManager.OnPagePreRenderComplete(Object sender, EventArgs e)
in System.Web.UI.Page.OnPreRenderComplete(EventArgs e)
in System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)

MarjaR wrote Mar 30, 2011 at 6:27 AM

This AjaxControlToolkit error occurs fairly often in my .NET 4.0 web application. I really need a solution!

Argawaen wrote Apr 25, 2011 at 5:37 PM

This is not fixed with the latest release (50401) =(

Why is this marked as Impact 'Low' - IMHO it's the only critical issue in ACT and worthy of a hotfix by itself.

Cheers
John

etolmachev wrote Apr 25, 2011 at 7:39 PM

gamecockguy2003 wrote Jun 20, 2011 at 1:19 PM

This is a serious problem for users of my application. Anyone with high load is going to experience this problem. The only work around seems to be disabling script combining, which trades one performance concern for another. Please fix this.

Yvanthar wrote Jul 13, 2011 at 3:43 PM

Unacceptable in a product this mature

Argawaen wrote Jul 14, 2011 at 5:10 AM

I added a lock around the relevant access (as suggested by etolmachev) and 2 other related dictionaries, compiled a local copy and it's been working great ever since =)

Still, it seems rather silly that there is a major crash problem with a simple fix that has not been incorporated into the mainline, and that users have to compile a local copy to make the library usable.

Cheers
John

rswank wrote Sep 27, 2011 at 2:29 PM

Impact LOW?!?!?!
Every load test we perform floods our logs with this error.
Setting CombineScripts = false in the ToolKitScriptManager control seemed to fix the error but after 5 consecutive load tests the error showed itself again.

If someone has a fix, can they post the actual code here?
We are running IIS 6, Is this still a problem in IIS 7.5?

r_k wrote Sep 27, 2011 at 5:27 PM

@rswank ... refer to this post made several months back...

"etolmachev wrote Apr 25 at 3:39 PM
I've added a lock around the cache in my fork: http://ajaxcontroltoolkit.codeplex.com/SourceControl/network/Forks/etolmachev/ToolkitScriptManager26752"

I implemented that version in our legacy production app and the issue has gone away for us.

Good enough short-term solution IMO, especially considering the AjaxControlToolkit is so beyond the wrong way to be doing Ajax... Your long term solution SHOULD be to stop using Web Forms and AjaxControlToolkit and look into using MVC and start learning how to use Jquery yourself to make a simple ajax call.

Paul_Weston wrote Jul 27, 2012 at 10:37 AM

Hi all,

I tried to implement the fix suggested by etolmachev, but I'm still seeing errors.

@Argawaen - by "2 other related dictionaries" are you referring to the '_scriptResourceAttributeCache' and the '_scriptCombineAttributeCache' dictionaries?

I'm not a C#er, so I confess to being lost when I look at those routines.

The recommended fix is to lock _scriptCombineAttributeCache while we access _webResourceAttributeCache. Is this right?

If so, should we also lock _scriptCombineAttributeCache while we access _scriptResourceAttributeCache and _scriptCombineAttributeCache?

Any help very much appreciated.

Thanks, Paul.

baswol wrote Aug 7, 2012 at 8:00 AM

I added the suggestion on, http://ajaxcontroltoolkit.codeplex.com/SourceControl/network/Forks/etolmachev/ToolkitScriptManager26752 the lock around the cache.

This didnt resolve the issue. The w3wp process hang with 100% cpu.

A few long threads gives error below

Thread 48 - System ID 58864
Entry point ntdll!RtlUserThreadStart+1d
Create time 7-8-2012 8:00:03
Time spent in user mode 0 Days 00:08:10.529
Time spent in kernel mode 0 Days 00:00:09.703




This thread is enumerating a System.Collections.Generic.Dictionary object



The thread has evidence of .net exceptions on the stack. Check the Previous .NET Exceptions Report (Exceptions in all .NET Thread Stacks) to view more details of the associated exception



.NET Call Stack



Function
System.Collections.Generic.Dictionary2[[System.__Canon, mscorlib],[System.__Canon, mscorlib]].FindEntry(System.__Canon)
System.Collections.Generic.Dictionary
2[[System.__Canon, mscorlib],[System.__Canon, mscorlib]].TryGetValue(System.__Canon, System.__Canon ByRef)
AjaxControlToolkit.ToolkitScriptManager.GetScriptCombineAttributes(System.Reflection.Assembly)
AjaxControlToolkit.ToolkitScriptManager.IsScriptCombinable(ScriptEntry)
AjaxControlToolkit.ToolkitScriptManager.OnResolveScriptReference(System.Web.UI.ScriptReferenceEventArgs)
System.Web.UI.ScriptManager.RegisterScripts()
System.Web.UI.ScriptManager.OnPagePreRenderComplete(System.Object, System.EventArgs)
System.Web.UI.Page.OnPreRenderComplete(System.EventArgs)
System.Web.UI.Page.ProcessRequestMain(Boolean, Boolean)
System.Web.UI.Page.ProcessRequest(Boolean, Boolean)
System.Web.UI.Page.ProcessRequest()
System.Web.UI.Page.ProcessRequest(System.Web.HttpContext)
ASP.search_engine_showobject_aspx.ProcessRequest(System.Web.HttpContext)
System.Web.HttpApplication+CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
System.Web.HttpApplication.ExecuteStep(IExecutionStep, Boolean ByRef)
System.Web.HttpApplication+PipelineStepManager.ResumeSteps(System.Exception)
System.Web.HttpApplication.BeginProcessRequestNotification(System.Web.HttpContext, System.AsyncCallback)
System.Web.HttpRuntime.ProcessRequestNotificationPrivate(System.Web.Hosting.IIS7WorkerRequest, System.Web.HttpContext)
System.Web.Hosting.PipelineRuntime.ProcessRequestNotificationHelper(IntPtr, IntPtr, IntPtr, Int32)
System.Web.Hosting.PipelineRuntime.ProcessRequestNotification(IntPtr, IntPtr, IntPtr, Int32)
DomainNeutralILStubClass.IL_STUB(Int64, Int64, Int64, Int32)
DomainNeutralILStubClass.IL_STUB(IntPtr, System.Web.RequestNotificationStatus ByRef)
System.Web.Hosting.PipelineRuntime.ProcessRequestNotificationHelper(IntPtr, IntPtr, IntPtr, Int32)
System.Web.Hosting.PipelineRuntime.ProcessRequestNotification(IntPtr, IntPtr, IntPtr, Int32)
DomainNeutralILStubClass.IL_STUB(Int64, Int64, Int64, Int32)


Full Call Stack



Function Source
mscorlib_ni+30d650
mscorlib_ni+30d544
0x000007ff00bb5f77
0x000007ff
00bb577a
0x000007ff00383726
System_Web_Extensions_ni+20e24f
System_Web_Extensions_ni+20f498
System_Web_ni+d2a6ae
System_Web_ni+29cfc8
System_Web_ni+29bff0
System_Web_ni+29bf1b
System_Web_ni+29beb0
0x000007ff
0035d189
System_Web_ni+2a3357
System_Web_ni+26671b
System_Web_ni+953971
System_Web_ni+9443b2
System_Web_ni+925df9
System_Web_ni+a4dde1
System_Web_ni+a4d9ab
System_Web_ni+a4cc14
mscorwks!UMThunkStubAMD64+7a
webengine!W3_MGD_HANDLER::ProcessNotification+107
webengine!W3_MGD_HANDLER::DoWork+312
webengine!RequestDoWork+44e
webengine!CMgdEngHttpModule::OnExecuteRequestHandler+24
iiscore!NOTIFICATION_CONTEXT::RequestDoWork+233
iiscore!NOTIFICATION_CONTEXT::CallModulesInternal+174
iiscore!NOTIFICATION_CONTEXT::CallModules+25
iiscore!W3_CONTEXT::DoWork+34d
iiscore!W3_CONTEXT::IndicateCompletion+8c
webengine!MgdIndicateCompletion+61
mscorwks!DoNDirectCall__PatchGetThreadCall+7b
System_Web_ni+94e47b
System_Web_ni+a4df10
System_Web_ni+a4d9ab
System_Web_ni+a4cc14
mscorwks!UM2MThunk_WrapperHelper+4b
mscorwks!UM2MThunk_WrapperWorker+99
mscorwks!UM2MThunk_Wrapper+19
mscorwks!Thread::DoADCallBack+145
mscorwks!UM2MDoADCallBack+90
mscorwks!UMThunkStubAMD64+276
webengine!W3_MGD_HANDLER::ProcessNotification+107
webengine!ProcessNotificationCallback+46
mscorwks!UnManagedPerAppDomainTPCount::DispatchWorkItem+157
mscorwks!ThreadpoolMgr::WorkerThreadStart+1ba
mscorwks!Thread::intermediateThreadProc+78
kernel32!BaseThreadInitThunk+d
ntdll!RtlUserThreadStart+1d

jspraul wrote Dec 18, 2012 at 10:38 PM

Attached a Visual Studio 2012 project that will reproduce this error using IIS Express and the September 2012 release for .NET 3.5.
  1. Press F5 to start the site (ToolkitScriptManager and thread synchronization)
  2. Right click the 'repro' project, 'Debug' > 'Start new instance' (10 simultaneous requests)
  3. Press Enter to choose the default IIS Express test url
  4. Watch up to 10 attempts to trigger the error, which is reported as error code 555.
  5. Beware: IIS Express may hit 100% CPU usage
Reproduced on Windows 8 Enterprise x64, i7-3770 3.4GHz (4 cores, 8 logical processors) with 16GB RAM. Some of the delays to allow time for various steps to complete are a little low for slower machines.

Would appreciate a fix for this highest-voted, 2.5 year-old issue.

r_k wrote Dec 19, 2012 at 1:06 PM

AjaxControlToolkit is antiquated technology. It is unlikely that MS will be fixing it. The right thing to do is to think about the future and begin planning for an upgrade to MVC.

There was a fork a ways back that did help us for a while at least while we transitioned away from web forms over to MVC... We still have some older pages in our main CMS that use it and with the fork at the url below, we do not have this issue at all.

http://ajaxcontroltoolkit.codeplex.com/SourceControl/network/Forks/etolmachev/ToolkitScriptManager26752

If the above does not help you then I would say the urgency on that MVC upgrade is a lot higher for you...

That or go in and make your own fix/fork of the toolkit and post it here for others to use. :)

jspraul wrote Dec 19, 2012 at 10:00 PM

As evidenced by the recent September 2012 release, 'Microsoft has asked [...] Superexpert Consulting, to take ownership of the development and maintenance of the Ajax Control Toolkit' blog, and Superexpert Consulting is fixing issues in the Ajax Control Toolkit. I do not know if they get anything from Microsoft for this effort.

Recognizing the need to transition away from the Ajax Control Toolkit is orthogonal to fixing issues that affect existing applications. The only reason I could think of for not fixing this issue was the inability to reproduce it reliably, which I attached yesterday. This made it easier for me to test my attempts at a fix. As has been pointed out multiple times in the comments, CodePlex user etolmachev checked in a partial solution back in April 2011; the root cause was reported in Dec 2010 -- copied from a StackOverflow post made in May 2010. There has been almost no additional work required to fix this issue for between 1 to 2.5 years.

I have submitted a pull request which locks all three of the static custom attribute dictionary caches in ToolkitScriptManager. This is only a slight tweak of etolmachev's fix, as recommended by CodePlex user Argawaen (John) below, and sacrifices a little performance for simplicity. I am attaching a build for .NET 3.5 and 4.0 which includes this fix (2012sep-ajaxcontroltoolkit-bin-2bbbad3d241c.zip).

jspraul wrote Dec 19, 2012 at 10:06 PM

Hey, I was promised markdown! : )