Mister SQL

The plug in needs a way to talk to the host app. In ASP.NET, this means things like updating the UI by adding controls, etc. In fact, since in ASP.NET only creates pages, you will eventually have to talk to Page. The host Page isn’t serializable or remotable, so if your Contract has a Page, you won’t be able to create the Add in in a separate AppDomain, i.e. no sandboxing or other isolation, at least not directly. Fortunately, we like to write code and can solve this by writing another layer of indirection.

What I tried next was creating a set of Page and Control wrappers and factory functions. I called the add-in-side wrappers “Recipes” because they where recipes for building controls and recipes for building pages. The new contracts no longer had references to System.Web since no WebForms types there seem to be remotabe. In the hosting application I wrote a “Page Automator”, whose job was to implement recipes recommended by the Add-In and to describe the page for the Add-In by looping through the controls, putting their current state into a wrapper and then providing it to the Add-In.

Finally, out-of-AppDomain Add-In activiation works. And I was able to run the add in in minimal trust. If you’ve ever tried to get a non-trivial website to run under medium trust, you know this is just incredible! I gave up on medium trust because too many of my third party components didn’t play the medium trust way. In a medium trust website, everything in the AppDomain (i.e. the entire website) must refrain from calling forbidden APIs. As soon as there is a single line of code that requires Full-Trust and if you can’t remove it or change it (because it is both necessary and 3rd party closed source), you have to bump the entire website up to Full Trust. Unless you are using Add-Ins.

The plug in may need to communicate several things at once in the AddIn’s interface. System.AddIn makes this very, very hard. I tried string[]. I tried List. I tried IListContract. They all failed. When I used IListContract, the pipeline generator generated interfaces of type IList, and then the addin finder stopped finding addins that supported the contract. Humorously, string[] failed because System.AddIn said string[] wasn’t serializable. WTF? These are strings. I finally decided to use Control NextControl(); as a signature to return a collection one at a time. I finally did get IListContract to work, but the type of T needs to be remotable.

Tags: System.AddIn

Plug ins allow developers to collaborate with zero co-ordination. The host application provides extension points and another developer writes a module that the host application can use without re-compile. A typical plug in architecture also includes:

- Activation. A way to enable and disable plug ins at runtime
- Discovery. Plug ins are registered by scanning a directory for code artifacts.
- Code artifacts that implement an expected interface are used by the host application
- Plug ins are invoked with some sort of reflection
- Plug ins might need to run with fewer security rights than host code.

MAF/System.AddIn exists.
MEF is on the way,but it looks like it will be fully baked in .NET 4.0, no sign if it will support 3.5.

MEF is getting much more attention on the blog and the web and I attribute that entirely to the MEF team promoting their work more than the MAF team.

Benefits of MAF
Its a documented solution, that will always be better known and understood than any plug-in architecture you could throw together.
It is available in release form *now* and doesn’t require 4.0. MEF is pre-release and will require upgrading to 4.0 when it is release. (I think)

Challenges with MAF:
They team assumes you want version resilience. This is the most expensive feature. It achieves version resilience by adding 5 assemblies between your host and plug ins. Without code generation, the extra five assemblies adds too much work.

Also, when it comes time to write the V2 to V1 adapters, you will have to make ugly choices like, should the adapters throw errors or should they similate the new functionality. For example, in the calculator example where V1 supported adding and subtracting and V2 supported multiplication, you could have simulated multiplication by calling the V1 addition method in a loop to replicate multiplication. This would be better backwards compatibility, but now you got some business logic in your ViewAdapters. If you don’t put that logic there, then you will throw NotImplementedExceptions. Now how is a component backwards compatible if it throws runtime errors? (Well, admittedly the adapter lets the old plug in work as well as it used to just so long as you keep away from “new funcitonality” that arrived in V2)

They assume you need the extra security and reliability you get from loading plug ins in a sandbox. This means MAF wants you to declare contracts with primative datatypes if you can. Just like you can’t pass database connections or the HttpContext back and forth across a webservice, you aren’t supposed to do that in MAF (but you can, as long as you don’t load plug ins in a new AppDomain.)

Plug in UI. Officially, you aren’t supposed to pass winforms controls across MAF contracts, ie only the host has UI. WPF controls are supposed to be able to cross, ie. the plugin can have UI.

In ASP.NET, my own experimentation shows you can have server controls in the plug in, but it is hard beyond being worth it to put a Page or UserControl in a plug in and then get it to render on an ASP.NET page. This means that all ASP.NET plugins must mix code and UI more than I like. Some of this can be mitigated by the host providing enough extension points that it wouldn’t matter. For example, if a pluggable form page queried the plug in for the controls and used a host owned layout engine to layout the labels and input boxes, then the plug in writer wouldn’t have to mix too much presentation code and C#.

How does MAF relate to other similar ASP.NET technologies?
ASP.NET already can cope with new pages and classes being added at run time with minimal effect on the running application. A pattern I’ve seen is where V1 of the website is deployed, then when V2 is deployed, only the classes and pages that the developer thinks changed are copied into the V1 website. This creates a blended version V1/V2 website. If the contracts changed, e.g. a property of a webcontrol has a different data type, it is up to the developer to also track down the dependencies and deploy new versions of those too. Because pages are compiled on first request, the developer may not learn about broken contracts until long after the code in in production.

WebParts lets chunks of code and UI get plugged into or removed from a page at runtime. WebParts assume the webparts are being developed by the same team as the host, so the there isn’t necessarily any features for versioning, isolation or what have you. As a component technology, WebParts is all about a specific type of UI widget, where AddIns is *anything*, including things like adding an Icelandic spell checker to your website.

ASP.NET and process isolation.
ASP.NET and JSP and most other non CGI web technologies were created because spawning a new process for each request crushed server when they were under load. A common MAF recommendation is to design for the possibility of running your Add Ins in a separate process. But why would I want to constrain my design so that future developers will have the option of running a new process per page request and destroying the ability to support more than a few dozen concurrent requests? I’m convinced that a MAF addin running in ASP.NET should not take advantage of process isolation.

MAF and COM
It’s been noted that MAF has a lot of COM nostalgia. I think this is because MAF was created to solve the problems of creating Microsoft Office add ins. Microsoft office is a COM application and as such will work best with a component technology that has a lot of parallels with COM. This is a challenge for people like me because I never wrote C++ and don’t have an internal mental model of how COM works.

ASP.NET will kick off a little profile thingy when it finds a <property> section. It appears to go into the ASP.NET Temporary Folder. It seems to be a private method in ProfileBase that loads this.

Some errors that happen:
Sometimes deleting the <property> section and readding it works. This happens all the time for all sorts of reasons, not just profile related, but dynamic compilation and shadow assembly related.

ProfileCommon not available- this is supposed to be compiled either at design time, run time or sometime. It fails to work in a variety of circumstances.

Precompiled apps can have problems with the ProfileBase class
Ambiguous reference errors- can happen when App_Code.Compiled or PrecompiledApp.config exist or fail to exist and ProfileCommon gets compiled twice.
Similarly copying the bin of the new version into the bin of the old is bad. You get a blended set of .dlls
Deleting files in ASP.NET Temporary Files sometimes works.

Interestingly, all the above don’t seem to be addressable by writing a custom provider. It doesn’t seem possible to override the settings part, where it reads web.config and starts trying to create that problematic ProfileCommon.

Conclusion
It turned out to be a dll that was referencing a native dll that wasn’t on the PATH. The profile was just triggering a compile. The compile then check references for everything in the \bin\ folder, even if the profile or invoked page isn’t using it. Oddly, this doesn’t happen on every single page, so there are some code paths where ASP.NET doesn’t feel the need to recompile every !@#$@!#$ thing in the world. This, I suppose is good or else we’d see pathalogical compile churning.

I diagnosed this by watching the list of assemblies that were getting loaded in Visual Studio while the debugger was attached. If it couldn’t load symbols or if it was obviously a COM dll, I considered it a candidate and finally narrowed it down to a SMO library, which happens to have non-managed dependencies.

Superpiplining. Methods that call methods that call methods do not magically add value along the way.

If one tier is bad, two tiers is better, why not n tiers (or layers), I mean, literally an infinite number of tiers, each adding zero value along the way. I call this patter “Super Pipelining” Super pipelining makes dependency tracing a pain. A typial super-pipelined application will still put all authorization, validation, calculations in the UI and pass System.Data or SqlClient objects back and forth the super pipeline.


class Customer() {
public void AddCustomer(Dto customer){ (new CustomerBusinessObject()).AddCustomer(); }
}

class CustomerBusinessLogicDomainScenarioDatabasePersistenceObject() {
public void AddCustomer(Dto customer){ (new CustomerBusinessObject()).AddCustomer(); }
}


Ziggurat. A class that inherits from a class that inherits from a class that inherits from a class where the middle classes and the base don’t do anything. The classes in the middle do not magically make your code better.


class BusinessObject: BaseObject {}
class BaseObject: BasementObject{}
class BasementObject: UndergroundObject{}
class UndergroundObject: UbberBasementObject{}
class UbberBasementObject: SuperDuperBasementObject{}
class SuperDuperBasementObject: AceOfBaseObject{}
class AceOfBaseObject: China{}


I got the following error when using Enterprise Library 3.1. It mean it can’t find the reference to ObjectBuilder or it is resolving to the wrong version in GAC. So to use VAB, you need Common, ObjectBuilder and the VAB assembly. If not that then check this link.

System.TypeLoadException: Method ‘CreateObject’ in type ‘Microsoft.Practices.EnterpriseLibrary.Validation.Instrumentation.ValidationInstrumentationListenerCustomFactory’ from assembly ‘Microsoft.Practices.EnterpriseLibrary.Validation, Version=3.1.0.0, Culture=neutral, PublicKeyToken=21a151ab59c9591d’ does not have an implementation.

This is a misleading error that means you forgot to put the connection string into web.config.

So if you get:


WebHost failed to process a request.
Sender Information: System.ServiceModel.Activation.HostedHttpRequestAsyncResult/63432468
Exception: System.ServiceModel.ServiceActivationException: The service '/Services/Astoria.svc' cannot be activated due to an exception during compilation. The exception message is: Service 'Astoria' implements multiple ServiceContract types, and no endpoints are defined in the configuration file. WebServiceHost can set up default endpoints, but only if the service implements only a single ServiceContract. Either change the service to only implement a single ServiceContract, or else define endpoints for the service explicitly in the configuration file.. ---> System.InvalidOperationException: Service 'Astoria' implements multiple ServiceContract types, and no endpoints are defined in the configuration file. WebServiceHost can set up default endpoints, but only if the service implements only a single ServiceContract. Either change the service to only implement a single ServiceContract, or else define endpoints for the service explicitly in the configuration file.
at System.ServiceModel.Web.WebServiceHost.AddAutomaticWebHttpBindingEndpoints(ServiceHost host, IDictionary`2 implementedContracts, String multipleContractsErrorMessage)
at System.ServiceModel.Web.WebServiceHost.OnOpening()
at System.ServiceModel.Channels.CommunicationObject.Open(TimeSpan timeout)
at System.ServiceModel.ServiceHostingEnvironment.HostingManager.ActivateService(String normalizedVirtualPath)
at System.ServiceModel.ServiceHostingEnvironment.HostingManager.EnsureServiceAvailable(String normalizedVirtualPath)
--- End of inner exception stack trace ---
at System.ServiceModel.AsyncResult.End[TAsyncResult](IAsyncResult result)
at System.ServiceModel.Activation.HostedHttpRequestAsyncResult.End(IAsyncResult result)
Process Name: WebDev.WebServer
Process ID: 8168


This is what a connection string for an entity data model looks like. If you created your edmx file in a different project from your .svc file, you will need to manually copy the connection string. In this case I’m using Sqlite (which can’t be design using Visual Studio Express, in case you were wondering)


<connectionStrings>
<add name="calendarEm" connectionString="metadata=res://*/CalendarV1.csdl|res://*/CalendarV1.ssdl|res://*/CalendarV1.msl;provider=System.Data.SQLite;provider connection string="data source=C:\code\CommunityCalendar\Calendar\App_Data\calendar.db3;useutf16encoding=True"" providerName="System.Data.EntityClient" />
</connectionStrings>


Tags: Astoria

I’d like to start an opensource project.

Pick the social contract. I’ve learned the distinction between open source products and open source products. The latter is a social group of people collaborating to create an application. The former is an application that the developer will give the source code for, but doesn’t necessarily do collaborative development outside of processing bug reports, etc.

Pick a host Codeplex is a good host for MS centric projects. Sourceforge is good because it is big. Google code is also not bad.

Check your employment contract and pick a license I can’t help you with the first. As for picking a license, pick something that protects you from lawsuits and is compatible with other open source licenses. Licensing is partly governance, so put some thought into how you plan to govern your application and your relationship with users and other developers.

Pick tools that are widely available. If you can only open the source code with an expensive proprietary IDE, you won’t find many friends to help you. In the .NET world, the ideal project would be able to compile in Visual Studio Express and MonoDevelop.

Pick compatible libraries. Closed source applications have an easier time here. If you are completely ripping off someone’s libraries in an intranet application, the rights holders might never discover. In an open source application, it will eventually get indexed and lawyers will look over your code looking for lawsuit opportunities.

Don’t make it hard to install You may love SQL Server. But taking a dependency on something as user friendly as SQL Server will decimate your potential user base. In the case of SQL Server, either choose SQL Compact or Sqlite. MySql has the the problems as SQL–it is a bit to hard for people to use. Your app will not likely have a DBA– even if a company picks up your product, it will likely be a small company that still doesn’t have a DBA. It should work by unzipping into a directory and be portable on a thumb drive.

Ask for help Ask for help in your app– set up something like a uservoice page for feature requests and discussion, a bug tracker that accepts public input.

Tags: opensource

SQL Express 2008 Hates WHS. So I have Windows Home Server. Finally, I think, I can run a publicly accessible Windows 2003 box with SQL from my bedroom and the licensing is all pretty legit. I try to install SQL2008 from the Web plantform installer. It barfs. I read this blog post and try again, without the platform installer. It appears to install, but then it fails to start. The error logs claim that the master DB is on an E: drive. I try to uninstall. The un-installer says the attributes on the folders are different, so it refuses to uninstall. Reported bug on MS connecte.

Ok, 1/2 GB of space on my WHS burned with no way of deleting it. I even tried zipping the folder and then deleting the program files\Microsoft SQL folder, but it says some file is in use. Sigh.

SQL 2005 Reporting Services Hates SSL certs on IIS 6 on WHS. So I try to install SQL Express. Reporting services barfs on install (”SQL Server Setup failed to retrieve the SSL Certificate Name”) because it doesn’t like the perfectly valid SSL certificate. The web says rip out your SSL certificate and SSRS will install. Right, like I’d want to rip out my left eyeball. WHS doesn’t have a louse selfssl cert, it has a green ssl cert from homeserver.com. Green! Do you know how much it costs to get a green certificate? So I try again without RS. This time, the SqlNativeClient fails. It says it can’t find an MSI, sqlncli.msi. (follow link for the real fix, which is to uninstall the snc from previous attempts) And the install fails. I suspect it is because the installer automatically unpacks to the D:\guid folder. But the D: drive is some super weird NTFS configuration that most programs don’t know how to read or write to, unless they use the UNC or shared folders.

I finally succeeded before I had to try MSDB or MySQL. Next time, I’m going to seriously consider MySQL probably with XAMPP because XAMPP hasn’t sold their soul to the registry, the team that uses MSI installers and other evil things.

Tags: WHS, Windows Home Server

Technical Anonymity. You use proxies, avoid cookies, you browser leaks no information about your IP, computer or other characteristics about your network traffic and computer. Your cover is blown as soon as you include your name, or other correlating information in a blog entry. Your cover will also be blown as soon as the intermediate parties (the proxy owner, the email provider) decides to hand over your information, say under legal pressures from the government.

Who needs technical anonymity? Spammers and people who’ve violated TOS clauses only need technical anonymity, because the traces that technical anonymity techniques remove the clues that web sites use to ban bad actors. Spammers don’t care if people know who they are as long as they can send email with being automatically ID’d as spammers.

Real humans need social and legal anonymity more than technical anonymity.

Social Anonymity. You don’t include your name or any other correlating information. You even try to change your writing style to avoid tools that correlated your favorite ways to misspell words with other documents on the web you publish under your real name. If the people who want to know who you are lack the resources or skill to trace IP addresses and correlate your ID from scraps of info like cookies and information leaked by your browser, then this can be done probably without any technical anonymity.

Who needs social anonymity? Bloggers and people writing about anything that might piss of the Jones up the street. If you decide you don’t want to be harassed for your religion, politics, personal behavior, etc, you might rather to be anonymous instead of being silenced by your personal enemies. For example, if you want to blog about politics, you don’t want it to ruin your chances of getting a job. These people will not user the courts to extract your ID from a commercial proxy service.

Legal anonymity. You are at risk of being throw into jail or losing your job if your ID is discovered– either because the government is corrupt, your company is evil or you really are a criminal. This is the hardest level to achieve because getting the email and proxy requires relying on 3rd parties to move your network traffic and those parties can be subject to government pressure to fork over your ID. Even commercial anonymity services don’t want to protect tax dodgers, mass murderers or spammers.

Sounds hard, is this a fools errand? Technical anonymity is possible, but you are likely to make mistakes, especially over a long period of time. Social anonymity is possible, but the odds of subtle mistakes are very high. Legal anonymity is probably impossible unless you make sure all your intermediate parties are outside your legal jurisdiction. However, as soon as your ID is being protected by a party outside the law, they you can’t enforce your contract with them regarding how they keep your ID safe! Imagine a proxy server company in Bahama that goes bankrupt and sells it’s assets with scraps of your ID to a US company and then your cover is blown again.

Why not use Tor? Tor would make legal anonymity easier because it isn’t a company and it mixes up tons of legal juridsictions, but the performance of the Tor network makes it unusable.

This is the flips side of what I usually work with: figuring out who really is at the other end of an HTTP request. What if some one doesn’t want to be known at all? It turns out to be as hard as proving you are who you say you are!

Tor. Re-route your traffic through a few nodes donated by nice people until your point of origin is hard to trace. Free. Down side, performance is about 100 times slower than without Tor. And I measure that. It’s slower than a modem on a noisy line. The system is currently overrun by bittorrent traffic and there are no incentives to be an exit node: no money and the exit node owners get blamed for the IP laws broken while traversing them.

Commercial Proxies. I tried out Anonymizer. Performance is good, but you can’t directly sign up for email accounts when your IP is proxied. The email providers assume you are a spammer if you are trying to create an account while proxied.

Java Applets, your browser, etc.. All of these fancy client features leak information about who you are. Java applets can even get an honest answer about your unproxied IP address even when your browser is only reporting the proxied IP address! I think, for a nosy website to use this info, they’d have to have a special crafted java applet, AFAIK, an ordinary applet doesn’t leak information.

You’d think that with all this leaked info, we could use it to ID someone routinely, but IP tracing is hard detective work. You’d have to be doing something remarkable to make it worth someone’s effort to track you down by the crumbs you leave when surfing.

Social Engineering Still Rules. I imagine that if a user covered all their tracks, as soon as they accidentally say something online that ties them to a particular person in real life, the game is over.