Converting from Access to SQL Server
Author: Eli Robillard
Copyright: 2003, all rights reserved.
The document assists users migrating from Microsoft Access to Microsoft SQL Server. Though written specifically for users of GenericDB, the process may be applied to any Access database.
Of particular note are the steps to convert a Random Autonumber field to one which uses Incremental Autonumbers prior to migrating data. Such fields are mainly used as Primary Keys. SQL Server does not natively support randomly generated record ID's, but does support automatic incremental ID's with the IDENTITY property. When one simply moves a table into SQL Server and sets the key field to IDENTITY, SQL Server attempts to pick up the numbering with the next highest value, which may be near the upper bounds of an integer field type, effectively limiting the growth of the table.
Steps included in this document help a user convert an Autonumber table key from Random to Incremental, while maintaining referential integrity with any Child tables. The basic technique could also be used to convert GUIDs or any other Primary Key scheme to Incremental Autonumber.
The article assumes little prior knowledge and includes both one-time setup steps and conversion techniques which many databases do not need. Once you get the hang of it, the actual transfer process is pretty fast, a matter of minutes for most tables.
- Microsoft Access 2000. Earlier versions are acceptable.
- Microsoft SQL Server 2000, SQL Server 7 is acceptable.
- SQL Enterprise Manager, Version 8.0. Some steps may differ with earlier versions.
Step One - Clean the Access Data
Clean the data. Repair and compact the data. There is no sense in uploading junk. It pays to treat your SQL data well, with the sort of care and consideration as accountants give to financial logs and transactions. Access makes it easy to treat data informally, sloppily, lazily. Use the conversion as an opportunity to get your data into "a good place."
Clean out bad data. Remove test data. Replace Memo fields with long String fields where possible. SQL Server requires that table and field names be 30 characters or less, begin with a letter and contain no spaces.
For tables you access with GenericDB, you may need another structural change. GenericDB depends on a key field in each table to refer to records, usually an Autonumber field. The corresponding concept in SQL Server is IDENTITY, which is a property you can apply to an integer (type integer, bigint, smallint, or tinyint) or decimal field. Specifically, IDENTITY treats a field like an Incremental Autonumber field. When you upload Incremental Autonumber fields to SQL Server some work is done for you, and you can go ahead to Step Two. If you use Random Autonumbers, you have some conversion to do.
Random Autonumbers are recommended for Access keys. This makes it harder for a collision to happen when two or more users add new records at the same time. With Incrementals, two users can get the same "next available ID number" and one gets an error (colllision) when the record is saved. Random Autonumbers are also required for Database Replication. In fact making a database replicable automatically converts Incremental Autonumbers to Random. But Randoms present a problem when converting to SQL Server, because when the field is converted to an incremental IDENTITY, the "next available number" might be very high, close to the upper bound of integers, effectively limiting the size of the table. That won't do.
The solution is to convert Random Autonumbers to Incremental Autonumbers.
Converting from Random to Incremental Autonumbers
a) Backup your database.
b) Make sure any relationships are defined in Access. If your database contains parent-child relationships based on your key field, use the Relationship tool to make sure that Access knows about them, so Referential Integrity is enforced. If you don't have any relationships I'll still call your table the "parent" in the next steps.
c) In the parent table, change the Autonumber key field to Data Type: Number; Field Size: Long Integer. Press Ctrl-S to Save.
d) Also in the parent table, insert a new field called "NewID" of Data Type: Autonumber; Field Size: Long Integer; New Values: Increment. The values are automatically generated when you save this change. If you have no child relationships, skip ahead to (g).
e) In all related child tables, insert a new field also called "NewID" of Data Type: Number; Field Size: Long Integer; New Values: Increment. This field starts empty, we fill it next.
f) Create a Query in design view. Add the parent table and the child table to the view. The old relationship should be drawn automatically, if not then go back to step 2. Click the Query menu and select type "Update Query." Add the child's NewID field to the query grid. Beside "Update to:" type
[ChildTableName].[NewID] where ChildTableName is the name of your child table. Click the Query menu and select "Run." Repeat this step for every child table, you do not need to save this query when done.
g) Now go back to parent table's Design view and delete the old ID field. Also open any child tables in Design view and delete their old key fields too. When you do so, it will ask you if it should delete related indices and relationships, answer "Yes." If you feel sentimental about the fields nervous about this step, you can optionally rename these now-unused fields but do delete the relationship.
h) Rename your "NewID" fields to whatever the old ones were called in both the parent and child tables.
i) If you had parent-child relationships, you can recreate them now with the Relationship Tool.
j) You've invested some time in this. Make another backup.
k) Update your database's documentation. If you don't keep any, now is a good time to start a Change Log, be sure to note where you put the backups made in (b) and (j).
Step Two - Create user accounts and a database on SQL Server
If you host your own SQL Server and do not already have one, you should create a new Database. This requires credentials for the sa account. If several people use the same SQL Server, you should create a separate User account to create and administer your own Database. A third User account should exist or be created to represent the rights of people browsing the site, similar to the IUSR_machinename account in IIS.
I use an ISP so I do not have that level of control. I requested a new MS SQL database and kept a record of the dbname, username, and password granted by the ISP. I use this account to administer my database. My ISP also has an account called "PublicUser" with no password to whom I can assign rights to my tables and stored procedures. The ISP also provides an IP number to connect to. Save these to construct your connection string later.
Step Three - Setup Enterprise Manager, Connect to SQL Server
I didn't have Enterprise Manager on my desktop, so I installed it from the SQL Server 2000 Enterprise Manager CD. Choose the first option to Install SQL Server, and the installer advises that it cannot install SQL Server on a desktop, but that it can install the Client tools. This is what you want, the Client tools include Enterprise Manager.
Next, in Enterprise Manager I opened up Microsoft SQL Servers and right-clicked to add a New SQL Server Group which I called GenericDB. Name your own database according to its purpose. I also have an existing SQL Server group on my local machine which I set up previously for Visual Studio .NET. I could have added my ISP's server to this group, but each will be used for different purposes so I chose to keep them separate.
Then I added a New SQL Server Registration to my new group. The wizard asks for the server name -- this is where I provide the IP address, username and password I recorded in Step Two. If everything is correct, the server will appear on the list with several sub-folders representing the database entities -- Databases, Data Transformation Services (DTS), Management, Replication, Security, Support Services, and Meta Data Services.
Step Four - Upload the Data
To upload your data you create a DTS Package. The DTS package contains all the required information about your source database, the destination server, and any transformations to be made along the way. You can save a DTS package for re-use and some ISP's will let you schedule DTS jobs. When you use the Import Wizard, the final step will provide the option to save your Import as a DTS package on the server for future use.
The Import Wizard
Right-click the Databases folder, and select "All Tasks, then "Import Data..." to start the wizard.
Page one selects the data source. If your data source is another SQL Server (e.g. the local SQL Server 2000 which installs with Visual Studio .NET 2003), select a machine name beside Server: and click Next. The default setting (local) does not work on my machine, it may work on yours. Depending on the server configuration you may need to provide a Username and Password, but for local databases Windows Authentication usually works.
If your data source is Microsoft Access, now set the Data Source as Microsoft Access. Then specify your source *.mdb file. Most Access databases aren't secured so unless yours is, leave Username and Password blank.
Page two of the Wizard double-checks the Destination. All should be good here, look it over and click Next.
Page three provides the option to either Copy entire tables, or Use a query to define source data. The next step lets us pick specific tables, I don't want to filter my data with a query, so I go with Copy.
Page four lists my Access tables. I select the ones I want. To alter the table structure on the destination, I choose the Transform option.
Transform is where you deselect fields, adjust destination field types, and Enable Identity Insert. Enable Identity Insert is important for GenericDB users, it lets us specify the key field to SQL Server. Check the box.
Also decide now whether this import will Create the destination table or Append to it. For your first import you want to Create the table. After that you can Append to it, though how to just append the records created since the last import is not described here.
You can also choose to Drop (delete) and recreate the destination table every time. This would be desirable if Access always held the "master copy" -- perhaps a lookup table or canned data used for a demo. Choosing this will of course lose any changes made to existing data in this table on the SQL Server when you run the import, and replace it with whatever your Access table holds.
Next, define your key field as an IDENTITY field. Do this only the first time you import and therefore CREATE the table, or every time if you choose to "Drop and recreate the desination table." After selecting "Create destination table" and editing the fields as you like, click the Edit SQL button. Find the line with your key field, mine is called MessageID and looks like this:
[MessageID] int NOT NULL,
Tell it to make this the "autonumbered" field by inserting the IDENTITY property:
[MessageID] int IDENTITY NOT NULL,
Click OK to finish. Note that if you make any further changes to the field types, you need to go back to Edit SQL to ensure that the IDENTITY property is included in the final conversion.
Do the same for any other tables you wish to migrate.
Page five has two parts, When and Save. The first provides options to: Run the task immediately, or Schedule it for later. The lower half lets you save your job as a DTS package for future use, the convenient location being on the SQL Server. If you do so you will be asked to name the DTS package, describe it, and provide passwords to change (Owner password) or execute (User password) the package. If you chose for the Import to Drop and recreate the destination table(s) each time, make sure this is the behaviour you want to repeat before saving a DTS Package.
That's it! Now the gears spring to life and your data moves. A MessageBox at the end confirms the success of the operation.
Back in Enterprise Manager you can now Refresh your Tables to see the newly imported tables.
Step Five - Update Connection Strings
It is time to use the connection information (IP_Address, Database_Name, Username, and Password) for that public account from back in Step Two.
DSN-less MS SQL connection strings use the following format:
"Provider=SQLOLEDB;Data Source=Server_IP;User Id=Username;Password=Password; Initial Catalog=Database_Name;"
Update your GenericDB config files accordingly, and remember that for SQL server, you should update dbType to:
Session("dbType") = "SQL"
GenericDB uses this to set the wildcard in searches (% instead of *), to set parameters are for ADO commands, and to write the DELETE query when you delete a record.
You are done.
If you have tried these steps with earlier versions of Access, SQL Server, or Enterprise Manager, please contact me. Likewise if you have any comments or improvements for the methods described. Your input will help improve this document.
When writing this, it most surprised me that not a single "upsizing" guide mentioned converting inappropriate keys like Random Autonumbers into Incremental keys. In fact a Google search on "random to incremental" does not return any results. The step is essential to convert any database with parent-child or master-detail relationships, and it baffles me that this might be the first document to address it. I hope you found it useful.