Shapefile Repair Tool 1.0
Landers Hoang
I have a shapefile that I was working on in ArcGIS Desktop 9.3 get corrupted somehow and it gives me the error "Number of shapes does not match the number of table records.". What I would want is to recover this shapefile as it has a lot of edits that other revisions of the file do not have.
Earlier I recovered from this error using ogr2gui but the tool now just crashes. I have even tried the command line version ogr2ogr but still no luck. This script, Shapefile Repair Tool from the ArcScripts website didn't help either.
This one's a bit more brute force, most of the time it helps, sometimes it doesn't, and while it's probably only a first step toward fixing the problem (rather than a solution unto itself, which it's often not), it can help get you to where you can open the shapefile. Most cases you'll still need to do more manual repair in ArcMap after the shapefile opens (corrupt features?, missing attributes? misaligned attributes? etc.)
Take that file size in bytes and subtract 100 bytes (the header). Of the remainder, divide by 8 (the size of each "word"). The result gives you the number of shape features inside the SHP part of the shapefile.
Procedure Caution:Back up the data before attempting these options.Start with the first option below. If one of the steps below fixes the shapefile, it is not necessary to perform the other steps. If the steps in this document do not salvage the shapefile, revert to the backup file.
Hi,
I would add, that the ogr2ogr command (or any other format translation tool) can be useful also if you have issues with text encoding of shapefile attribute data stored in .dbf file acompanying the .shp.
Quite common problem for us in countries using extended character sets. Many free tools to translate encodings could not properly handle .dbf files or lack support for needed encodings.
It is possible to get around this by converting the shapefile (to .gml for example), translate encoding of this file and then convert it back to shapefile.
The second method involves adding the shapefiles to a File Geodatabase and then working with them, which often fixes the problem. The first step in this method is to remove the shapefile in question from any running ArcMap sessions and open ArcCatalog.
Manifold seem happy to receive reports of new or "unusual" formats (shapefiles have a bit of a reputation in this respect) so suggest you send an e-mail to sales explaining the apparent problem. Include details of the origin of the data, any knowledge of the environment it was created in etc and perhaps a link to this thread as a source for the problematic.
We have had this problem with the shp format also. In our case the culprit was entities with shape type 51. If i remember correctly, this was supposed to be for internal use, but somehow made it out into the wild. The open source tools added a handler a year or so back to treat a couple of types like this as legal shape types. It works for us, but there might be some information that gets lost by doing so.
If an ESRI product violates the shapefile standard, that's a bug in the ESRI product. If you want Manifold to read that broken format, that's a suggestion, not a bug. No resistance either way, but it is important in develpment to know when you are departing from a standard so users can be warned what is going on.
Ifthere is a departure from the standard in this case there might be an easy fix. A simple use of a tool like SED could correct the files so you can batch import them now. In contrast, if you stop the debugging process believing a bug is in play you might miss the easy fix.
The Shapefile Repair Tool, developed by Black Mountain Software PL, is a software solution designed to resolve issues with shapefiles used in GIS applications. The tool was created to help users repair errors and inconsistencies within shapefiles quickly and efficiently.
The Shapefile Repair Tool is a tool for GIS analysts, cartographers, and data scientists, among others. It processes shapefiles by identifying broken or invalid geometries. It can repair specific issues or rebuild an entire shapefile from scratch.
The tool includes features such as identifying and deleting duplicate shapes, repairing broken geometries, and simplifying complex geometries. Data analysis becomes more efficient through the use of this tool because it identifies and removes discrepancies in shapefiles that can result in incorrect analyses.
The Shapefile Repair Tool is beneficial to all GIS analysts who need to ensure that their data is correct and reliable. It saves time and reduces frustration by fixing issues automatically while maintaining the original structure of the shapefile.
I have had the same problem and not found a good fix yet. I think the problem lies in that the geometry of the shapefile is in a PointZM format (which includes elevation in meters) rather than strictly a lat-lon format.
QGIS seems to choke on this but there is bound to be a solution easier than what I came up with: to import the shapefile into postgis, then convert the geometry to Point type. You lose elevation values so not a great fix.
Bonus question: I'm using shapefiles to create flat objects on an OpenStreetMap backdrop. I use vector > geoprocessing tools > buffer to create contour lines around these objects. For every line I have to make it make permanent and fix the symbology. Is there a more efficient/effective way to do this?
Ok. I Always solved this issue using GRASS v.clean tool on QGIS processing toolbox. Though yesterday I spent a few hours to fix a shapefile with invalid geometry. In my case, I obtained a vector from raster data. Then I was trying to use this vector as a mask to extract data from another raster.
Once the GRASS database is created and the project is accessed, use r.in.gdal (click on File -> Import raster data -> Import of common raster formats) to import the raster you wanna clip and v.in.ogr (click on File -> Import vector data -> Import of common vector formats) to import the shapefile you will use as a mask.
Use r.mask (click on Raster -> Mask) to create a mask file. Please note that you will need to input only your previously imported vector file on the Vector tab of the [r.mask] tool. The Raster tab of the [r.mask] tool must be empty.
After creating your Mask file, check on the Display window if the previously imported raster is shown only on the areas covered by the shapefile that was also previously imported (like the figure above). After all, this shapefile is now your mask.
As long as your .shp is in order, this program will re-generate your .shx which is the shapefile index (not the spatial index). Also, if you have too many or too few records in the dbf, it will add or delete records to match the shapefile. Thus, you can completely fix this problem automatically..
I'm trying to create a pipe network from an ESRI Shapefile in Model Builder but when I import it some pipes are mising, I imported the shapefile as a background to see if the pipes were where they are supposed to be and everything looks fine.
Also I forgot to add that I exported the shapefile to CAD and then imported it to WaterGEMS using the Model Builder and everythings looks fine but I need to import it from the ESRI Shapefile because I have the diameter, Hazen-Williams C, and other information as attributes in that file.
When you are importing shapefile using Modelbuilder, after completing modelbuilder run, what errors do you see in modelbuilder summary message tab? That would give clue about what is missing in the shapefile.
3) If the key field is unique and the shape type is a simple polyline, there may be something corrupt with the Shapefile. If you have access to ArcGIS< you can try running the "repair geometry" tool in ArcToolbox. Typically if this was the problem though, you would see some indication in the summary at the end of the ModelBuilder run that Sushma mentioned.
I'm trying to run a buffer on a large polyline shapefile so that I can use a spatial match to pick up point locations within the buffer zones, but when I run the workflow I get error - Point is out of range in Polyline.
The shapefile was created using the Spatial Process tool (intersection), which created a shapefile with different spatial object types - Lines and PolyPolyLines. As mixed spatial object types apparently cause the aforementioned error message, I used a Poly-Split and Poly-Build tools to split the PolyPolyLines into points and rebuild them back into Lines.
However when put back into the original workflow I was given the same error message. Looking into this further it appeared that shapefile created using the Spatial Process tool contained some damaged geometry. At the moment it's unclear why this happened but the Alteryx support team are currently looking into it.
Both the Esri and XBase file-formats are very simple in design and memoryefficient which is part of the reason the shapefile format remains populardespite the numerous ways to store and exchange GIS data available today.
Currently the sample census blockgroup shapefile referenced in the examples is available on the GitHub project site at Theseexamples are straight-forward and you can also easily run them against yourown shapefiles with minimal modification.
To read a shapefile create a new "Reader" object and pass it the name of anexisting shapefile. The shapefile format is actually a collection of threefiles. You specify the base filename of the shapefile or the complete filenameof any of the shapefile component files.
OR any of the other 5+ formats which are potentially part of a shapefile. Thelibrary does not care about file extensions. You can also specify that you onlywant to read some of the file extensions through the use of keyword arguments:
You can also load shapefiles from any Python file-like object using keywordarguments to specify any of the three files. This feature is very powerful andallows you to custom load shapefiles from arbitrary storage formats, such as a protected url or zip file, a serialized object, or in some cases a database.
aa06259810