Store Zip File In Database Using Java
Giovanna Qiu
I am working around my project and got some Problem, I searched for it , but couldn't find meaning full learning resource, What I need is to Store Image in SQL Server Database using my Java program , and will need that Back to Retrieve, images are not of larger size they are ranging between 30 and 50 K,I can load images from my Disk by using getImage() method in toolKit
, but i don't know How to convert that image to Binary Format and to store in Database, and then retrieve Back From Database.I want to Store that in VarBinary s by looking around several sites i found that the image type in SQL Server is soon going to Retire.
While it's not a good idea to store very large binary objects in the database, the Microsoft research paper "To BLOB or Not To BLOB: Large Object Storage in a Database or a Filesystem" indicates it's an efficient approach if object sizes are less than 256K. So it sounds like you've hit the sweet spot for database storage with 30K images.
Then store the byte array as recommended by this article as a VARBINARY in the database. In your JDBC code, you should wrap the byte array in a ByteArrayInputStream and set the PreparedStatement parameter as a BinaryStream:
It is generally not a very good idea to put into the database large resources like images, unless you really need ACID compliance when working with them. Database dumps and time required to produce them grow tremendously and you will need to use the more complex and fragile incremental backups.
[Note that I am not considering serialization as-is for this purpose, as it won't allow to retrieve arbitrary objects in the middle of an object graph. Neither am I considering DB4O because of its restrictive license. Thanks.]
"Easy" meaning: not having to handle low-level details such as key/value pairs to rebuild an object graph (as with BerkeleyDB or traditional caches). The same applies for rebuilding objects from a document- or column-oriented DB (CouchDB, HBase, ..., even Lucene).
I would suggest Hibernate because it will deal with most of the ugly details that bog developers down when using a database while still allowing for the optimizations that have been made to database software over the years.
Check out comments on Prevayler on this question. Prevayler is a transactional wrapper around object serialization - roughly, use objects in plain java and persist to disk through java API w/o sql, a bit neater than writing your own serialization.
Caveats- with serialization as a persistance mechanism, you run the risk of invalidating your saved data when you update the class. Even with a wrapper library you'll probably want to customize the serialization/deserialization handling. It also helps to include the serialVersionUID in the class so you override the JVM's idea of when the class is updated (and therefore can't reload your saved serialized data).
Hmm... without serialization, and without an ORM solution, I would fall back to some sort of XML based implementation? You'd still have to design it carefully if you want to pull out only some of the objects from the object graph - perhaps a different file for each object, where object relationships are referenced by a URI to another file?
I would have said that wasn't "easy" because I've always found designing the mapping of XML to objects to be somewhat time consuming, but I was really inspired by a conversation on Apache Betwixt that has me feeling hopeful that I'm just out of date, and easier solutions are now available.
Terracotta provides a highly available, highly scalable persistent to disk object store. You can use it for just this feature alone - or you can use it's breadth of features to implement a fully clustered application - your choice.
Here's a case study about how gnip uses Terracotta for in-memory persistence - no database. Gnip takes in all of the events on Facebook, Twitter, and the like and produces them for consumers in a normalized fashion. Their current solution is processing in excess of 50,000 messages / second.
CouchDB seems to fit the bill. It still could act as a key-value store but its great querying capabilities (map/reduce, view collations), concurrency readiness and language-agnostic HTTP access makes it my choice.
Only glitch is having to correclty define and map JSON structures to objects, but I'm confident I will come up with a simple solution for usage with relational models from Java and Scala (and worry about caching later on, as contention is moved away from the database). Terracotta could still be useful but certainly not as with an RDBMS scenario.
I'm trying to build a Java application that connects to a remote database that stores and retrieves video files that can be over 200MB. I could store/retrieve these files in MySQL directly as LONGBLOBs, yet I read again and again how that's bad practice for various reasons. One solution commonly offered is to store the files on the filesystem and have the database store the location file and serve it up from the server directly.
My problem is, I'm very new to storage and am completely unsure of how to go about doing that. Should I make an FTP connection to the front end once a video file has been requested, then deliver that video over that connection? Or are people usually talking about something else when they say to pull it from the filesystem?
With files being stored on the filesystem you need to make sure they are in a directory which is publicly available (should it be, of course). You then serve these files directly from the filesystem from their path.
Let's say your application is located at /home/dyslexit/www, where www is a publicly available directory from the browser. Within the www directory you would create a new one, eg. uploads, which would also be publicly available and used to store the files. Should you then serve some file to a client of the application it would be as easy as providing them a URI: -video.mp4, which would translate to the actual /home/dyslexit/www/uploads/videos/my-video.mp4 file on the server.
It is however not required to directly copy the structure. Should you need it instead of directly serving the files from a HDD when a file is requested you could instead forward the request to a script checking permissions of the requesting user, denying them access should they not be authorized to access a resource or forwarding them to the file being located anywhere respectively.
1 Obviously this can also be implemented if you store files in a database but requires additional setup and custom header manipulation. When serving files directly from the file system this is mostly native.
Either way, if your business goal is not actually storing files - such as your are creating a data-storage system - but it is merely a side-effect of the business (let's say you want videos in a Facebook-like application), the better choice would probably be to completely outsource file storage to a 3rd party system dedicated to this feature - such as Amazon S3.
In applications I've coded before, I used databases to store data, like user profiles, settings, etc. Now, obviously, I can't use a database to store data generated from this school project, otherwise what's the point?
I'm thinking about storing the data in files but that's the only idea I have in my mind right now and I'm kinda running dry.. and to be honest, I don't want to start banging at code and then I discover a better way of doing it.
--EDIT: just to be more clear, I can't use database engines to store the data, to put it this way, I'm coding a simple database engine. Ideas like what Galwegian, jkramer and Joe Skora suggested is what I'm looking for.
Sure, you could create your own database with a file system since that is how actual databases are implemented. For example, you could decide to store your data in fixed or variable length raw data files, and then create a separate index file with file pointers into that other file for quick indexed access for any queries based on what type of index information you want stored in your Index file
What you probably want is to use are random access files. Once you have a set of fields for a record, you can write them to disk as a block. You can keep an index separately on disk on in memory and access any record directly at any time. Hopefully that gives you enough to get started.
I would create a database that uses binary tables, one file per table. Take a look at the very handy DataInputStream and DataOutputStream classes. Using them you can easily go back and forth from binary files to Java types.
I would define a simple structure for the table: a header that describes the contents of the table, followed by the row data. Have each column in the table defined in the header - its name, data type, and maximum length. Keep it simple. Only handle a few data types using the capabilities of DataInput/OutputStream as your guide. Use a simple file-naming convention to associate table names to file names.
Create a test table with enough columns to have at least one of each data type. Then, create a simple way to populate tables with data, either by processing input files or via console input. Finally, create a simple way to display the contents of entire tables to the console.
If it's not necessary to have such a general-purpose solution (any number of tables, any number of columns, an actual query language, etc.) you can simply add methods to your API like:
The basics of storing records in blocks in data files have been around for decades. Obviously there are a great many variations on a theme, and all of them are designed to work around the fact that we have slow disk drives.
Indexes are stored in B+-Trees, with tree nodes distributed across blocks on the disk. The power of the B+Tree is that you can easily find a single key value within the tree, and then simply walk the leaf nodes to scroll through the data in key order.
For a simple format that's useful and popular, consider looking up the original DBase format -- DBF Files. It's evolved some over the years, but the foundation is quite simple, well documented, and there are lots of utilities that can work on it. It's a perfectly workable database format that deals with all of the fundamental issues with the problem.
7fc3f7cf58