SQLite

SQLite is an open source, multiplatform, free and small C library that implements a self-contained, embeddable, transactional, serverless and zero-configuration SQL database engine. It is the world’s most used SQL database engine.

Features at a glance

Key features include support for isolated, atomic,durable and consistent transactions, SQL92 implementation, database files can be freely shared between computers, supports databases up to 2 terabytes, support for gigabyte-sized blobs and strings, small code footprint, as well as an easy-to-use and very simple API (Application Programming Interface).

Additionally, SQLite does not require initial configuration or further administration, allows developers to store a complete database in a single, cross-platform disk file, which is perfect for using application file formats, has no external dependencies, comes with built-in TCL (Tool Command Language) bindings, and comprehensive documentation.

Bindings for numerous other programming languages are available separately. Its source code is well-commented and it comes with a standalone CLI (Command-line Interface) client that has been designed from the offset to be used for the administration of SQLite databases.

What can I use it for?

The SQLite database engine can be used for any purpose, personal or commercial. Suggested uses for SQLite include database for gadgets, website database, stand-in for an enterprise RDBMS (Relational Database Management System), as well as application file format.

Under the hood and supported OSes

The SQLite distribution comes with a standalone command-line access program (sqlite) that can be used to administer an SQLite database and which serves as an example of how to use the SQLite library. It is written entirely in the ANSI-C programming language.

Supported desktop operating systems include GNU/Linux, Mac OS X and Microsoft Windows. Supported mobile operating systems include Android and iOS. It has been successfully tested with both 32-bit and 64-bit hardware platforms, and it’s easily portable to other OSes.

What is new in this release:

• Highlights of this release include support for PostgreSQL-style UPSERT and improved performance, especially for ORDER BY LIMIT queries.

What is new in version :

• Take advantage of the atomic-write capabilities in the F2FS filesystem when available, for greatly reduced transaction overhead. This currently requires the SQLITE_ENABLE_BATCH_ATOMIC_WRITE compile-time option.
• Allow ATTACH and DETACH commands to work inside of a transaction.
• Allow WITHOUT ROWID virtual tables to be writable if the PRIMARY KEY contains exactly one column.
• The "fsync()" that occurs after the header is written in a WAL reset now uses the sync settings for checkpoints. This means it will use a "fullfsync" on macs if PRAGMA checkpoint_fullfsync set on.
• The sqlite3_sourceid() function tries to detect if the source code has been modified from what is checked into version control and if there are modifications, the last four characters of the version hash are shown as "alt1" or "alt2". The objective is to detect accidental and/or careless edits. A forger can subvert this feature.
• Improved de-quoting of column names for CREATE TABLE AS statements with an aggregate query on the right-hand side.
• Fewer "stat()" system calls issued by the unix VFS.
• Enhanced the LIKE optimization so that it works with an ESCAPE clause.
• Enhanced PRAGMA integrity_check and PRAGMA quick_check to detect obscure row corruption that they were formerly missing. Also update both pragmas so that they return error text rather than SQLITE_CORRUPT when encountering corruption in records.
• The query planner now prefers to implement FROM-clause subqueries using co-routines rather using the query flattener optimization. Support for the use of co-routines for subqueries may no longer be disabled.
• Pass information about !=, IS, IS NOT, NOT NULL, and IS NULL constraints into the xBestIndex method of virtual tables.
• Enhanced the CSV virtual table so that it accepts the last row of input if the final new-line character is missing.
• Remove the rarely-used "scratch" memory allocator. Replace it with the SQLITE_CONFIG_SMALL_MALLOC configuration setting that gives SQLite a hint that large memory allocations should be avoided when possible.
• Added the swarm virtual table to the existing union virtual table extension.
• Added the sqlite_dbpage virtual table for providing direct access to pages of the database file. The source code is built into the amalgamation and is activated using the -DSQLITE_ENABLE_DBPAGE_VTAB compile-time option.
• Add a new type of fts5vocab virtual table - "instance" - that provides direct access to an FTS5 full-text index at the lowest possible level.
• Remove a call to rand_s() in the Windows VFS since it was causing problems in Firefox on some older laptops.
• The src/shell.c source code to the command-line shell is no longer under version control. That file is now generated as part of the build process.
• Miscellaneous microoptimizations reduce CPU usage by about 2.1%.
• Bug fixes:
• Fix a faulty assert() statement discovered by OSSFuzz. Ticket cb91bf4290c211d
• Fix an obscure memory leak in sqlite3_result_pointer(). Ticket 7486aa54b968e9b
• Avoid a possible use-after-free error by deferring schema resets until after the query planner has finished running. Ticket be436a7f4587ce5
• Only use indexes-on-expressions to optimize ORDER BY or GROUP BY if the COLLATE is correct. Ticket e20dd54ab0e4383
• Fix an assertion fault that was coming up when the expression in an index-on-expressions is really a constant. Ticket aa98619ad08ddca
• Fix an assertion fault that could occur following PRAGMA reverse_unordered_selects. Ticket cb91bf4290c211d
• Fix a segfault that can occur for queries that use table-valued functions in an IN or EXISTS subquery. Ticket b899b6042f97f5
• Fix a potential integer overflow problem when compiling a particular horrendous common table expression. This was another problem discovered by OSSFuzz. Check-in 6ee8cb6ae5.
• Fix a potential out-of-bound read when querying a corrupt database file, a problem detected by Natalie Silvanovich of Google Project Zero. Check-in 04925dee41a21f.

What is new in version 3.20.1:

• The version 3.20.1 patch release changes two lines of code in the sqlite3_result_pointer() interface in order to fix a rare memory leak.

What is new in version 3.9.2:

• SQLite version 3.9.2 is a patch release fixing two obscure bugs.

What is new in version 3.8.9:

• New features in this release include the PRAGMA index_xinfo command, the sqlite3_status64() interface, and the ".dbinfo" command of the command-line shell.

What is new in version 3.8.8.2:

• The 3.8.8.2 patch release fixes a single minor problem: It ensures that the sqlite3_wal_checkpoint(TRUNCATE) operation will always truncate the write-ahead log even if log had already been reset and contained no new content. It is unclear if this is a bug fix or a new feature.
• Something like this would normally go into the next regularly scheduled release, but a prominent SQLite user needed the change in a hurry so we were happy to rush it out via this patch.
• There is no reason to upgrade unless you actually need the enhanced behavior of sqlite3_wal_checkpoint(TRUNCATE).

What is new in version 3.8.8.1:

• Fix a bug in the sorting logic, present since version 3.8.4, that can cause output to appear in the wrong order on queries that contains an ORDER BY clause, a LIMIT clause, and that have approximately 60 or more columns in the result set. Ticket f97c4637102a3ae72b79.
• SQLITE_SOURCE_ID: "2015-01-20 16:51:25 f73337e3e289915a76ca96e7a05a1a8d4e890d55"
• SHA1 for sqlite3.c: 33987fb50dcc09f1429a653d6b47672f5a96f19e

What is new in version 3.8.8:

• New Features:
• Added the PRAGMA data_version command that can be used to determine if a database file has been modified by another process.
• Added the SQLITE_CHECKPOINT_TRUNCATE option to the sqlite3_wal_checkpoint_v2() interface, with corresponding enhancements to PRAGMA wal_checkpoint.
• Added the sqlite3_stmt_scanstatus() interface, available only when compiled with SQLITE_ENABLE_STMT_SCANSTATUS.
• The sqlite3_table_column_metadata() is enhanced to work correctly on WITHOUT ROWID tables and to check for the existence of a a table if the column name parameter is NULL. The interface is now also included in the build by default, without requiring the SQLITE_ENABLE_COLUMN_METADATA compile-time option.
• Added the SQLITE_ENABLE_API_ARMOR compile-time option.
• Added the SQLITE_REVERSE_UNORDERED_SELECTS compile-time option.
• Added the SQLITE_SORTER_PMASZ compile-time option and SQLITE_CONFIG_PMASZ start-time option.
• Added the SQLITE_CONFIG_PCACHE_HDRSZ option to sqlite3_config() which makes it easier for applications to determine the appropriate amount of memory for use with SQLITE_CONFIG_PAGECACHE.
• The number of rows in a VALUES clause is no longer limited by SQLITE_LIMIT_COMPOUND_SELECT.
• Added the eval.c loadable extension that implements an eval() SQL function that will recursively evaluate SQL.
• Performance Enhancements:
• Reduce the number of memcpy() operations involved in balancing a b-tree, for 3.2% overall performance boost.
• Improvements to cost estimates for the skip-scan optimization.
• The automatic indexing optimization is now capable of generating a partial index if that is appropriate.
• Bug fixes:
• Ensure durability following a power loss with "PRAGMA journal_mode=TRUNCATE" by calling fsync() right after truncating the journal file.
• The query planner now recognizes that any column in the right-hand table of a LEFT JOIN can be NULL, even if that column has a NOT NULL constraint. Avoid trying to optimize out NULL tests in those cases. Fix for ticket 6f2222d550f5b0ee7ed.
• Make sure ORDER BY puts rows in ascending order even if the DISTINCT operator is implemented using a descending index. Fix for ticket c5ea805691bfc4204b1cb9e.
• Fix data races that might occur under stress when running with many threads in shared cache mode where some of the threads are opening and closing connections.
• Fix obscure crash bugs found by american fuzzy lop. Ticket a59ae93ee990a55.
• Work around a GCC optimizer bug (for gcc 4.2.1 on MacOS 10.7) that caused the R-Tree extension to compute incorrect results when compiled with -O3.
• Other changes:
• Disable the use of the strchrnul() C-library routine unless it is specifically enabled using the -DHAVE_STRCHRNULL compile-time option.
• Improvements to the effectiveness and accuracy of the likelihood(), likely(), and unlikely() SQL hint functions.
• SQLITE_SOURCE_ID: "2015-01-16 12:08:06 7d68a42face3ab14ed88407d4331872f5b243fdf"
• SHA1 for sqlite3.c: 91aea4cc722371d58aae3d22e94d2a4165276905

What is new in version 3.8.7.4:

• This release fixes adds in a mutex that is required by the changes of the 3.8.7.3 patch but was accidently omitted. The mutex was not required by any of the internal SQLite tests, but Firefox crashes without it. Test cases have been added to ensure that mutex is never again missed.

What is new in version 3.8.7.3:

• Bug fix: Ensure the cached KeyInfo objects (an internal abstraction not visible to the application) do not go stale when operating in shared cache mode and frequently closing and reopening some database connections while leaving other database connections on the same shared cache open continuously. Ticket e4a18565a36884b00edf.
• Bug fix: Recognize that any column in the right-hand table of a LEFT JOIN can be NULL even if the column has a NOT NULL constraint. Do not apply optimizations that assume the column is never NULL. Ticket 6f2222d550f5b0ee7ed.
• SQLITE_SOURCE_ID: "2014-12-05 22:29:24 647e77e853e81a5effeb4c33477910400a67ba86"
• SHA1 for sqlite3.c: 3ad2f5ba3a4a3e3e51a1dac9fda9224b359f0261

What is new in version 3.8.7.2:

• The primary reason for this release is to enhance the ROLLBACK command so that it allows running queries on the same database connection to continue running as long as the ROLLBACK does not change the schema. In all previous versions of SQLite, a ROLLBACK would cause pending queries to stop immediately and return SQLITE_ABORT or SQLITE_ABORT_ROLLBACK. Pending queries still abort if the ROLLBACK changes the database schema, but as of this patch release, the queries are allowed to continue running if the schema is unmodified.
• In addition to the ROLLBACK enhancement, this patch release also includes fixes for three obscure bugs.

What is new in version 3.8.7.1:

• The primary reason for this bug-fix release is to address a problem with updating the value of fields at the end of a table that were added using ALTER TABLE ADD COLUMN. This problem 1 first appeared in the 3.8.7 release.
• Another minor annoyance in the 3.8.7 release was the fact that the Android build tried to use the strchrnul() function from the standard C library but that function is not available on Android. Android builds had to add -DHAVE_STRCHRNUL=0 to work around the problem. This patch fixes that so that Android builds should now work without any changes.
• The operation of PRAGMA journal_mode=TRUNCATE has been enhanced so that it invokes fsync() after truncating the journal file when PRAGMA synchronous=FULL. This helps to preserve transaction durability in the case of a power loss occurring shortly after commit.
• Finally, a couple of long-standing and obscure problems associated with run UPDATE and DELETE on VIEWs were fixed.

What is new in version 3.8.7:

• Most of the changes from the previous release have been micro-optimizations designed to help SQLite run a little faster. Each individual optimization has an unmeasurably small performance impact. But the improvements add up. Measured on a well-defined workload (which the SQLite developers use as a proxy for a typical application workload) using cachegrind on Linux and compiled with gcc 4.8.1 and -Os on x64 linux, the current release does over 20% more work for the same number of CPU cycles compared to the previous release. Cachegrind is not a real CPU, and the workload used for measurement is only a proxy. So your performance may vary. We expect to see about half the measured and reported improvement in real-world applications. 10% is less than 20% but it is still pretty good, we think.
• This release includes a new set of C-language interfaces that have unsigned 64-bit instead of signed 32-bit length parameters. The new APIs do not provide any new capabilities. But they do make it easier to write applications that are more resistant to integer overflow vulnerabilities.
• This release also includes a new sorter that is able to use multiple threads to help with large sort operations. (Sort operations are sometimes required to implement ORDER BY and/or GROUP BY clauses and are almost always required for CREATE INDEX.) The multi-threads sorter is turned off by default and must be enabled using the PRAGMA threads SQL command. Note that the multi-threaded sorter provides faster real-time performance for large sorts, but it also uses more CPU cycles and more energy.

What is new in version 3.8.3.1:

• SQLite version 3.8.3.1 fixes a bug present in versions 3.8.1, 3.8.2 and 3.8.3 that can cause queries to omit valid out rows. Upgrading from those versions is recommended.
• The problem only comes up if SQLite is compiled with either the SQLITE_ENABLE_STAT3 or SQLITE_ENABLE_STAT4 compile-time options. In that case, if a query has a WHERE clause that contains expressions like this:
• WHERE (expr1 OR expr2 OR ... OR exprN) AND column IS NOT NULL
• Where all of expr1 through exprN are suitable for use by indexes, then during query planning SQLite might mistakenly converted the "column IS NOT NULL" term into "column>NULL". But the latter term is never true, and so the query would return no rows.

What is new in version 3.8.3:

• Added support for common table expressions and the WITH clause.
• Added the printf() SQL function.
• Added SQLITE_DETERMINISTIC as an optional bit in the 4th argument to the sqlite3_create_function() and related interfaces, providing applications with the ability to create new functions that can be factored out of inner loops when they have constant arguments.
• Add SQLITE_READONLY_DBMOVED error code, returned at the beginning of a transaction, to indicate that the underlying database file has been renamed or moved out from under SQLite.
• Allow arbitrary expressions, including function calls and subqueries, in the filename argument to ATTACH.
• Allow a VALUES clause to be used anywhere a SELECT statement is valid.
• Reseed the PRNG used by sqlite3_randomness(N,P) when invoked with N==0. Automatically reseed after a fork() on unix.
• Enhance the spellfix1 virtual table so that it can search efficiently by rowid.
• Performance enhancements.
• Improvements to the comments in the VDBE byte-code display when running EXPLAIN.
• Add the "%token_class" directive to LEMON parser generator and use it to simplify the grammar.
• Change the LEMON source code to avoid calling C-library functions that OpenBSD considers dangerous. (Ex: sprintf).
• Bug fix: In the command-line shell CSV import feature, do not end a field when an escaped double-quote occurs at the end of a CRLN line.
• SQLITE_SOURCE_ID: "2014-02-03 13:52:03 e816dd924619db5f766de6df74ea2194f3e3b538"
• SHA1 for sqlite3.c: 98a07da78f71b0275e8d9c510486877adc31dbee