1.3 Changelog

1.3.25

no release date

orm

  • [orm] [bug]

    Fixed issue in Session.bulk_save_objects() when used with persistent objects which would fail to track the primary key of mappings where the column name of the primary key were different than the attribute name.

    References: #6392

schema

  • [schema] [bug]

    The Table object now raises an informative error message if it is instantiated without passing at least the Table.name and Table.metadata arguments positionally. Previously, if these were passed as keyword arguments, the object would silently fail to initialize correctly.

    References: #6135

postgresql

  • [postgresql] [bug] [regression]

    Fixed regression caused by #6023 where the PostgreSQL cast operator applied to elements within an ARRAY when using psycopg2 would fail to use the correct type in the case that the datatype were also embedded within an instance of the Variant adapter.

    Additionally, repairs support for the correct CREATE TYPE to be emitted when using a Variant(ARRAY(some_schema_type)).

    References: #6182

mysql

  • [mysql] [bug] [mariadb]

    Fixes to accommodate for the MariaDB 10.6 series, including backwards incompatible changes in both the mariadb-connector Python driver (supported on SQLAlchemy 1.4 only) as well as the native 10.6 client libraries that are used automatically by the mysqlclient DBAPI (applies to both 1.3 and 1.4). The “utf8mb3” encoding symbol is now reported by these client libraries when the encoding is stated as “utf8”, leading to lookup and encoding errors within the MySQL dialect that does not expect this symbol. Updates to both the MySQL base library to accommodate for this utf8mb3 symbol being reported as well as to the test suite. Thanks to Georg Richter for support.

    References: #7115, #7136

sqlite

  • [sqlite] [bug]

    Add note regarding encryption-related pragmas for pysqlcipher passed in the url.

    References: #6589

1.3.24

Released: March 30, 2021

orm

  • [orm] [bug]

    Removed very old warning that states that passive_deletes is not intended for many-to-one relationships. While it is likely that in many cases placing this parameter on a many-to-one relationship is not what was intended, there are use cases where delete cascade may want to be disallowed following from such a relationship.

    References: #5983

  • [orm] [bug]

    Fixed issue where the process of joining two tables could fail if one of the tables had an unrelated, unresolvable foreign key constraint which would raise NoReferenceError within the join process, which nonetheless could be bypassed to allow the join to complete. The logic which tested the exception for significance within the process would make assumptions about the construct which would fail.

    References: #5952

  • [orm] [bug]

    Fixed issue where the MutableComposite construct could be placed into an invalid state when the parent object was already loaded, and then covered by a subsequent query, due to the composite properties’ refresh handler replacing the object with a new one not handled by the mutable extension.

    References: #6001

engine

  • [engine] [bug]

    Fixed bug where the “schema_translate_map” feature failed to be taken into account for the use case of direct execution of DefaultGenerator objects such as sequences, which included the case where they were “pre-executed” in order to generate primary key values when implicit_returning was disabled.

    References: #5929

schema

  • [schema] [bug]

    Fixed bug first introduced in as some combination of #2892, #2919 nnd #3832 where the attachment events for a TypeDecorator would be doubled up against the “impl” class, if the “impl” were also a SchemaType. The real-world case is any TypeDecorator against Enum or Boolean would get a doubled CheckConstraint when the create_constraint=True flag is set.

    References: #6152

  • [schema] [bug] [sqlite]

    Fixed issue where the CHECK constraint generated by Boolean or Enum would fail to render the naming convention correctly after the first compilation, due to an unintended change of state within the name given to the constraint. This issue was first introduced in 0.9 in the fix for issue #3067, and the fix revises the approach taken at that time which appears to have been more involved than what was needed.

    References: #6007

  • [schema] [bug]

    Repaired / implemented support for primary key constraint naming conventions that use column names/keys/etc as part of the convention. In particular, this includes that the PrimaryKeyConstraint object that’s automatically associated with a Table will update its name as new primary key Column objects are added to the table and then to the constraint. Internal failure modes related to this constraint construction process including no columns present, no name present or blank name present are now accommodated.

    References: #5919

  • [schema] [bug]

    Adjusted the logic that emits DROP statements for Sequence objects among the dropping of multiple tables, such that all Sequence objects are dropped after all tables, even if the given Sequence is related only to a Table object and not directly to the overall MetaData object. The use case supports the same Sequence being associated with more than one Table at a time.

    References: #6071

postgresql

  • [postgresql] [bug]

    Fixed issue where using aggregate_order_by would return ARRAY(NullType) under certain conditions, interfering with the ability of the result object to return data correctly.

    References: #5989

  • [postgresql] [bug] [reflection]

    Fixed issue in PostgreSQL reflection where a column expressing “NOT NULL” will supersede the nullability of a corresponding domain.

    References: #6161

  • [postgresql] [bug] [types]

    Adjusted the psycopg2 dialect to emit an explicit PostgreSQL-style cast for bound parameters that contain ARRAY elements. This allows the full range of datatypes to function correctly within arrays. The asyncpg dialect already generated these internal casts in the final statement. This also includes support for array slice updates as well as the PostgreSQL-specific ARRAY.contains() method.

    References: #6023

mssql

  • [mssql] [bug] [reflection]

    Fixed issue regarding SQL Server reflection for older SQL Server 2005 version, a call to sp_columns would not proceed correctly without being prefixed with the EXEC keyword. This method is not used in current 1.4 series.

    References: #5921

1.3.23

Released: February 1, 2021

sql

postgresql

  • [postgresql] [bug]

    For SQLAlchemy 1.3 only, setup.py pins pg8000 to a version lower than 1.16.6. Version 1.16.6 and above is supported by SQLAlchemy 1.4. Pull request courtesy Giuseppe Lumia.

    References: #5645

  • [postgresql] [bug]

    Fixed issue where using Table.to_metadata() (called Table.tometadata() in 1.3) in conjunction with a PostgreSQL ExcludeConstraint that made use of ad-hoc column expressions would fail to copy correctly.

    References: #5850

mysql

  • [mysql] [usecase]

    Casting to FLOAT is now supported in MySQL >= (8, 0, 17) and MariaDb >= (10, 4, 5).

    References: #5808

  • [mysql] [bug] [reflection]

    Fixed bug where MySQL server default reflection would fail for numeric values with a negation symbol present.

    References: #5860

  • [mysql] [bug]

    Fixed long-lived bug in MySQL dialect where the maximum identifier length of 255 was too long for names of all types of constraints, not just indexes, all of which have a size limit of 64. As metadata naming conventions can create too-long names in this area, apply the limit to the identifier generator within the DDL compiler.

    References: #5898

  • [mysql] [bug]

    Fixed deprecation warnings that arose as a result of the release of PyMySQL 1.0, including deprecation warnings for the “db” and “passwd” parameters now replaced with “database” and “password”.

    References: #5821

  • [mysql] [bug]

    Fixed regression from SQLAlchemy 1.3.20 caused by the fix for #5462 which adds double-parenthesis for MySQL functional expressions in indexes, as is required by the backend, this inadvertently extended to include arbitrary text() expressions as well as Alembic’s internal textual component, which are required by Alembic for arbitrary index expressions which don’t imply double parenthesis. The check has been narrowed to include only binary/ unary/functional expressions directly.

    References: #5800

oracle

  • [oracle] [bug]

    Fixed regression in Oracle dialect introduced by #4894 in SQLAlchemy 1.3.11 where use of a SQL expression in RETURNING for an UPDATE would fail to compile, due to a check for “server_default” when an arbitrary SQL expression is not a column.

    References: #5813

  • [oracle] [bug]

    Fixed bug in Oracle dialect where retrieving a CLOB/BLOB column via Insert.returning() would fail as the LOB value would need to be read when returned; additionally, repaired support for retrieval of Unicode values via RETURNING under Python 2.

    References: #5812

misc

  • [bug] [ext]

    Fixed issue where the stringification that is sometimes called when attempting to generate the “key” for the .c collection on a selectable would fail if the column were an unlabeled custom SQL construct using the sqlalchemy.ext.compiler extension, and did not provide a default compilation form; while this seems like an unusual case, it can get invoked for some ORM scenarios such as when the expression is used in an “order by” in combination with joined eager loading. The issue is that the lack of a default compiler function was raising CompileError and not UnsupportedCompilationError.

    References: #5836

1.3.22

Released: December 18, 2020

oracle

  • [oracle] [bug]

    Fixed regression which occurred due to #5755 which implemented isolation level support for Oracle. It has been reported that many Oracle accounts don’t actually have permission to query the v$transaction view so this feature has been altered to gracefully fallback when it fails upon database connect, where the dialect will assume “READ COMMITTED” is the default isolation level as was the case prior to SQLAlchemy 1.3.21. However, explicit use of the Connection.get_isolation_level() method must now necessarily raise an exception, as Oracle databases with this restriction explicitly disallow the user from reading the current isolation level.

    References: #5784

1.3.21

Released: December 17, 2020

orm

  • [orm] [bug]

    Added a comprehensive check and an informative error message for the case where a mapped class, or a string mapped class name, is passed to relationship.secondary. This is an extremely common error which warrants a clear message.

    Additionally, added a new rule to the class registry resolution such that with regards to the relationship.secondary parameter, if a mapped class and its table are of the identical string name, the Table will be favored when resolving this parameter. In all other cases, the class continues to be favored if a class and table share the identical name.

    References: #5774

  • [orm] [bug]

    Fixed bug in Query.update() where objects in the _ormsession.Session that were already expired would be unnecessarily SELECTed individually when they were refreshed by the “evaluate”synchronize strategy.

    References: #5664

  • [orm] [bug]

    Fixed bug involving the restore_load_context option of ORM events such as InstanceEvents.load() such that the flag would not be carried along to subclasses which were mapped after the event handler were first established.

    References: #5737

sql

  • [sql] [bug]

    A warning is emitted if a returning() method such as Insert.returning() is called multiple times, as this does not yet support additive operation. Version 1.4 will support additive operation for this. Additionally, any combination of the Insert.returning() and ValuesBase.return_defaults() methods now raises an error as these methods are mutually exclusive; previously the operation would fail silently.

    References: #5691

  • [sql] [bug]

    Fixed structural compiler issue where some constructs such as MySQL / PostgreSQL “on conflict / on duplicate key” would rely upon the state of the Compiler object being fixed against their statement as the top level statement, which would fail in cases where those statements are branched from a different context, such as a DDL construct linked to a SQL statement.

    References: #5656

postgresql

  • [postgresql] [usecase]

    Added new parameter ExcludeConstraint.ops to the ExcludeConstraint object, to support operator class specification with this constraint. Pull request courtesy Alon Menczer.

    References: #5604

  • [postgresql] [bug] [mysql]

    Fixed regression introduced in 1.3.2 for the PostgreSQL dialect, also copied out to the MySQL dialect’s feature in 1.3.18, where usage of a non Table construct such as text() as the argument to Select.with_for_update.of would fail to be accommodated correctly within the PostgreSQL or MySQL compilers.

    References: #5729

mysql

  • [mysql] [bug] [reflection]

    Fixed issue where reflecting a server default on MariaDB only that contained a decimal point in the value would fail to be reflected correctly, leading towards a reflected table that lacked any server default.

    References: #5744

  • [mysql] [sql]

    Added missing keywords to the RESERVED_WORDS list for the MySQL dialect: action, level, mode, status, text, time. Pull request courtesy Oscar Batori.

    References: #5696

sqlite

  • [sqlite] [usecase]

    Added sqlite_with_rowid=False dialect keyword to enable creating tables as CREATE TABLE WITHOUT ROWID. Patch courtesy Sean Anderson.

    References: #5685

mssql

  • [mssql] [bug]

    Fixed bug where a CREATE INDEX statement was rendered incorrectly when both mssql-include and mssql_where were specified. Pull request courtesy @Adiorz.

    References: #5751

  • [mssql] [bug]

    Added SQL Server code “01000” to the list of disconnect codes.

    References: #5646

  • [mssql] [reflection] [sqlite]

    Fixed issue with composite primary key columns not being reported in the correct order. Patch courtesy @fulpm.

    References: #5661

oracle

1.3.20

Released: October 12, 2020

orm

  • [orm] [bug]

    An ArgumentError with more detail is now raised if the target parameter for Query.join() is set to an unmapped object. Prior to this change a less detailed AttributeError was raised. Pull request courtesy Ramon Williams.

    References: #4428

  • [orm] [bug]

    Fixed issue where using a loader option against a string attribute name that is not actually a mapped attribute, such as a plain Python descriptor, would raise an uninformative AttributeError; a descriptive error is now raised.

    References: #4589

engine

  • [engine] [bug]

    Fixed issue where a non-string object sent to SQLAlchemyError or a subclass, as occurs with some third party dialects, would fail to stringify correctly. Pull request courtesy Andrzej Bartosiński.

    References: #5599

  • [engine] [bug]

    Repaired a function-level import that was not using SQLAlchemy’s standard late-import system within the sqlalchemy.exc module.

    References: #5632

sql

  • [sql] [bug]

    Fixed issue where the pickle.dumps() operation against Over construct would produce a recursion overflow.

    References: #5644

  • [sql] [bug]

    Fixed bug where an error was not raised in the case where a column() were added to more than one table() at a time. This raised correctly for the Column and Table objects. An ArgumentError is now raised when this occurs.

    References: #5618

postgresql

  • [postgresql] [usecase]

    The psycopg2 dialect now support PostgreSQL multiple host connections, by passing host/port combinations to the query string. Pull request courtesy Ramon Williams.

    References: #4392

  • [postgresql] [bug]

    Adjusted the Comparator.any() and Comparator.all() methods to implement a straight “NOT” operation for negation, rather than negating the comparison operator.

    References: #5518

  • [postgresql] [bug]

    Fixed issue where the ENUM type would not consult the schema translate map when emitting a CREATE TYPE or DROP TYPE during the test to see if the type exists or not. Additionally, repaired an issue where if the same enum were encountered multiple times in a single DDL sequence, the “check” query would run repeatedly rather than relying upon a cached value.

    References: #5520

mysql

  • [mysql] [usecase]

    Adjusted the MySQL dialect to correctly parenthesize functional index expressions as accepted by MySQL 8. Pull request courtesy Ramon Williams.

    References: #5462

  • [mysql] [change]

    Add new MySQL reserved words: cube, lateral added in MySQL 8.0.1 and 8.0.14, respectively; this indicates that these terms will be quoted if used as table or column identifier names.

    References: #5539

  • [mysql] [bug]

    The “skip_locked” keyword used with with_for_update() will emit a warning when used on MariaDB backends, and will then be ignored. This is a deprecated behavior that will raise in SQLAlchemy 1.4, as an application that requests “skip locked” is looking for a non-blocking operation which is not available on those backends.

    References: #5568

  • [mysql] [bug]

    Fixed bug where an UPDATE statement against a JOIN using MySQL multi-table format would fail to include the table prefix for the target table if the statement had no WHERE clause, as only the WHERE clause were scanned to detect a “multi table update” at that particular point. The target is now also scanned if it’s a JOIN to get the leftmost table as the primary table and the additional entries as additional FROM entries.

    References: #5617

mssql

  • [mssql] [bug]

    Fixed issue where a SQLAlchemy connection URI for Azure DW with authentication=ActiveDirectoryIntegrated (and no username+password) was not constructing the ODBC connection string in a way that was acceptable to the Azure DW instance.

    References: #5592

tests

  • [tests] [bug]

    Fixed incompatibilities in the test suite when running against Pytest 6.x.

    References: #5635

misc

  • [bug] [pool]

    Fixed issue where the following pool parameters were not being propagated to the new pool created when Engine.dispose() were called: pre_ping, use_lifo. Additionally the recycle and reset_on_return parameter is now propagated for the AssertionPool class.

    References: #5582

  • [bug] [associationproxy] [ext]

    An informative error is now raised when attempting to use an association proxy element as a plain column expression to be SELECTed from or used in a SQL function; this use case is not currently supported.

    References: #5541, #5542

1.3.19

Released: August 17, 2020

orm

  • [orm] [usecase]

    Adjusted the workings of the Mapper.all_orm_descriptors() accessor to represent the attributes in the order that they are located in a deterministic way, assuming the use of Python 3.6 or higher which maintains the sorting order of class attributes based on how they were declared. This sorting is not guaranteed to match the declared order of attributes in all cases however; see the method documentation for the exact scheme.

    References: #5494

orm declarative

  • [orm] [declarative] [usecase]

    The name of the virtual column used when using the AbstractConcreteBase and ConcreteBase classes can now be customized, to allow for models that have a column that is actually named type. Pull request courtesy Jesse-Bakker.

    References: #5513

sql

  • [sql] [bug]

    Repaired an issue where the “ORDER BY” clause rendering a label name rather than a complete expression, which is particularly important for SQL Server, would fail to occur if the expression were enclosed in a parenthesized grouping in some cases. This case has been added to test support. The change additionally adjusts the “automatically add ORDER BY columns when DISTINCT is present” behavior of ORM query, deprecated in 1.4, to more accurately detect column expressions that are already present.

    References: #5470

  • [sql] [bug] [datatypes]

    The LookupError message will now provide the user with up to four possible values that a column is constrained to via the Enum. Values longer than 11 characters will be truncated and replaced with ellipses. Pull request courtesy Ramon Williams.

    References: #4733

  • [sql] [bug]

    Fixed issue where the Connection.execution_options.schema_translate_map feature would not take effect when the Sequence.next_value() function function for a Sequence were used in the Column.server_default parameter and the create table DDL were emitted.

    References: #5500

postgresql

  • [postgresql] [bug]

    Fixed issue where the return type for the various RANGE comparison operators would itself be the same RANGE type rather than BOOLEAN, which would cause an undesirable result in the case that a TypeDecorator that defined result-processing behavior were in use. Pull request courtesy Jim Bosch.

    References: #5476

mysql

  • [mysql] [usecase]

    The MySQL dialect will render FROM DUAL for a SELECT statement that has no FROM clause but has a WHERE clause. This allows things like “SELECT 1 WHERE EXISTS (subquery)” kinds of queries to be used as well as other use cases.

    References: #5481

  • [mysql] [bug]

    Fixed an issue where CREATE TABLE statements were not specifying the COLLATE keyword correctly.

    References: #5411

  • [mysql] [bug]

    Added MariaDB code 1927 to the list of “disconnect” codes, as recent MariaDB versions apparently use this code when the database server was stopped.

    References: #5493

sqlite

  • [sqlite] [bug] [mssql] [reflection]

    Applied a sweep through all included dialects to ensure names that contain single or double quotes are properly escaped when querying system tables, for all Inspector methods that accept object names as an argument (e.g. table names, view names, etc). SQLite and MSSQL contained two quoting issues that were repaired.

    References: #5456

mssql

  • [mssql] [bug] [sql]

    Fixed bug where the mssql dialect incorrectly escaped object names that contained ‘]’ character(s).

    References: #5467

misc

  • [usecase] [py3k]

    Added a **kw argument to the DeclarativeMeta.__init__() method. This allows a class to support the PEP 487 metaclass hook __init_subclass__. Pull request courtesy Ewen Gillies.

    References: ##5357

1.3.18

Released: June 25, 2020

orm

examples

  • [examples] [change]

    Added new option --raw to the examples.performance suite which will dump the raw profile test for consumption by any number of profiling visualizer tools. Removed the “runsnake” option as runsnake is very hard to build at this point;

engine

  • [engine] [bug]

    Further refinements to the fixes to the “reset” agent fixed in #5326, which now emits a warning when it is not being correctly invoked and corrects for the behavior. Additional scenarios have been identified and fixed where this warning was being emitted.

    References: #5326

  • [engine] [bug]

    Fixed issue in URL object where stringifying the object would not URL encode special characters, preventing the URL from being re-consumable as a real URL. Pull request courtesy Miguel Grinberg.

    References: #5341

sql

  • [sql] [usecase]

    Added a “.schema” parameter to the table() construct, allowing ad-hoc table expressions to also include a schema name. Pull request courtesy Dylan Modesitt.

    References: #5309

  • [sql] [change] [sybase]

    Added .offset support to sybase dialect. Pull request courtesy Alan D. Snow.

    References: #5294

  • [sql] [bug]

    Correctly apply self_group in type_coerce element.

    The type coerce element did not correctly apply grouping rules when using in an expression

    References: #5344

  • [sql] [bug]

    Added Select.with_hint() output to the generic SQL string that is produced when calling str() on a statement. Previously, this clause would be omitted under the assumption that it was dialect specific. The hint text is presented within brackets to indicate the rendering of such hints varies among backends.

    References: #5353

  • [sql] [schema]

    Introduce IdentityOptions to store common parameters for sequences and identity columns.

    References: #5324

schema

  • [schema] [bug]

    Fixed issue where dialect_options were omitted when a database object (e.g., Table) was copied using tometadata().

    References: #5276

mysql

  • [mysql] [usecase]

    Implemented row-level locking support for mysql. Pull request courtesy Quentin Somerville.

    References: #4860

sqlite

  • [sqlite] [usecase]

    SQLite 3.31 added support for computed column. This change enables their support in SQLAlchemy when targeting SQLite.

    References: #5297

  • [sqlite] [bug]

    Added “exists” to the list of reserved words for SQLite so that this word will be quoted when used as a label or column name. Pull request courtesy Thodoris Sotiropoulos.

    References: #5395

mssql

  • [mssql] [change]

    Moved the supports_sane_rowcount_returning = False requirement from the PyODBCConnector level to the MSDialect_pyodbc since pyodbc does work properly in some circumstances.

    References: #5321

  • [mssql] [bug]

    Refined the logic used by the SQL Server dialect to interpret multi-part schema names that contain many dots, to not actually lose any dots if the name does not have bracking or quoting used, and additionally to support a “dbname” token that has many parts including that it may have multiple, independently-bracketed sections.

    References: #5364, #5366

  • [mssql] [bug] [pyodbc]

    Fixed an issue in the pyodbc connector such that a warning about pyodbc “drivername” would be emitted when using a totally empty URL. Empty URLs are normal when producing a non-connected dialect object or when using the “creator” argument to create_engine(). The warning now only emits if the driver name is missing but other parameters are still present.

    References: #5346

  • [mssql] [bug]

    Fixed issue with assembling the ODBC connection string for the pyodbc DBAPI. Tokens containing semicolons and/or braces “{}” were not being correctly escaped, causing the ODBC driver to misinterpret the connection string attributes.

    References: #5373

  • [mssql] [bug]

    Fixed issue where datetime.time parameters were being converted to datetime.datetime, making them incompatible with comparisons like >= against an actual TIME column.

    References: #5339

  • [mssql] [bug]

    Fixed an issue where the is_disconnect function in the SQL Server pyodbc dialect was incorrectly reporting the disconnect state when the exception message had a substring that matched a SQL Server ODBC error code.

    References: #5359

oracle

  • [oracle] [bug] [reflection]

    Fixed bug in Oracle dialect where indexes that contain the full set of primary key columns would be mistaken as the primary key index itself, which is omitted, even if there were multiples. The check has been refined to compare the name of the primary key constraint against the index name itself, rather than trying to guess based on the columns present in the index.

    References: #5421

1.3.17

Released: May 13, 2020

orm

  • [orm] [usecase]

    Added an accessor Comparator.expressions which provides access to the group of columns mapped under a multi-column ColumnProperty attribute.

    References: #5262

  • [orm] [usecase]

    Introduce relationship.sync_backref flag in a relationship to control if the synchronization events that mutate the in-Python attributes are added. This supersedes the previous change #5149, which warned that viewonly=True relationship target of a back_populates or backref configuration would be disallowed.

    References: #5237

  • [orm] [bug]

    Fixed bug where using with_polymorphic() as the target of a join via RelationshipComparator.of_type() on a mapper that already has a subquery-based with_polymorphic setting that’s equivalent to the one requested would not correctly alias the ON clause in the join.

    References: #5288

  • [orm] [bug]

    Fixed issue in the area of where loader options such as selectinload() interact with the baked query system, such that the caching of a query is not supposed to occur if the loader options themselves have elements such as with_polymorphic() objects in them that currently are not cache-compatible. The baked loader could sometimes not fully invalidate itself in these some of these scenarios leading to missed eager loads.

    References: #5303

  • [orm] [bug]

    Modified the internal “identity set” implementation, which is a set that hashes objects on their id() rather than their hash values, to not actually call the __hash__() method of the objects, which are typically user-mapped objects. Some methods were calling this method as a side effect of the implementation.

    References: #5304

  • [orm] [bug]

    An informative error message is raised when an ORM many-to-one comparison is attempted against an object that is not an actual mapped instance. Comparisons such as those to scalar subqueries aren’t supported; generalized comparison with subqueries is better achieved using Comparator.has().

    References: #5269

engine

  • [engine] [bug]

    Fixed fairly critical issue where the DBAPI connection could be returned to the connection pool while still in an un-rolled-back state. The reset agent responsible for rolling back the connection could be corrupted in the case that the transaction was “closed” without being rolled back or committed, which can occur in some scenarios when using ORM sessions and emitting .close() in a certain pattern involving savepoints. The fix ensures that the reset agent is always active.

    References: #5326

schema

  • [schema] [bug]

    Fixed issue where an Index that is deferred in being associated with a table, such as as when it contains a Column that is not associated with any Table yet, would fail to attach correctly if it also contained a non table-oriented expression.

    References: #5298

  • [schema] [bug]

    A warning is emitted when making use of the MetaData.sorted_tables attribute as well as the sort_tables() function, and the given tables cannot be correctly sorted due to a cyclic dependency between foreign key constraints. In this case, the functions will no longer sort the involved tables by foreign key, and a warning will be emitted. Other tables that are not part of the cycle will still be returned in dependency order. Previously, the sorted_table routines would return a collection that would unconditionally omit all foreign keys when a cycle was detected, and no warning was emitted.

    References: #5316

  • [schema]

    Add comment attribute to Column __repr__ method.

    References: #4138

postgresql

mssql

  • [mssql] [bug] [reflection]

    Fix a regression introduced by the reflection of computed column in MSSQL when using the legacy TDS version 4.2. The dialect will try to detect the protocol version of first connect and run in compatibility mode if it cannot detect it.

    References: #5255

  • [mssql] [bug] [reflection]

    Fix a regression introduced by the reflection of computed column in MSSQL when using SQL server versions before 2012, which does not support the concat function.

    References: #5271

oracle

  • [oracle] [bug]

    Some modifications to how the cx_oracle dialect sets up per-column outputtype handlers for LOB and numeric datatypes to adjust for potential changes coming in cx_Oracle 8.

    References: #5246

  • [oracle] [bug] [performance]

    Changed the implementation of fetching CLOB and BLOB objects to use cx_Oracle’s native implementation which fetches CLOB/BLOB objects inline with other result columns, rather than performing a separate fetch. As always, this can be disabled by setting auto_convert_lobs to False.

    As part of this change, the behavior of a CLOB that was given a blank string on INSERT now returns None on SELECT, which is now consistent with that of VARCHAR on Oracle.

    References: #5314

misc

  • [change] [firebird]

    Adjusted dialect loading for firebird:// URIs so the external sqlalchemy-firebird dialect will be used if it has been installed, otherwise fall back to the (now deprecated) internal Firebird dialect.

    References: #5278

1.3.16

Released: April 7, 2020

orm

  • [orm] [performance]

    Modified the queries used by subqueryload and selectinload to no longer ORDER BY the primary key of the parent entity; this ordering was there to allow the rows as they come in to be copied into lists directly with a minimal level of Python-side collation. However, these ORDER BY clauses can negatively impact the performance of the query as in many scenarios these columns are derived from a subquery or are otherwise not actual primary key columns such that SQL planners cannot make use of indexes. The Python-side collation uses the native itertools.group_by() to collate the incoming rows, and has been modified to allow multiple row-groups-per-parent to be assembled together using list.extend(), which should still allow for relatively fast Python-side performance. There will still be an ORDER BY present for a relationship that includes an explicit order_by parameter, however this is the only ORDER BY that will be added to the query for both kinds of loading.

    References: #5162

  • [orm] [bug]

    Fixed bug in selectinload() loading option where two or more loaders that represent different relationships with the same string key name as referenced from a single with_polymorphic() construct with multiple subclass mappers would fail to invoke each subqueryload separately, instead making use of a single string-based slot that would prevent the other loaders from being invoked.

    References: #5228

  • [orm] [bug]

    Fixed issue where a lazyload that uses session-local “get” against a target many-to-one relationship where an object with the correct primary key is present, however it’s an instance of a sibling class, does not correctly return None as is the case when the lazy loader actually emits a load for that row.

    References: #5210

orm declarative

  • [orm] [declarative] [bug]

    The string argument accepted as the first positional argument by the relationship() function when using the Declarative API is no longer interpreted using the Python eval() function; instead, the name is dot separated and the names are looked up directly in the name resolution dictionary without treating the value as a Python expression. However, passing a string argument to the other relationship() parameters that necessarily must accept Python expressions will still use eval(); the documentation has been clarified to ensure that there is no ambiguity that this is in use.

    See also

    Evaluation of relationship arguments - details on string evaluation

    References: #5238

sql

  • [sql] [types]

    Add ability to literal compile a DateTime, Date or Time when using the string dialect for debugging purposes. This change does not impact real dialect implementation that retain their current behavior.

    References: #5052

schema

  • [schema] [reflection]

    Added support for reflection of “computed” columns, which are now returned as part of the structure returned by Inspector.get_columns(). When reflecting full Table objects, computed columns will be represented using the Computed construct.

    References: #5063

postgresql

  • [postgresql] [bug]

    Fixed issue where a “covering” index, e.g. those which have an INCLUDE clause, would be reflected including all the columns in INCLUDE clause as regular columns. A warning is now emitted if these additional columns are detected indicating that they are currently ignored. Note that full support for “covering” indexes is part of #4458. Pull request courtesy Marat Sharafutdinov.

    References: #5205

mysql

  • [mysql] [bug]

    Fixed issue in MySQL dialect when connecting to a pseudo-MySQL database such as that provided by ProxySQL, the up front check for isolation level when it returns no row will not prevent the dialect from continuing to connect. A warning is emitted that the isolation level could not be detected.

    References: #5239

sqlite

  • [sqlite] [usecase]

    Implemented AUTOCOMMIT isolation level for SQLite when using pysqlite.

    References: #5164

mssql

oracle

  • [oracle] [usecase]

    Implemented AUTOCOMMIT isolation level for Oracle when using cx_Oracle. Also added a fixed default isolation level of READ COMMITTED for Oracle.

    References: #5200

  • [oracle] [bug] [reflection]

    Fixed regression / incorrect fix caused by fix for #5146 where the Oracle dialect reads from the “all_tab_comments” view to get table comments but fails to accommodate for the current owner of the table being requested, causing it to read the wrong comment if multiple tables of the same name exist in multiple schemas.

    References: #5146

tests

  • [tests] [bug]

    Fixed an issue that prevented the test suite from running with the recently released py.test 5.4.0.

    References: #5201

misc

  • [enum] [types]

    The Enum type now supports the parameter Enum.length to specify the length of the VARCHAR column to create when using non native enums by setting Enum.native_enum to False

    References: #5183

  • [installer]

    Ensured that the “pyproject.toml” file is not included in builds, as the presence of this file indicates to pip that a pep-517 installation process should be used. As this mode of operation appears to be not well supported by current tools / distros, these problems are avoided within the scope of SQLAlchemy installation by omitting the file.

    References: #5207

1.3.15

Released: March 11, 2020

orm

  • [orm] [bug]

    Adjusted the error message emitted by Query.join() when a left hand side can’t be located that the Query.select_from() method is the best way to resolve the issue. Also, within the 1.3 series, used a deterministic ordering when determining the FROM clause from a given column entity passed to Query so that the same expression is determined each time.

    References: #5194

  • [orm] [bug]

    Fixed regression in 1.3.14 due to #4849 where a sys.exc_info() call failed to be invoked correctly when a flush error would occur. Test coverage has been added for this exception case.

    References: #5196

1.3.14

Released: March 10, 2020

general

  • [general] [bug] [py3k]

    Applied an explicit “cause” to most if not all internally raised exceptions that are raised from within an internal exception catch, to avoid misleading stacktraces that suggest an error within the handling of an exception. While it would be preferable to suppress the internally caught exception in the way that the __suppress_context__ attribute would, there does not as yet seem to be a way to do this without suppressing an enclosing user constructed context, so for now it exposes the internally caught exception as the cause so that full information about the context of the error is maintained.

    References: #4849

orm

  • [orm] [usecase]

    Added a new flag InstanceEvents.restore_load_context and SessionEvents.restore_load_context which apply to the InstanceEvents.load(), InstanceEvents.refresh(), and SessionEvents.loaded_as_persistent() events, which when set will restore the “load context” of the object after the event hook has been called. This ensures that the object remains within the “loader context” of the load operation that is already ongoing, rather than the object being transferred to a new load context due to refresh operations which may have occurred in the event. A warning is now emitted when this condition occurs, which recommends use of the flag to resolve this case. The flag is “opt-in” so that there is no risk introduced to existing applications.

    The change additionally adds support for the raw=True flag to session lifecycle events.

    References: #5129

  • [orm] [bug]

    Fixed regression caused in 1.3.13 by #5056 where a refactor of the ORM path registry system made it such that a path could no longer be compared to an empty tuple, which can occur in a particular kind of joined eager loading path. The “empty tuple” use case has been resolved so that the path registry is compared to a path registry in all cases; the PathRegistry object itself now implements __eq__() and __ne__() methods which will take place for all equality comparisons and continue to succeed in the not anticipated case that a non- PathRegistry object is compared, while emitting a warning that this object should not be the subject of the comparison.

    References: #5110

  • [orm] [bug]

    Setting a relationship to viewonly=True which is also the target of a back_populates or backref configuration will now emit a warning and eventually be disallowed. back_populates refers specifically to mutation of an attribute or collection, which is disallowed when the attribute is subject to viewonly=True. The viewonly attribute is not subject to persistence behaviors which means it will not reflect correct results when it is locally mutated.

    References: #5149

  • [orm] [bug]

    Fixed an additional regression in the same area as that of #5080 introduced in 1.3.0b3 via #4468 where the ability to create a joined option across a with_polymorphic() into a relationship against the base class of that with_polymorphic, and then further into regular mapped relationships would fail as the base class component would not add itself to the load path in a way that could be located by the loader strategy. The changes applied in #5080 have been further refined to also accommodate this scenario.

    References: #5121

engine

  • [engine] [bug]

    Expanded the scope of cursor/connection cleanup when a statement is executed to include when the result object fails to be constructed, or an after_cursor_execute() event raises an error, or autocommit / autoclose fails. This allows the DBAPI cursor to be cleaned up on failure and for connectionless execution allows the connection to be closed out and returned to the connection pool, where previously it waiting until garbage collection would trigger a pool return.

    References: #5182

sql

  • [sql] [bug] [postgresql]

    Fixed bug where a CTE of an INSERT/UPDATE/DELETE that also uses RETURNING could then not be SELECTed from directly, as the internal state of the compiler would try to treat the outer SELECT as a DELETE statement itself and access nonexistent state.

    References: #5181

postgresql

  • [postgresql] [bug]

    Fixed issue where the “schema_translate_map” feature would not work with a PostgreSQL native enumeration type (i.e. Enum, ENUM) in that while the “CREATE TYPE” statement would be emitted with the correct schema, the schema would not be rendered in the CREATE TABLE statement at the point at which the enumeration was referenced.

    References: #5158

  • [postgresql] [bug] [reflection]

    Fixed bug where PostgreSQL reflection of CHECK constraints would fail to parse the constraint if the SQL text contained newline characters. The regular expression has been adjusted to accommodate for this case. Pull request courtesy Eric Borczuk.

    References: #5170

mysql

  • [mysql] [bug]

    Fixed issue in MySQL Insert.on_duplicate_key_update() construct where using a SQL function or other composed expression for a column argument would not properly render the VALUES keyword surrounding the column itself.

    References: #5173

mssql

  • [mssql] [bug]

    Fixed issue where the DATETIMEOFFSET type would not accommodate for the None value, introduced as part of the series of fixes for this type first introduced in #4983, #5045. Additionally, added support for passing a backend-specific date formatted string through this type, as is typically allowed for date/time types on most other DBAPIs.

    References: #5132

oracle

  • [oracle] [bug]

    Fixed a reflection bug where table comments could only be retrieved for tables actually owned by the user but not for tables visible to the user but owned by someone else. Pull request courtesy Dave Hirschfeld.

    References: #5146

misc

  • [usecase] [ext]

    Added keyword arguments to the MutableList.sort() function so that a key function as well as the “reverse” keyword argument can be provided.

    References: #5114

  • [bug] [performance]

    Revised an internal change to the test system added as a result of #5085 where a testing-related module per dialect would be loaded unconditionally upon making use of that dialect, pulling in SQLAlchemy’s testing framework as well as the ORM into the module import space. This would only impact initial startup time and memory to a modest extent, however it’s best that these additional modules aren’t reverse-dependent on straight Core usage.

    References: #5180

  • [bug] [installation]

    Vendored the inspect.formatannotation function inside of sqlalchemy.util.compat, which is needed for the vendored version of inspect.formatargspec. The function is not documented in cPython and is not guaranteed to be available in future Python versions.

    References: #5138

1.3.13

Released: January 22, 2020

orm

  • [orm] [performance]

    Identified a performance issue in the system by which a join is constructed based on a mapped relationship. The clause adaption system would be used for the majority of join expressions including in the common case where no adaptation is needed. The conditions under which this adaptation occur have been refined so that average non-aliased joins along a simple relationship without a “secondary” table use about 70% less function calls.

  • [orm] [bug] [engine]

    Added test support and repaired a wide variety of unnecessary reference cycles created for short-lived objects, mostly in the area of ORM queries. Thanks much to Carson Ip for the help on this.

    References: #5050, #5056, #5071

  • [orm] [bug]

    Fixed regression in loader options introduced in 1.3.0b3 via #4468 where the ability to create a loader option using PropComparator.of_type() targeting an aliased entity that is an inheriting subclass of the entity which the preceding relationship refers to would fail to produce a matching path. See also #5082 fixed in this same release which involves a similar kind of issue.

    References: #5107

  • [orm] [bug]

    Fixed regression in joined eager loading introduced in 1.3.0b3 via #4468 where the ability to create a joined option across a with_polymorphic() into a polymorphic subclass using RelationshipProperty.of_type() and then further along regular mapped relationships would fail as the polymorphic subclass would not add itself to the load path in a way that could be located by the loader strategy. A tweak has been made to resolve this scenario.

    References: #5082

  • [orm] [bug]

    Repaired a warning in the ORM flush process that was not covered by test coverage when deleting objects that use the “version_id” feature. This warning is generally unreachable unless using a dialect that sets the “supports_sane_rowcount” flag to False, which is not typically the case however is possible for some MySQL configurations as well as older Firebird drivers, and likely some third party dialects.

    References: #5068

  • [orm] [bug]

    Fixed bug where usage of joined eager loading would not properly wrap the query inside of a subquery when Query.group_by() were used against the query. When any kind of result-limiting approach is used, such as DISTINCT, LIMIT, OFFSET, joined eager loading embeds the row-limited query inside of a subquery so that the collection results are not impacted. For some reason, the presence of GROUP BY was never included in this criterion, even though it has a similar effect as using DISTINCT. Additionally, the bug would prevent using GROUP BY at all for a joined eager load query for most database platforms which forbid non-aggregated, non-grouped columns from being in the query, as the additional columns for the joined eager load would not be accepted by the database.

    References: #5065

engine

  • [engine] [bug]

    Fixed issue where the collection of value processors on a Compiled object would be mutated when “expanding IN” parameters were used with a datatype that has bind value processors; in particular, this would mean that when using statement caching and/or baked queries, the same compiled._bind_processors collection would be mutated concurrently. Since these processors are the same function for a given bind parameter namespace every time, there was no actual negative effect of this issue, however, the execution of a Compiled object should never be causing any changes in its state, especially given that they are intended to be thread-safe and reusable once fully constructed.

    References: #5048

sql

  • [sql] [usecase]

    A function created using GenericFunction can now specify that the name of the function should be rendered with or without quotes by assigning the quoted_name construct to the .name element of the object. Prior to 1.3.4, quoting was never applied to function names, and some quoting was introduced in #4467 but no means to force quoting for a mixed case name was available. Additionally, the quoted_name construct when used as the name will properly register its lowercase name in the function registry so that the name continues to be available via the func. registry.

    See also

    GenericFunction

    References: #5079

postgresql

  • [postgresql] [usecase]

    Added support for prefixes to the CTE construct, to allow support for Postgresql 12 “MATERIALIZED” and “NOT MATERIALIZED” phrases. Pull request courtesy Marat Sharafutdinov.

    See also

    HasCTE.cte()

    References: #5040

  • [postgresql] [bug]

    Fixed issue where the PostgreSQL dialect would fail to parse a reflected CHECK constraint that was a boolean-valued function (as opposed to a boolean-valued expression).

    References: #5039

mssql

  • [mssql] [bug]

    Fixed issue where a timezone-aware datetime value being converted to string for use as a parameter value of a DATETIMEOFFSET column was omitting the fractional seconds.

    References: #5045

tests

  • [tests] [bug]

    Fixed a few test failures which would occur on Windows due to SQLite file locking issues, as well as some timing issues in connection pool related tests; pull request courtesy Federico Caselli.

    References: #4946

  • [tests] [postgresql]

    Improved detection of two phase transactions requirement for the PostgreSQL database by testing that max_prepared_transactions is set to a value greater than 0. Pull request courtesy Federico Caselli.

    References: #5057

misc

  • [bug] [ext]

    Fixed bug in sqlalchemy.ext.serializer where a unique BindParameter object could conflict with itself if it were present in the mapping itself, as well as the filter condition of the query, as one side would be used against the non-deserialized version and the other side would use the deserialized version. Logic is added to BindParameter similar to its “clone” method which will uniquify the parameter name upon deserialize so that it doesn’t conflict with its original.

    References: #5086

1.3.12

Released: December 16, 2019

orm

  • [orm] [bug]

    Fixed issue involving lazy="raise" strategy where an ORM delete of an object would raise for a simple “use-get” style many-to-one relationship that had lazy=”raise” configured. This is inconsistent vs. the change introduced in 1.3 as part of #4353, where it was established that a history operation that does not expect emit SQL should bypass the lazy="raise" check, and instead effectively treat it as lazy="raise_on_sql" for this case. The fix adjusts the lazy loader strategy to not raise for the case where the lazy load was instructed that it should not emit SQL if the object were not present.

    References: #4997

  • [orm] [bug]

    Fixed regression introduced in 1.3.0 related to the association proxy refactor in #4351 that prevented composite() attributes from working in terms of an association proxy that references them.

    References: #5000

  • [orm] [bug]

    Setting persistence-related flags on relationship() while also setting viewonly=True will now emit a regular warning, as these flags do not make sense for a viewonly=True relationship. In particular, the “cascade” settings have their own warning that is generated based on the individual values, such as “delete, delete-orphan”, that should not apply to a viewonly relationship. Note however that in the case of “cascade”, these settings are still erroneously taking effect even though the relationship is set up as “viewonly”. In 1.4, all persistence-related cascade settings will be disallowed on a viewonly=True relationship in order to resolve this issue.

    References: #4993

  • [orm] [bug] [py3k]

    Fixed issue where when assigning a collection to itself as a slice, the mutation operation would fail as it would first erase the assigned collection inadvertently. As an assignment that does not change the contents should not generate events, the operation is now a no-op. Note that the fix only applies to Python 3; in Python 2, the __setitem__ hook isn’t called in this case; __setslice__ is used instead which recreates the list item-by-item in all cases.

    References: #4990

  • [orm] [bug]

    Fixed issue where by if the “begin” of a transaction failed at the Core engine/connection level, such as due to network error or database is locked for some transactional recipes, within the context of the Session procuring that connection from the connection pool and then immediately returning it, the ORM Session would not close the connection despite this connection not being stored within the state of that Session. This would lead to the connection being cleaned out by the connection pool weakref handler within garbage collection which is an unpreferred codepath that in some special configurations can emit errors in standard error.

    References: #5034

sql

  • [sql] [bug]

    Fixed bug where “distinct” keyword passed to select() would not treat a string value as a “label reference” in the same way that the select.distinct() does; it would instead raise unconditionally. This keyword argument and the others passed to select() will ultimately be deprecated for SQLAlchemy 2.0.

    References: #5028

  • [sql] [bug]

    Changed the text of the exception for “Can’t resolve label reference” to include other kinds of label coercions, namely that “DISTINCT” is also in this category under the PostgreSQL dialect.

sqlite

  • [sqlite] [bug]

    Fixed issue to workaround SQLite’s behavior of assigning “numeric” affinity to JSON datatypes, first described at Support for SQLite JSON Added, which returns scalar numeric JSON values as a number and not as a string that can be JSON deserialized. The SQLite-specific JSON deserializer now gracefully degrades for this case as an exception and bypasses deserialization for single numeric values, as from a JSON perspective they are already deserialized.

    References: #5014

mssql

  • [mssql] [bug]

    Repaired support for the DATETIMEOFFSET datatype on PyODBC, by adding PyODBC-level result handlers as it does not include native support for this datatype. This includes usage of the Python 3 “timezone” tzinfo subclass in order to set up a timezone, which on Python 2 makes use of a minimal backport of “timezone” in sqlalchemy.util.

    References: #4983

1.3.11

Released: November 11, 2019

orm

  • [orm] [usecase]

    Added accessor Query.is_single_entity() to Query, which will indicate if the results returned by this Query will be a list of ORM entities, or a tuple of entities or column expressions. SQLAlchemy hopes to improve upon the behavior of single entity / tuples in future releases such that the behavior would be explicit up front, however this attribute should be helpful with the current behavior. Pull request courtesy Patrick Hayes.

    References: #4934

  • [orm] [bug]

    The relationship.omit_join flag was not intended to be manually set to True, and will now emit a warning when this occurs. The omit_join optimization is detected automatically, and the omit_join flag was only intended to disable the optimization in the hypothetical case that the optimization may have interfered with correct results, which has not been observed with the modern version of this feature. Setting the flag to True when it is not automatically detected may cause the selectin load feature to not work correctly when a non-default primary join condition is in use.

    References: #4954

  • [orm] [bug]

    A warning is emitted if a primary key value is passed to Query.get() that consists of None for all primary key column positions. Previously, passing a single None outside of a tuple would raise a TypeError and passing a composite None (tuple of None values) would silently pass through. The fix now coerces the single None into a tuple where it is handled consistently with the other None conditions. Thanks to Lev Izraelit for the help with this.

    References: #4915

  • [orm] [bug]

    The BakedQuery will not cache a query that was modified by a QueryEvents.before_compile() event, so that compilation hooks that may be applying ad-hoc modifications to queries will take effect on each run. In particular this is helpful for events that modify queries used in lazy loading as well as eager loading such as “select in” loading. In order to re-enable caching for a query modified by this event, a new flag bake_ok is added; see Using the before_compile event for details.

    A longer term plan to provide a new form of SQL caching should solve this kind of issue more comprehensively.

    References: #4947

  • [orm] [bug]

    Fixed ORM bug where a “secondary” table that referred to a selectable which in some way would refer to the local primary table would apply aliasing to both sides of the join condition when a relationship-related join, either via Query.join() or by joinedload(), were generated. The “local” side is now excluded.

    References: #4974

engine

  • [engine] [bug]

    Fixed bug where parameter repr as used in logging and error reporting needs additional context in order to distinguish between a list of parameters for a single statement and a list of parameter lists, as the “list of lists” structure could also indicate a single parameter list where the first parameter itself is a list, such as for an array parameter. The engine/connection now passes in an additional boolean indicating how the parameters should be considered. The only SQLAlchemy backend that expects arrays as parameters is that of psycopg2 which uses pyformat parameters, so this issue has not been too apparent, however as other drivers that use positional gain more features it is important that this be supported. It also eliminates the need for the parameter repr function to guess based on the parameter structure passed.

    References: #4902

  • [engine] [bug] [postgresql]

    Fixed bug in Inspector where the cache key generation did not take into account arguments passed in the form of tuples, such as the tuple of view name styles to return for the PostgreSQL dialect. This would lead the inspector to cache too generally for a more specific set of criteria. The logic has been adjusted to include every keyword element in the cache, as every argument is expected to be appropriate for a cache else the caching decorator should be bypassed by the dialect.

    References: #4955

sql

  • [sql] [usecase]

    Added new accessors to expressions of type JSON to allow for specific datatype access and comparison, covering strings, integers, numeric, boolean elements. This revises the documented approach of CASTing to string when comparing values, instead adding specific functionality into the PostgreSQL, SQlite, MySQL dialects to reliably deliver these basic types in all cases.

    References: #4276

  • [sql] [usecase]

    The text() construct now supports “unique” bound parameters, which will dynamically uniquify themselves on compilation thus allowing multiple text() constructs with the same bound parameter names to be combined together.

    References: #4933

  • [sql] [bug] [py3k]

    Changed the repr() of the quoted_name construct to use regular string repr() under Python 3, rather than running it through “backslashreplace” escaping, which can be misleading.

    References: #4931

schema

  • [schema] [usecase]

    Added DDL support for “computed columns”; these are DDL column specifications for columns that have a server-computed value, either upon SELECT (known as “virtual”) or at the point of which they are INSERTed or UPDATEd (known as “stored”). Support is established for Postgresql, MySQL, Oracle SQL Server and Firebird. Thanks to Federico Caselli for lots of work on this one.

    References: #4894