C贸mo empez贸
Aclaraci贸n: A pesar de que esto es el resultado de un problema laboral, todas las opiniones son m铆as; mi empleador no est谩 involucrado ni respalda de ninguna manera esta publicaci贸n, m谩s all谩 de una autorizaci贸n impl铆cita para escribir p煤blicamente sobre problemas t茅cnicos y soluciones.
En una charla diaria con algunos de mis colegas en ShipHero, alguien mencion贸 que un problema que est谩bamos teniendo (no relevante para este post) podr铆a estar causado por sesiones thread-local de sqlalchemy que se pasaban a otro hilo.
El camino f谩cil
Podr铆amos haber hecho una auditor铆a de toda nuestra base de c贸digo en torno a las conexiones. De hecho, la hicimos, pero solo en el core. Una revisi贸n m谩s a fondo habr铆a tomado m谩s tiempo del realistamente posible.
Suger铆 una soluci贸n de compromiso: pod铆amos agregar un peque帽o hack, que de ninguna manera era eficiente, pero usado solo en un nodo podr铆a ayudarnos a determinar si este era el caso.
El c贸digo
Hace a帽os que no trabajo en Python como mi lenguaje principal; reci茅n ahora estoy volviendo a 茅l, as铆 que este c贸digo puede estar bastante oxidado (salt谩 hasta el final para verlo completo).
El envoltorio externo
La idea general es envolver el objeto scoped_session(session_factory) de modo que, cada vez que instancia una nueva sesi贸n ligada a un hilo, obtengamos en cambio un objeto envuelto que tiene metadatos sobre el hilo al que est谩 ligado.
class SessionBoundaries:
"""A wrapper class for thread local sessions
This should help in following each session during their lifetime and err if the session is used
in a thread that is not the origin one
"""
def __init__(self, thread_session):
self._session = thread_session
def __call__(self, *args, **kwargs):
"""A replacement for session_factory()
A new thread local session is created by invoking scoped_session(SessionFactory)()
this implements the wrapper () to maintain behavior"""
new_sess = self._session(*args, **kwargs)
current_thread_id = threading.get_ident()
log = logging.getLogger( "SessionBoundaries.wrapper" )
return Wrapper(new_sess, current_thread_id, log)
def __getattr__(self, name):
"""Intercept every attribute access
Except for __call__ all others are relayed"""
return getattr(self.__session, name)El envoltorio interno
El objeto tiene muy poca l贸gica: deja pasar cada argumento solicitado, pero registrar谩 un mensaje si la sesi贸n se usa fuera de su hilo inicial.
class Wrapper:
"""The actual inner wrapper class for the session object"""
def __init__(self, sub_s, t_id, log):
"""
:param sub_s: the subjacent connection
:param t_id: the id of this thread
"""
self._sub_s = sub_s
self._t_id = t_id
self._log = log
def __getattr__(self, name):
"""Intercept every attribute access"""
# get the current thread information
t_ident = threading.get_ident()
# log a warning if we are not in the original thread
if self.t_id != t_ident:
import traceback
stack = "".join(traceback.format_stack())
self._log.debug(f"boundaries crossed in {stack}")
# finally relay the attr access to the alchemy conn
return getattr(self.sub_s, name)Uso
El uso es bastante simple:
# Create a session factory and a scoped_session.
session_factory = sessionmaker(bind=engine)
scoped_session_manager = SessionBoundaries(scoped_session(session_factory))Un test de ejemplo
Este es un test simple que verifica que funciona; requiere una conexi贸n mysql.
# Pull the latest MySQL image (using MySQL 8.0 in this example)
docker pull mysql:8.0
# Run a new MySQL container:
# - Container name: mysql-test
# - Root password: my-secret-pw
# - Create a database named test_db
# - Map port 3306 on the container to 3306 on the host
docker run --name mysql-test \
-e MYSQL_ROOT_PASSWORD=my-secret-pw \
-e MYSQL_DATABASE=test_db \
-p 3306:3306 \
-d mysql:8.0
# Wait until MySQL is ready to accept connections.
echo "Waiting for MySQL to start..."
sleep 30 # This might not be enough in any system
echo "MySQL should now be running on localhost:3306"Y corre como un test normal.
Con requisitos:
SQLAlchemy==2.0.38
import logging
import sys
import unittest
import threading
from sqlalchemy import create_engine, Column, Integer, String, select
from sqlalchemy.orm import sessionmaker, scoped_session, declarative_base
# Define the base and model.
Base = declarative_base()
class TheMostGenericModelNameEver(Base):
"""A very generic model to exemplify sqlalchemy usage"""
__tablename__ = 'the_most_generic_model_name_ever'
id = Column(Integer, primary_key=True)
name = Column(String(256), nullable=False)
class Wrapper:
"""The actual inner wrapper class for the session object"""
def __init__(self, sub_s, t_id, log):
"""
:param sub_s: the subjacent connection
:param t_id: the id of this thread
"""
self._sub_s = sub_s
self._t_id = t_id
self._log = log
def __getattr__(self, name):
"""Intercept every attribute access"""
# get the current thread information
t_ident = threading.get_ident()
# log a warning if we are not in the original thread
if self.t_id != t_ident:
import traceback
stack = "".join(traceback.format_stack())
self._log.debug(f"boundaries crossed in {stack}")
# finally relay the attr access to the alchemy conn
return getattr(self.sub_s, name)
class SessionBoundaries:
"""A wrapper class for thread local sessions
This should help in following each session during their lifetime and err if the session is used
in a thread that is not the origin one
"""
def __init__(self, thread_session):
self._session = thread_session
def __call__(self, *args, **kwargs):
"""A replacement for session_factory()
A new thread local session is created by invoking scoped_session(SessionFactory)()
this implements the wrapper () to maintain behavior"""
new_sess = self._session(*args, **kwargs)
current_thread_id = threading.get_ident()
log = logging.getLogger( "SessionBoundaries.wrapper" )
return Wrapper(new_sess, current_thread_id, log)
def __getattr__(self, name):
"""Intercept every attribute access
Except for __call__ all others are relayed"""
return getattr(self.__session, name)
class TestScopedSessionThreading(unittest.TestCase):
def test_scoped_session_with_threads_and_model(self):
log= logging.getLogger( "TestScopedSessionThreading.test_scoped_session_with_threads_and_model" )
# This test was trying to be accurate to the problematic project.
engine = create_engine(
"mysql+pymysql://root:my-secret-pw@localhost:3306/test_db",
pool_pre_ping=True, # Helps to ensure connections are still valid.
echo=True # Optional: echoes SQL for debugging.
)
# Create the table.
Base.metadata.create_all(engine)
# Create a session factory and a scoped_session.
session_factory = sessionmaker(bind=engine)
scoped_session_manager = SessionBoundaries(scoped_session(session_factory))
# Insert some records before starting threads.
main_session = scoped_session_manager()
try:
models = main_session.query(TheMostGenericModelNameEver).all()
for model in models:
main_session.delete(model)
except Exception as e:
log.debug(e)
main_session.add_all([TheMostGenericModelNameEver(name=f"Record {i}") for i in range(10)])
main_session.commit()
scoped_session_manager.remove() # Remove main thread session.
errors = [] # To collect errors from threads.
session = scoped_session_manager()
def worker(thread_id):
try:
# Use session.scalars() to directly obtain a list of MyModel instances.
stmt = select(TheMostGenericModelNameEver)
models = session.scalars(stmt).all()
if len(models) != 10:
errors.append(f"Thread {thread_id}: Expected 10 records, got {len(models)} q: {str(stmt)}")
# Optionally verify that the names are as expected.
expected_names = {f"Record {i}" for i in range(10)}
record_names = {model.name for model in models}
if record_names != expected_names:
errors.append(f"Thread {thread_id}: Record names mismatch. q: {str(stmt)}")
except Exception as e:
errors.append(f"Thread {thread_id}: Exception: {e} q: {str(stmt)}")
finally:
pass
threads = []
for i in range(3):
t = threading.Thread(target=worker, args=(i,))
threads.append(t)
t.start()
for t in threads:
t.join()
if errors:
self.fail("Errors occurred in threads: " + "\n * ".join(errors))
else:
log.debug("All threads executed successfully.")
scoped_session_manager.remove()
if __name__ == '__main__':
logging.basicConfig( stream=sys.stderr )
logging.getLogger( "TestScopedSessionThreading.test_scoped_session_with_threads_and_model" ).setLevel( logging.DEBUG )
logging.getLogger( "SessionBoundaries.wrapper" ).setLevel( logging.DEBUG )
unittest.main()
Comments via 馃Γ
With an account on the Fediverse or Mastodon, you can respond to this post. Known replies are displayed below:
Credits on this comments implementation to Andreas Scherbaum.Note: This will load data from hachyderm.io.
Comments via 馃