API Reference¶
Database¶
Database is the main entrypoint for interacting with
alertbase. A Database provides functions for adding
new data, as well as for retrieving it.
Database hold some state, so you should generally use them in as
context managers, like this:
import alertbase
with alertbase.Database.open("alerts.db") as db:
for alert in db.get_by_object_id("ZTF19abf123g"):
print(alert.candidate_id)
If you like, you can alternatively call Database.close directly:
import alertbase
db = alertbase.Database.open("alerts.db")
for alert in db.get_by_object_id("ZTF19abf123g"):
print(alert.candidate_id)
db.close()
If you don’t remember to close a database, then database metadata might get corrupted, and the LevelDB indexes might be left in a strange state.
-
class
alertbase.Database¶ -
classmethod
open(db_path)¶ Opens a database from disk.
The database should already exist. To create a new database, use
create.
-
classmethod
create(region, bucket, db_path)¶ Creates a new database on disk and returns the opened database.
- Parameters
region (
strstr) – The name of the S3 region that houses the bucket with data. For example, ‘us-west-2’.bucket (
strstr) – The name of the S3 bucket that stores data. The bucket must already exist.db_path (
str|PathUnion[str,Path]) – A path on disk to a directory where the LevelDB index will be stored. This path should already exist; the database will be created within it.
- Return type
- Returns
The newly-created Database.
-
close()¶ Close the Database. If any new alerts were written into the database since it was opened, then the stored metadata will be updated by recomputing summary statistics, which can take a little while.
After calling this, the Database’s underlying storage handles will be closed, so the Database will no longer work for queries.
- Return type
NoneNone
-
get_by_candidate_id(candidate_id)¶ Fetch a single
AlertRecordgiven its candidate ID number.If no alert is indexed with that ID, this returns None.
- Parameters
candidate_id (
intint) – The ID of the alert candidate to retrieve.- Return type
AlertRecord|NoneOptional[AlertRecord]- Returns
The full alert payload.
-
get_by_object_id(object_id)¶ Fetch all alerts associated with a particular ZTF object ID. The returned list may be empty if there are no alerts or if the object ID isn’t known.
- Parameters
object_id (
strstr) – The ZTF Object ID to search for.- Return type
List[AlertRecord]List[AlertRecord]- Returns
A list of all alerts associated with the object.
-
get_by_object_id_stream(object_id)¶ Asynchronously start retrieving all alerts associated with a particular object.
This is the async equivalent of
get_by_object_id.- Parameters
object_id (
strstr) – The ZTF Object ID to search for.- Return type
AsyncGenerator[AlertRecord,None]AsyncGenerator[AlertRecord,None]- Returns
An asynchronous stream of the alerts associated with the object.
-
get_by_time_range(start, end)¶ Fetch all alerts from between the given start and end times. This list can be very large!
More precisely, an alert’s timestamp is defined as the
jdfield of the Avro payload from ZTF.- Parameters
start (astropy.time.Time) – Start of the time range to search over (inclusive).
end (astropy.time.Time) – End of the time range to search over (exclusive).
- Return type
List[AlertRecord]List[AlertRecord]- Returns
A list of all alerts between the two times.
-
get_by_time_range_stream(start, end)¶ Asynchronously start retrieving all alerts from between the given start and end times.
More precisely, an alert’s timestamp is defined as the
jdfield of the Avro payload from ZTF.This is the async equivalent of
get_by_time_range.- Parameters
start (astropy.time.Time) – Start of the time range to search over (inclusive).
end (astropy.time.Time) – End of the time range to search over (exclusive).
- Return type
AsyncGenerator[AlertRecord,None]AsyncGenerator[AlertRecord,None]- Returns
An asynchronous stream of all alerts between the two times.
-
get_by_cone_search(center, radius)¶ Fetch all alerts in a circular region of the sky.
- Parameters
center (astropy.coordinates.SkyCoord) – The center of the circular region to search.
radius (astropy.coordinates.Angle) – The radius of the disc to search over.
- Return type
List[AlertRecord]List[AlertRecord]- Returns
A list of all alerts in the region.
-
get_by_cone_search_stream(center, radius)¶ - Return type
AsyncGenerator[AlertRecord,None]AsyncGenerator[AlertRecord,None]- Parameters
center (astropy.coordinates.sky_coordinate.SkyCoord) –
radius (astropy.coordinates.angles.Angle) –
-
write(alert)¶ Synchronously write a single alert into the database.
- Parameters
alert (
AlertRecordAlertRecord) – The alert to write to the database.- Return type
NoneNone
-
async
write_many(alerts, n_worker)¶ Asynchronously write many alerts into the database.
Writes are spread across upload workers. Each worker maintains a connection with S3 to write data. More workers can improve performance, until you add _so_ many that either switching between them becomes a bottleneck or S3 starts throttling you.
In addition, each worker has a fixed startup cost, so for very brief iterators of alerts, using a small number of workers will be faster overall.
- Parameters
alerts (
Iterator[AlertRecord]Iterator[AlertRecord]) – An iterator which produces the alerts to be written.n_worker (
intint) – The number of upload workers to use. Usually, this should be between about 2 and 10.
- Return type
NoneNone
-
__enter__()¶ Returns the Database itself so that it can be used in context managers.
-
__exit__(exc_type, exc_value, traceback)¶ Closes the Database when exiting from a context manager.
- Return type
bool|NoneOptional[bool]- Parameters
exc_type (Optional[Type[BaseException]]) –
exc_value (Optional[BaseException]) –
traceback (Optional[traceback]) –
-
classmethod
AlertRecord¶
An AlertRecord is a wrapper around a ZTF Alert. It provides a few utility functions, but for the most part it is used as a type that’s passed into and out of a
Database.
-
class
alertbase.AlertRecord(candidate_id, object_id, position, timestamp, raw_data, raw_dict)¶ -
classmethod
from_dict(d)¶ Constructs an AlertRecord from an avro-style dictionary representing a ZTF alert.
- Parameters
d ({
str:Any}Dict[str,Any]) – The dictionary to load from.- Return type
- Returns
The AlertRecord with all fields filled out.
-
classmethod
from_file_safe(fp)¶ Read from an Avro alert file stored on disk.
- Parameters
fp (
IO[bytes]IO[bytes]) – A file-like object to read raw bytes from for deserialization.- Return type
- Returns
The fully deserialized AlertRecord.
-
classmethod
from_file_unsafe(fp)¶ Read from an alert file stored on disk, recklessly making assumptions about its schema. This is much faster (~20-50x) than the safe call, but can yield errors, or severely incorrect data in the worst case.
The returned AlertRecord will have its
raw_datafield filled out, but notraw_dict.- Parameters
fp (
IO[bytes]IO[bytes]) – A file-like object to ead raw bytes from for deserialization.- Return type
- Returns
The deserialized AlertRecord.
-
candidate_id: int¶ The ZTF Candidate ID for this alert.
-
object_id: str¶ The ZTF Object ID for this alert.
-
position: astropy.coordinates.sky_coordinate.SkyCoord¶ The RA-Dec position of this alert.
-
raw_data: Optional[bytes]¶ The Avro serialization of this dalert’s data.
-
raw_dict: Optional[Dict[str, Any]]¶ The full alert payload.
-
timestamp: astropy.time.core.Time¶ The time of the exposure that sourced this alert.
-
classmethod