sslivkoff
|
e8c046fe7c
|
fix defillama stablecoins schema
|
2025-10-03 15:32:01 -07:00 |
|
sslivkoff
|
ef08c9df48
|
add retries and graceful skips to defillama.fees
|
2025-10-03 15:31:52 -07:00 |
|
sslivkoff
|
1a37adcffa
|
update kalshi collectors
|
2025-09-26 19:07:09 -07:00 |
|
sslivkoff
|
75c8076b2b
|
fix chunk size check in path generation
|
2025-09-26 19:06:59 -07:00 |
|
sslivkoff
|
be88e0c500
|
update defillama schema
|
2025-09-26 19:06:42 -07:00 |
|
sslivkoff
|
02922990fe
|
fix divide by zero edgecase
|
2025-09-26 19:06:29 -07:00 |
|
sslivkoff
|
5c77376e4f
|
update timezone checks
|
2025-09-26 19:06:21 -07:00 |
|
sslivkoff
|
977e921c1b
|
bump to 0.3.1
|
2025-08-07 23:12:28 -07:00 |
|
sslivkoff
|
094764d6dd
|
Update README.md
|
2025-08-07 23:11:35 -07:00 |
|
sslivkoff
|
7eca8a3a38
|
improve ls verbosity levels
|
2025-08-06 16:08:41 -07:00 |
|
sslivkoff
|
a62deea1c2
|
further refinement of Table api and toplevel api
|
2025-08-06 15:59:21 -07:00 |
|
sslivkoff
|
5af4654e53
|
add print_table_info as toplevel function
|
2025-08-06 15:03:35 -07:00 |
|
sslivkoff
|
e0dad1f8bf
|
clean up python api and add cli<-->python equivalence
|
2025-08-06 14:51:47 -07:00 |
|
sslivkoff
|
232230db25
|
revise toplevel absorb.__init__ functions
|
2025-08-06 13:44:11 -07:00 |
|
sslivkoff
|
8baa6c0c9b
|
update readme
|
2025-08-06 13:28:47 -07:00 |
|
sslivkoff
|
2085a821c4
|
improve narration of bucket uploads/downloads
|
2025-08-01 15:30:21 -07:00 |
|
sslivkoff
|
eb98a17463
|
add better bucket setup
|
2025-08-01 15:29:58 -07:00 |
|
sslivkoff
|
b3bf46d587
|
clean verbose output of abs collect
|
2025-08-01 12:03:25 -07:00 |
|
sslivkoff
|
75e5bc7c2d
|
validate that when write_range=overwrite_all there is at most 1 data file
|
2025-08-01 01:20:14 -07:00 |
|
sslivkoff
|
82b40a7458
|
clear old data after collection when write_range=overwrite_all
|
2025-08-01 01:14:26 -07:00 |
|
sslivkoff
|
1259dad121
|
chunk_glob --> data_glob
|
2025-08-01 01:00:36 -07:00 |
|
sslivkoff
|
bd73b70350
|
clean output of abs info
|
2025-08-01 00:57:01 -07:00 |
|
sslivkoff
|
72c16f5a5d
|
make query eager unless lazy=True
|
2025-08-01 00:44:46 -07:00 |
|
sslivkoff
|
7c2309723a
|
standardize allium timestamp column
|
2025-08-01 00:12:55 -07:00 |
|
sslivkoff
|
72aba68d9f
|
check if rclone_remote is set
|
2025-08-01 00:12:28 -07:00 |
|
sslivkoff
|
425f4bc78d
|
add in-repo checks for git tracking
|
2025-08-01 00:12:10 -07:00 |
|
sslivkoff
|
9d3d918784
|
change ls formatting to trim overflow of each source line when nonverbose
|
2025-07-29 23:49:34 -07:00 |
|
sslivkoff
|
0c4be15dfd
|
more index unifying
|
2025-07-29 22:58:28 -07:00 |
|
sslivkoff
|
8aebb9fc74
|
refine the flowchart of update time, latency, and other temporal table properties
|
2025-07-29 22:58:15 -07:00 |
|
sslivkoff
|
29ded7d8e2
|
add --provider arg to upload and download
|
2025-07-28 15:04:18 -07:00 |
|
sslivkoff
|
fee84ff010
|
unify all tables on same timestamp name, precision, and tz
|
2025-07-28 15:04:08 -07:00 |
|
sslivkoff
|
8ba4956b1e
|
separate dynamic table properties into special methods
|
2025-07-27 20:59:46 -07:00 |
|
sslivkoff
|
128496f0d9
|
make function names more specific
|
2025-07-27 17:57:50 -07:00 |
|
sslivkoff
|
1060882324
|
clean bucket pathing up
|
2025-07-27 17:45:52 -07:00 |
|
sslivkoff
|
d3ab784089
|
clean up path functions
|
2025-07-27 17:25:18 -07:00 |
|
sslivkoff
|
46050d40cb
|
organize config ops by module
|
2025-07-27 13:33:34 -07:00 |
|
sslivkoff
|
1c7ab33945
|
ChunkData --> ChunkResult
|
2025-07-27 12:44:54 -07:00 |
|
sslivkoff
|
a5510bcf55
|
refactor indexing to use much cleaner abstractions
|
2025-07-27 12:11:23 -07:00 |
|
sslivkoff
|
8dd7d1cfd7
|
when parsing table strs, check config before using general search
|
2025-07-26 10:06:58 -07:00 |
|
sslivkoff
|
6d189de5ed
|
improve validation for missing metadata
|
2025-07-26 09:51:52 -07:00 |
|
sslivkoff
|
a417abeef8
|
unify add and remove of cli/python interfaces
|
2025-07-26 09:43:29 -07:00 |
|
sslivkoff
|
b1fda4b826
|
add bucket scanning directly to query()
|
2025-07-26 08:08:31 -07:00 |
|
sslivkoff
|
054210e2a2
|
bump to 0.3.0
|
2025-07-25 23:50:27 -07:00 |
|
sslivkoff
|
c12632b367
|
add [all] option for installations
|
2025-07-25 23:50:12 -07:00 |
|
sslivkoff
|
a124c45de7
|
split types module into cleaner set of submodules
|
2025-07-25 23:45:45 -07:00 |
|
sslivkoff
|
159380bb15
|
datasets --> catalog
|
2025-07-25 23:41:38 -07:00 |
|
sslivkoff
|
5d3e4a6d07
|
add config path to setup output
|
2025-07-25 23:28:09 -07:00 |
|
sslivkoff
|
1122ffd60e
|
change key bucket-->bucket_name
|
2025-07-25 16:31:43 -07:00 |
|
sslivkoff
|
c19446e059
|
enable querying buckets directly
|
2025-07-25 16:24:58 -07:00 |
|
sslivkoff
|
9e91228a02
|
add config validation check for dict subkeys
|
2025-07-25 16:15:43 -07:00 |
|
sslivkoff
|
745377414a
|
fix FRED timestamp datatype
|
2025-07-25 10:28:57 -07:00 |
|
sslivkoff
|
d704bee71a
|
allow preview to preview uncollected datasets
|
2025-07-25 10:28:38 -07:00 |
|
sslivkoff
|
3a98654acc
|
simplify table reference parameter flow
|
2025-07-25 10:22:50 -07:00 |
|
sslivkoff
|
5a01cb5ee6
|
implement query() which collects uncollected before querying and can update
|
2025-07-25 09:56:44 -07:00 |
|
sslivkoff
|
4cd4e9de14
|
add more descriptive error handling and explicit message for --confirm
|
2025-07-25 09:46:20 -07:00 |
|
sslivkoff
|
3e001b3840
|
add name validation to table instantiation
|
2025-07-25 01:05:00 -07:00 |
|
sslivkoff
|
8351aeb1d6
|
support dune and snowflake backends for sql queries
|
2025-07-25 00:30:32 -07:00 |
|
sslivkoff
|
652795853a
|
add cli sql subcommand
|
2025-07-25 00:16:30 -07:00 |
|
sslivkoff
|
0da111d266
|
add ability to upload and download tables from buckets
|
2025-07-24 23:32:06 -07:00 |
|
sslivkoff
|
7ae6353d4a
|
print config in abs setup
|
2025-07-24 23:19:43 -07:00 |
|
sslivkoff
|
a453483926
|
add name collision check when tracking new tables
|
2025-07-24 00:36:08 -07:00 |
|
sslivkoff
|
6be5030400
|
allow info recipe
|
2025-07-24 00:30:18 -07:00 |
|
sslivkoff
|
96cb9c1c31
|
allow info source subcommand
|
2025-07-23 23:36:40 -07:00 |
|
sslivkoff
|
5409ff03af
|
name --> id
|
2025-07-23 23:03:01 -07:00 |
|
sslivkoff
|
516e12bd71
|
add enable and disable git operations
|
2025-07-23 22:58:14 -07:00 |
|
sslivkoff
|
d7764d50ce
|
add --verbose to info
|
2025-07-23 22:35:00 -07:00 |
|
sslivkoff
|
14e148894e
|
add url to abs info output
|
2025-07-23 14:32:43 -07:00 |
|
sslivkoff
|
302dc13ddf
|
implement auto schema fetching for snowflake tables
|
2025-07-23 14:30:53 -07:00 |
|
sslivkoff
|
dd2a5163f5
|
add url and description metadata to tables
|
2025-07-23 14:08:42 -07:00 |
|
sslivkoff
|
9a6f4fd016
|
add proper checks around whether index_type is None for certain operations
|
2025-07-23 12:16:17 -07:00 |
|
sslivkoff
|
e18e3e1f32
|
add allium.stablecoin_supply table
|
2025-07-22 18:17:22 -07:00 |
|
sslivkoff
|
03bc7b75ba
|
update snowflake flow
|
2025-07-22 18:17:02 -07:00 |
|
sslivkoff
|
dcc8424d2c
|
improve metavar name of cli args
|
2025-07-22 15:46:59 -07:00 |
|
sslivkoff
|
e7d8fa777b
|
remove redundant path manipulations
|
2025-07-22 15:34:30 -07:00 |
|
sslivkoff
|
17f0acb945
|
introduce full_name() where optimal
|
2025-07-22 14:35:10 -07:00 |
|
sslivkoff
|
8864492f2c
|
put all metadata changes under version control
|
2025-07-22 14:27:37 -07:00 |
|
sslivkoff
|
a0f357adae
|
add background git tracking of table metadatas
|
2025-07-22 00:53:11 -07:00 |
|
sslivkoff
|
e9968b9bbd
|
fix dune schema
|
2025-07-22 00:12:42 -07:00 |
|
sslivkoff
|
7ed879ec58
|
enable Table.collect_chunk to write files to shortcircuit dataframes
|
2025-07-22 00:04:12 -07:00 |
|
sslivkoff
|
2618adc90c
|
add validation checks to collected chunks
|
2025-07-21 23:42:57 -07:00 |
|
sslivkoff
|
4034cbe8ae
|
add warnings for missing packages/credentials when adding new table
|
2025-07-21 23:33:51 -07:00 |
|
sslivkoff
|
f2db762e2a
|
add --delete option to abs remove
|
2025-07-21 23:21:45 -07:00 |
|
sslivkoff
|
cb14fd22e7
|
add global --absorb-root arg and set_absorb_root() function
|
2025-07-21 23:12:34 -07:00 |
|
sslivkoff
|
c51eaf1315
|
add Table.description field
|
2025-07-21 23:12:02 -07:00 |
|
sslivkoff
|
a6463dd47e
|
refactor Table by splitting out creation + naming functions
|
2025-07-21 22:16:57 -07:00 |
|
sslivkoff
|
1149c85bd0
|
get_dir_path --> get_table_path
|
2025-07-21 21:42:40 -07:00 |
|
sslivkoff
|
19ce5c27b6
|
update spice version
|
2025-07-21 21:42:17 -07:00 |
|
sslivkoff
|
46779592cb
|
add credential checks prior to data collection
|
2025-07-21 13:26:50 -07:00 |
|
sslivkoff
|
53702757b1
|
add proper req's to dune datasets
|
2025-07-21 10:22:43 -07:00 |
|
sslivkoff
|
7aa77db9f3
|
clean up metadata path formulation
|
2025-07-21 00:25:31 -07:00 |
|
sslivkoff
|
54f389b5c3
|
add placeholder available ranges for binance markets
|
2025-07-20 20:37:49 -07:00 |
|
sslivkoff
|
abcb78808c
|
fix specific instance name instead of class name in tracked ls
|
2025-07-20 20:37:36 -07:00 |
|
sslivkoff
|
2631b9e5db
|
improve dynamic name parsing
|
2025-07-20 20:29:43 -07:00 |
|
sslivkoff
|
12f5751c0d
|
add better error handling
|
2025-07-20 20:29:27 -07:00 |
|
sslivkoff
|
1d5d15eafb
|
use placeholder for binance available range
|
2025-07-20 20:28:43 -07:00 |
|
sslivkoff
|
48e79c9eb7
|
simplify much of the table parsing logic
|
2025-07-20 17:43:31 -07:00 |
|
sslivkoff
|
919cf25e9b
|
enable interactive data previews
|
2025-07-20 14:24:18 -07:00 |
|
sslivkoff
|
1e4d63f563
|
fix slow test by hardcoding schema instead of using dynamic
|
2025-07-20 13:56:12 -07:00 |
|
sslivkoff
|
8ff79a1ee1
|
refactor TableDict handling and flow
|
2025-07-20 13:37:26 -07:00 |
|
sslivkoff
|
13fba0ac5e
|
separate validation logic into separate module
|
2025-07-20 12:42:36 -07:00 |
|
sslivkoff
|
e9e28fe780
|
load table parameters from filesystem instead of regeneration
|
2025-07-20 11:58:25 -07:00 |
|
sslivkoff
|
15ce34d64d
|
add config regeneration to setup
|
2025-07-19 13:25:55 -07:00 |
|
sslivkoff
|
6199a194a6
|
add validity checks for tracked metadata
|
2025-07-19 00:09:51 -07:00 |
|
sslivkoff
|
c920d8a9bf
|
enforce that parameter_types are strict types rather than annotations
|
2025-07-19 00:09:33 -07:00 |
|
sslivkoff
|
0398162960
|
enforce TableDict parameters are JSON-serializable
|
2025-07-19 00:09:33 -07:00 |
|
sslivkoff
|
d503db01f9
|
use better table name template matching
|
2025-07-18 23:09:50 -07:00 |
|
sslivkoff
|
83d97defbf
|
add more data integrity checks
|
2025-07-18 22:59:06 -07:00 |
|
sslivkoff
|
b18635244f
|
clean up binance names
|
2025-07-18 22:04:31 -07:00 |
|
sslivkoff
|
2d1b513fc9
|
refactor default parameter system
|
2025-07-18 22:02:17 -07:00 |
|
sslivkoff
|
b8041b9d35
|
add setup and validate subcommands
|
2025-07-18 21:18:13 -07:00 |
|
sslivkoff
|
7a75efb0cc
|
comments in Table for clarity
|
2025-07-18 17:24:35 -07:00 |
|
sslivkoff
|
8ff36118a0
|
track table metadata in filesystem
|
2025-07-18 16:55:30 -07:00 |
|
sslivkoff
|
5db2df2e0f
|
track table versioning
|
2025-07-18 16:55:06 -07:00 |
|
sslivkoff
|
af22856247
|
remove struct fields from kalshi metadata
|
2025-07-18 16:54:46 -07:00 |
|
sslivkoff
|
22c2a36501
|
rename chain_ids.py --> chains.py
|
2025-07-18 13:44:09 -07:00 |
|
sslivkoff
|
1cdb1db7af
|
fix cd hidden arguments
|
2025-07-18 13:18:13 -07:00 |
|
sslivkoff
|
b7b6c8bed4
|
add abs preview
|
2025-07-16 17:54:09 -07:00 |
|
sslivkoff
|
8b38174917
|
TrackedTable --> TableDict
|
2025-07-16 16:58:21 -07:00 |
|
sslivkoff
|
320e7b37b7
|
fix cd setup instructions
|
2025-07-16 16:48:08 -07:00 |
|
sslivkoff
|
8ebcbbe433
|
improve ls outputs
|
2025-07-16 16:40:08 -07:00 |
|
sslivkoff
|
65b186c4b7
|
add tables doc
|
2025-07-16 15:58:38 -07:00 |
|
sslivkoff
|
2eb4d323d7
|
make annotations consistent
|
2025-07-15 22:02:56 -07:00 |
|
sslivkoff
|
210791934c
|
add initial message about number of datasets being collected
|
2025-07-15 15:43:16 -07:00 |
|
sslivkoff
|
093d5a1e0f
|
add more automation of dateset updates
|
2025-07-15 13:23:40 -07:00 |
|
sslivkoff
|
149bf96e57
|
rearrange dependencies
|
2025-07-15 12:26:23 -07:00 |
|
sslivkoff
|
2a44511e28
|
add package installation checks
|
2025-07-13 08:50:24 -07:00 |
|
sslivkoff
|
9a1a90a187
|
add status info to info subcommand
|
2025-07-12 22:05:27 -07:00 |
|
sslivkoff
|
f8124ef538
|
add tests for get_schema()
|
2025-07-12 21:45:30 -07:00 |
|
sslivkoff
|
ae1cb63113
|
clean toplevel module imports
|
2025-07-12 21:22:56 -07:00 |
|
sslivkoff
|
94252352a4
|
import polars as needed
|
2025-07-12 21:22:44 -07:00 |
|
sslivkoff
|
24507797aa
|
run experimental ty type checker
|
2025-07-12 20:15:39 -07:00 |
|
sslivkoff
|
c1a81e2664
|
remove flipside as data source
|
2025-07-12 19:47:53 -07:00 |
|
sslivkoff
|
5e2fc501eb
|
simply name formulation logic
|
2025-07-12 10:04:39 -07:00 |
|
sslivkoff
|
5db9fb8bae
|
fix class searching for source-specific parsing
|
2025-07-11 20:17:07 -07:00 |
|
sslivkoff
|
5aa39dcd18
|
use individual class in dateset parser
|
2025-07-11 19:28:17 -07:00 |
|
sslivkoff
|
234d516db9
|
fix range diff tests
|
2025-07-11 19:27:44 -07:00 |
|
sslivkoff
|
e6334e9a5a
|
add snake camel conversion tests
|
2025-07-11 19:18:08 -07:00 |
|
sslivkoff
|
9fc5a6e69f
|
sort ls entries
|
2025-07-11 19:08:17 -07:00 |
|
sslivkoff
|
a4c716df07
|
modularize dataset parsing
|
2025-07-11 18:26:21 -07:00 |
|
sslivkoff
|
081b3ca444
|
update l2beat schema
|
2025-07-11 18:26:12 -07:00 |
|
sslivkoff
|
a33a9b1f01
|
add source argument to ls
|
2025-07-11 09:57:46 -07:00 |
|
sslivkoff
|
2bc8c8f46a
|
add --collected argument to add
|
2025-07-11 09:41:29 -07:00 |
|
sslivkoff
|
c5236b7e8b
|
formatting
|
2025-07-10 21:26:19 -07:00 |
|
sslivkoff
|
2129e6598e
|
in coverage_to_list, also convert lists of ranges to simple lists
|
2025-07-10 21:26:06 -07:00 |
|
sslivkoff
|
4c12a2b7ca
|
fix error message variables
|
2025-07-10 21:25:23 -07:00 |
|
sslivkoff
|
2d4fbdec00
|
add cd subcommand
|
2025-07-10 21:25:08 -07:00 |
|
sslivkoff
|
0dbcd51c5a
|
add color to tracebacks if avaialable
|
2025-07-10 12:44:12 -07:00 |
|
sslivkoff
|
ce1971b15b
|
get debugger dependency in proper group
|
2025-07-09 23:28:09 -07:00 |
|
sslivkoff
|
87e1736187
|
git ignore pytestdebug.log
|
2025-07-09 23:06:40 -07:00 |
|
sslivkoff
|
73432326e4
|
clean up types for mypy
|
2025-07-09 21:30:18 -07:00 |
|
sslivkoff
|
79e5438066
|
add optional deps to pyproject.toml
|
2025-07-09 18:11:50 -07:00 |
|
sslivkoff
|
ea60121b77
|
change help subcommand to info
|
2025-07-09 16:30:11 -07:00 |
|
sslivkoff
|
b2a4150317
|
add mypy as dev dependency
|
2025-07-09 16:29:59 -07:00 |
|
sslivkoff
|
381ced522f
|
change TRUCK_ROOT to ABSORB_ROOT
|
2025-07-09 14:02:23 -07:00 |
|
sslivkoff
|
b88c196276
|
update missing env var warning
|
2025-07-09 13:51:02 -07:00 |
|
sslivkoff
|
cf6c0bcee9
|
update pyproject.toml for abs executable and venv config
|
2025-07-09 13:50:16 -07:00 |
|
sslivkoff
|
e3ef4827a0
|
short circuit collected table getter if root DNE
|
2025-07-07 16:37:14 -07:00 |
|
sslivkoff
|
90495dfc57
|
update min python version to satisfy uv dependency resolver
|
2025-07-07 16:36:46 -07:00 |
|
sslivkoff
|
3f98f48dfe
|
ignore uv versioning files
|
2025-07-07 16:36:26 -07:00 |
|
sslivkoff
|
0c547a919d
|
add collected untracked datasets to ls
|
2025-06-22 17:40:41 -07:00 |
|
sslivkoff
|
5f29b6bd19
|
use index_type=None for coingecko.categories
|
2025-06-22 17:37:15 -07:00 |
|
sslivkoff
|
22f000428a
|
for overwrite_all datasets use None as default available range
|
2025-06-22 17:35:26 -07:00 |
|
sslivkoff
|
d17002d09e
|
accomodate non-temporally indexed overwrite_all datasets (#1)
|
2025-06-12 10:09:29 +02:00 |
|
sslivkoff
|
eb8e19a8b7
|
add pair column to binance tables
|
2025-06-01 10:31:44 -07:00 |
|