I think it depends on the details and performance requirements. Otherwise the general sentiment of 'use any of the major technologies, one you're familiar with' stands.
For example, is this software aimed at the end user or will the startup use it themselves? If the end user will use it (like alerts for their portfolio/watch list), then the definition of "real-time" is going to be lax because you need human interaction. If it's used by an institution for trading, then you will need to choose highly performant language, architecture, and hardware so that your algorithm can place it's orders before other institutions. You would also likely need to integrate with Swift (financial).
Any major tech stack will work fine. I assume “real time alerts” will need an integration with a 3rd-party notification provider (email, sms, etc) and the bulk of your business logic will be defined with whoever is providing your share price API.
So, tech doesn’t really matter, pick something simple and mainstream so you can get help as needed.
Ruby/Rails, Python/Django, Nodejs, Java, whatever. Postgres for a database.
You likely won’t need any exotic cool tech, novel databases etc.
Take in consideration to not use floats for money. Use a `decimal` data format. I'd also advice against using javascript since it has problems dealing with big numbers. There are libraries of course to handle that.
Just if you use JS make sure to google about how to do money calculations accurately.
I see this statement a lot and would like to see it better illustrated or quantified.
I worked at a place that used doubles and there was no fallout. Also when dealing with very small quantities, fixed point might not work so well either.
Personally, as a general rule I use integer's and convert for display only (as mentioned in the above SO post). This removes the accuracy issue and works in every programming language that is in common use today (at least that I know of). I even do this in Postgres (and other DB's) which keeps everything consistent and removes the chance a driver between the DB and client jacks up the decimal conversion.
JVM or python just for the ecosystem.
kafka, debezium, workflow engines(camunda), ORMs etc - java libraries might not be as nice to use as ruby gems but some are really rock solid for your use-case.
But that being said, anything really - rails/node etc most would work just fine for your use-case
I didn't find debezium production ready when I tried it about a year ago. If I recall, no cascading deletes and it can't handle 0000-00-00 dates due to "some java limitation" (maybe in the kafka producer? I honestly forget) and when a dev on my team made a PR to the debezium project it was not merged because the team felt it was outside of scope for debezium. Also, because of implementation details, it will never do cascades.
End result was a lot of wasted effort. It worked for one team who didn't have those requirements and even then they spent more time than they should have and then ripped out debezium and went with standard mysql replication as it doesn't require the entire kafka stack.
My experience with it reaffirmed my standard technique of data migration of write to both and move reads over when ready.
I'm a fan of Kafka for streaming systems, so I'd recommend to fetch data via api, add it to kafka and then consume from kafka to monitor the price + fire alerts (assuming you don't really need a real-time system with very low latency).
What language you use is up to you/your team. If you like python, I'd look into Airflow or Celery as frameworks to build your workflows.
For java/scala I'd take a look at Google Dataflow or Flink or Kafka Streams, depending on how heavy the "monitor share prices" part is.
I think the major issue you're going to encounter is not building a system that works, but building a process that allows your team to upgrade the system, test new features, debug, etc.
Dataflow, for example, lets you run more or less the same code both on a stream of data and on batches of historical data.
- The tech stack that is best is the one you/team are most comfortable with using.
Specific Answer:
- If you were using Elixir, you could have 1 process (GenServer) per stock ticker that calls the API for just that stock. GenServers are lightweight processes that run concurrently in the BEAM. That process could then have a set of subscribing user processes to notify which would then do an alert to a user. This is assuming you can call an API for one stock vs. having to parse a feed of data. If I were doing this I wouldn't store any of the stock data in any way, its ephemeral and by the time a DB commit happens the price changed.
Caveats:
You mention API so I assume that means you are calling an API, parsing data, (dealing with rate limits) then alerting. I'd caution the use of the term "real-time" as you are farther away from trading data than big name brokers. My intent isn't to downplay the idea, just thinking it through based on the info in the one sentence question.
I would use the new k(9) if I have a small team and we can pick. Otherwise .NET Core 5. We use this for all our fintech stuff and it is very good and 'enterprises approved'. We use it with Postgres and Redis.
I work in fintech; we use a mix of Scala and Node. Scala for heavy data processing pipelines, Node for our client API, due to Node’s very fast cold start times in lambda.
Readit News
For example, is this software aimed at the end user or will the startup use it themselves? If the end user will use it (like alerts for their portfolio/watch list), then the definition of "real-time" is going to be lax because you need human interaction. If it's used by an institution for trading, then you will need to choose highly performant language, architecture, and hardware so that your algorithm can place it's orders before other institutions. You would also likely need to integrate with Swift (financial).
So, tech doesn’t really matter, pick something simple and mainstream so you can get help as needed.
Ruby/Rails, Python/Django, Nodejs, Java, whatever. Postgres for a database.
You likely won’t need any exotic cool tech, novel databases etc.
Just if you use JS make sure to google about how to do money calculations accurately.
I worked at a place that used doubles and there was no fallout. Also when dealing with very small quantities, fixed point might not work so well either.
Personally, as a general rule I use integer's and convert for display only (as mentioned in the above SO post). This removes the accuracy issue and works in every programming language that is in common use today (at least that I know of). I even do this in Postgres (and other DB's) which keeps everything consistent and removes the chance a driver between the DB and client jacks up the decimal conversion.
But that being said, anything really - rails/node etc most would work just fine for your use-case
End result was a lot of wasted effort. It worked for one team who didn't have those requirements and even then they spent more time than they should have and then ripped out debezium and went with standard mysql replication as it doesn't require the entire kafka stack.
My experience with it reaffirmed my standard technique of data migration of write to both and move reads over when ready.
What language you use is up to you/your team. If you like python, I'd look into Airflow or Celery as frameworks to build your workflows.
For java/scala I'd take a look at Google Dataflow or Flink or Kafka Streams, depending on how heavy the "monitor share prices" part is.
I think the major issue you're going to encounter is not building a system that works, but building a process that allows your team to upgrade the system, test new features, debug, etc.
Dataflow, for example, lets you run more or less the same code both on a stream of data and on batches of historical data.
- The tech stack that is best is the one you/team are most comfortable with using.
Specific Answer:
- If you were using Elixir, you could have 1 process (GenServer) per stock ticker that calls the API for just that stock. GenServers are lightweight processes that run concurrently in the BEAM. That process could then have a set of subscribing user processes to notify which would then do an alert to a user. This is assuming you can call an API for one stock vs. having to parse a feed of data. If I were doing this I wouldn't store any of the stock data in any way, its ephemeral and by the time a DB commit happens the price changed.
Caveats:
You mention API so I assume that means you are calling an API, parsing data, (dealing with rate limits) then alerting. I'd caution the use of the term "real-time" as you are farther away from trading data than big name brokers. My intent isn't to downplay the idea, just thinking it through based on the info in the one sentence question.