设计Mint.com


文档摘要

设计 Mint.com 注意:这个文档中的链接会直接指向系统设计主题索引中的有关部分,以避免重复的内容。您可以参考链接的相关内容,来了解其总的要点、方案的权衡取舍以及可选的替代方案。 第一步:简述用例与约束条件 搜集需求与问题的范围。 提出问题来明确用例与约束条件。 讨论假设。 我们将在没有面试官明确说明问题的情况下,自己定义一些用例以及限制条件。 用例 我们将把问题限定在仅处理以下用例的范围中 用户 连接到一个财务账户 服务 从账户中提取交易 每日更新 分类交易 允许用户手动分类 不自动重新分类 按类别分析每月支出 服务 推荐预算 允许用户手动设置预算 当接近或者超出预算时,发送通知 服务 具有高可用性 非用例范围 服务 执行附加的日志记录和分析 限制条件与假设 提出假设

设计 Mint.com

注意:这个文档中的链接会直接指向系统设计主题索引中的有关部分,以避免重复的内容。您可以参考链接的相关内容,来了解其总的要点、方案的权衡取舍以及可选的替代方案。

第一步:简述用例与约束条件

搜集需求与问题的范围。
提出问题来明确用例与约束条件。
讨论假设。

我们将在没有面试官明确说明问题的情况下,自己定义一些用例以及限制条件。

用例

我们将把问题限定在仅处理以下用例的范围中

  • 用户 连接到一个财务账户
  • 服务 从账户中提取交易
    • 每日更新
    • 分类交易
      • 允许用户手动分类
      • 不自动重新分类
    • 按类别分析每月支出
  • 服务 推荐预算
    • 允许用户手动设置预算
    • 当接近或者超出预算时,发送通知
  • 服务 具有高可用性

非用例范围

  • 服务 执行附加的日志记录和分析

限制条件与假设

提出假设

  • 网络流量非均匀分布
  • 自动账户日更新只适用于 30 天内活跃的用户
  • 添加或者移除财务账户相对较少
  • 预算通知不需要及时
  • 1000 万用户
    • 每个用户10个预算类别= 1亿个预算项
    • 示例类别:
      • Housing = $1,000
      • Food = $200
      • Gas = $100
    • 卖方确定交易类别
      • 50000 个卖方
  • 3000 万财务账户
  • 每月 50 亿交易
  • 每月 5 亿读请求
  • 10:1 读写比
    • Write-heavy,用户每天都进行交易,但是每天很少访问该网站

计算用量

如果你需要进行粗略的用量计算,请向你的面试官说明。

  • 每次交易的用量:
    • user_id - 8 字节
    • created_at - 5 字节
    • seller - 32 字节
    • amount - 5 字节
    • Total: ~50 字节
  • 每月产生 250 GB 新的交易内容
    • 每次交易 50 比特 * 50 亿交易每月
    • 3年内新的交易内容 9 TB
    • Assume most are new transactions instead of updates to existing ones
  • 平均每秒产生 2000 次交易
  • 平均每秒产生 200 读请求

便利换算指南:

  • 每个月有 250 万秒
  • 每秒一个请求 = 每个月 250 万次请求
  • 每秒 40 个请求 = 每个月 1 亿次请求
  • 每秒 400 个请求 = 每个月 10 亿次请求

第二步:概要设计

列出所有重要组件以规划概要设计。

Imgur

第三步:设计核心组件

深入每个核心组件的细节。

用例:用户连接到一个财务账户

我们可以将 1000 万用户的信息存储在一个关系数据库中。我们应该讨论一下选择SQL或NoSQL之间的用例和权衡了。

  • 客户端 作为一个反向代理,发送请求到 Web 服务器
  • Web 服务器 转发请求到 账户API 服务器
  • 账户API 服务器将新输入的账户信息更新到 SQL数据库accounts

告知你的面试官你准备写多少代码

accounts表应该具有如下结构:

id int NOT NULL AUTO_INCREMENT created_at datetime NOT NULL last_update datetime NOT NULL account_url varchar(255) NOT NULL account_login varchar(32) NOT NULL account_password_hash char(64) NOT NULL user_id int NOT NULL PRIMARY KEY(id) FOREIGN KEY(user_id) REFERENCES users(id)

我们将在iduser_idcreated_at等字段上创建一个索引以加速查找(对数时间而不是扫描整个表)并保持数据在内存中。从内存中顺序读取 1 MB数据花费大约250毫秒,而从SSD读取是其4倍,从磁盘读取是其80倍。1

我们将使用公开的REST API:

$ curl -X POST --data '{ "user_id": "foo", "account_url": "bar", \ "account_login": "baz", "account_password": "qux" }' \ https://mint.com/api/v1/account

对于内部通信,我们可以使用远程过程调用

接下来,服务从账户中提取交易。

用例:服务从账户中提取交易

如下几种情况下,我们会想要从账户中提取信息:

  • 用户首次链接账户
  • 用户手动更新账户
  • 为过去 30 天内活跃的用户自动日更新

数据流:

  • 客户端Web服务器 发送请求
  • Web服务器 将请求转发到 帐户API 服务器
  • 帐户API 服务器将job放在 队列 中,如 Amazon SQS 或者 RabbitMQ
    • 提取交易可能需要一段时间,我们可能希望与队列异步地来做,虽然这会引入额外的复杂度。
  • 交易提取服务 执行如下操作:
    • Queue 中拉取并从金融机构中提取给定用户的交易,将结果作为原始日志文件存储在 对象存储区
    • 使用 分类服务 来分类每个交易
    • 使用 预算服务 来按类别计算每月总支出
      • 预算服务 使用 通知服务 让用户知道他们是否接近或者已经超出预算
    • 更新具有分类交易的 SQL数据库transactions
    • 按类别更新 SQL数据库 monthly_spending表的每月总支出
    • 通过 通知服务 提醒用户交易完成
      • 使用一个 队列 (没有画出来) 来异步发送通知

transactions表应该具有如下结构:

id int NOT NULL AUTO_INCREMENT created_at datetime NOT NULL seller varchar(32) NOT NULL amount decimal NOT NULL user_id int NOT NULL PRIMARY KEY(id) FOREIGN KEY(user_id) REFERENCES users(id)

我们将在 iduser_id,和 created_at字段上创建索引

monthly_spending表应该具有如下结构:

id int NOT NULL AUTO_INCREMENT month_year date NOT NULL category varchar(32) amount decimal NOT NULL user_id int NOT NULL PRIMARY KEY(id) FOREIGN KEY(user_id) REFERENCES users(id)

我们将在iduser_id字段上创建索引

分类服务

对于 分类服务,我们可以生成一个带有最受欢迎卖家的卖家-类别字典。如果我们估计 50000 个卖家,并估计每个条目占用不少于 255 个字节,该字典只需要大约 12 MB内存。

告知你的面试官你准备写多少代码

class DefaultCategories(Enum): HOUSING = 0 FOOD = 1 GAS = 2 SHOPPING = 3 ... seller_category_map = {} seller_category_map['Exxon'] = DefaultCategories.GAS seller_category_map['Target'] = DefaultCategories.SHOPPING ...

对于一开始没有在映射中的卖家,我们可以通过评估用户提供的手动类别来进行众包。在 O(1) 时间内,我们可以用堆来快速查找每个卖家的顶端的手动覆盖。

class Categorizer(object): def __init__(self, seller_category_map, self.seller_category_crowd_overrides_map): self.seller_category_map = seller_category_map self.seller_category_crowd_overrides_map = \ seller_category_crowd_overrides_map def categorize(self, transaction): if transaction.seller in self.seller_category_map: return self.seller_category_map[transaction.seller] elif transaction.seller in self.seller_category_crowd_overrides_map: self.seller_category_map[transaction.seller] = \ self.seller_category_crowd_overrides_map[transaction.seller].peek_min() return self.seller_category_map[transaction.seller] return None

交易实现:

class Transaction(object): def __init__(self, created_at, seller, amount): self.timestamp = timestamp self.seller = seller self.amount = amount

用例:服务推荐预算

首先,我们可以使用根据收入等级分配每类别金额的通用预算模板。使用这种方法,我们不必存储在约束中标识的 1 亿个预算项目,只需存储用户覆盖的预算项目。如果用户覆盖预算类别,我们可以在
TABLE budget_overrides中存储此覆盖。

class Budget(object): def __init__(self, income): self.income = income self.categories_to_budget_map = self.create_budget_template() def create_budget_template(self): return { 'DefaultCategories.HOUSING': income * .4, 'DefaultCategories.FOOD': income * .2 'DefaultCategories.GAS': income * .1, 'DefaultCategories.SHOPPING': income * .2 ... } def override_category_budget(self, category, amount): self.categories_to_budget_map[category] = amount

对于 预算服务 而言,我们可以在transactions表上运行SQL查询以生成monthly_spending聚合表。由于用户通常每个月有很多交易,所以monthly_spending表的行数可能会少于总共50亿次交易的行数。

作为替代,我们可以在原始交易文件上运行 MapReduce 作业来:

  • 分类每个交易
  • 按类别生成每月总支出

对交易文件的运行分析可以显著减少数据库的负载。

如果用户更新类别,我们可以调用 预算服务 重新运行分析。

告知你的面试官你准备写多少代码.

日志文件格式样例,以tab分割:

user_id timestamp seller amount

MapReduce 实现:

class SpendingByCategory(MRJob): def __init__(self, categorizer): self.categorizer = categorizer self.current_year_month = calc_current_year_month() ... def calc_current_year_month(self): """返回当前年月""" ... def extract_year_month(self, timestamp): """返回时间戳的年,月部分""" ... def handle_budget_notifications(self, key, total): """如果接近或超出预算,调用通知API""" ... def mapper(self, _, line): """解析每个日志行,提取和转换相关行。 参数行应为如下形式: user_id timestamp seller amount 使用分类器来将卖家转换成类别,生成如下形式的key-value对: (user_id, 2016-01, shopping), 25 (user_id, 2016-01, shopping), 100 (user_id, 2016-01, gas), 50 """ user_id, timestamp, seller, amount = line.split('\t') category = self.categorizer.categorize(seller) period = self.extract_year_month(timestamp) if period == self.current_year_month: yield (user_id, period, category), amount def reducer(self, key, value): """将每个key对应的值求和。 (user_id, 2016-01, shopping), 125 (user_id, 2016-01, gas), 50 """ total = sum(values) yield key, sum(values)

第四步:设计扩展

根据限制条件,找到并解决瓶颈。

Imgur

重要提示:不要从最初设计直接跳到最终设计中!

现在你要 1) 基准测试、负载测试。2) 分析、描述性能瓶颈。3) 在解决瓶颈问题的同时,评估替代方案、权衡利弊。4) 重复以上步骤。请阅读「设计一个系统,并将其扩大到为数以百万计的 AWS 用户服务」 来了解如何逐步扩大初始设计。

讨论初始设计可能遇到的瓶颈及相关解决方案是很重要的。例如加上一个配置多台 Web 服务器负载均衡器是否能够解决问题?CDN呢?主从复制呢?它们各自的替代方案和需要权衡的利弊又有什么呢?

我们将会介绍一些组件来完成设计,并解决架构扩张问题。内置的负载均衡器将不做讨论以节省篇幅。

为了避免重复讨论,请参考系统设计主题索引相关部分来了解其要点、方案的权衡取舍以及可选的替代方案。

我们将增加一个额外的用例:用户 访问摘要和交易数据。

用户会话,按类别统计的统计信息,以及最近的事务可以放在 内存缓存(如 Redis 或 Memcached )中。

  • 客户端 发送读请求给 Web 服务器
  • Web 服务器 转发请求到 读 API 服务器
    • 静态内容可通过 对象存储 比如缓存在 CDN 上的 S3 来服务
  • 读 API 服务器做如下动作:
    • 检查 内存缓存 的内容
      • 如果URL在 内存缓存中,返回缓存的内容
      • 否则
        • 如果URL在 SQL 数据库中,获取该内容
          • 以其内容更新 内存缓存

参考 何时更新缓存 中权衡和替代的内容。以上方法描述了 cache-aside缓存模式.

我们可以使用诸如 Amazon Redshift 或者 Google BigQuery 等数据仓库解决方案,而不是将monthly_spending聚合表保留在 SQL 数据库 中。

我们可能只想在数据库中存储一个月的交易数据,而将其余数据存储在数据仓库或者 对象存储区 中。对象存储区 (如Amazon S3) 能够舒服地解决每月 250 GB新内容的限制。

为了解决每秒 平均 2000 次读请求数(峰值时更高),受欢迎的内容的流量应由 内存缓存 而不是数据库来处理。 内存缓存 也可用于处理不均匀分布的流量和流量尖峰。 只要副本不陷入重复写入的困境,SQL 读副本 应该能够处理高速缓存未命中。

平均 200 次交易写入每秒(峰值时更高)对于单个 SQL 写入主-从服务 来说可能是棘手的。我们可能需要考虑其它的 SQL 性能拓展技术:

我们也可以考虑将一些数据移至 NoSQL 数据库

其它要点

是否深入这些额外的主题,取决于你的问题范围和剩下的时间。

NoSQL

缓存

异步与微服务

通信

安全性

请参阅「安全」一章。

延迟数值

请参阅「每个程序员都应该知道的延迟数」

持续探讨

  • 持续进行基准测试并监控你的系统,以解决他们提出的瓶颈问题。
  • 架构拓展是一个迭代的过程。

Design Mint.com

Note: This document links directly to relevant areas found in the system design topics to avoid duplication. Refer to the linked content for general talking points, tradeoffs, and alternatives.

Step 1: Outline use cases and constraints

Gather requirements and scope the problem.
Ask questions to clarify use cases and constraints.
Discuss assumptions.

Without an interviewer to address clarifying questions, we'll define some use cases and constraints.

Use cases

We'll scope the problem to handle only the following use cases

  • User connects to a financial account
  • Service extracts transactions from the account
    • Updates daily
    • Categorizes transactions
      • Allows manual category override by the user
      • No automatic re-categorization
    • Analyzes monthly spending, by category
  • Service recommends a budget
    • Allows users to manually set a budget
    • Sends notifications when approaching or exceeding budget
  • Service has high availability

Out of scope

  • Service performs additional logging and analytics

Constraints and assumptions

State assumptions

  • Traffic is not evenly distributed
  • Automatic daily update of accounts applies only to users active in the past 30 days
  • Adding or removing financial accounts is relatively rare
  • Budget notifications don't need to be instant
  • 10 million users
    • 10 budget categories per user = 100 million budget items
    • Example categories:
      • Housing = $1,000
      • Food = $200
      • Gas = $100
    • Sellers are used to determine transaction category
      • 50,000 sellers
  • 30 million financial accounts
  • 5 billion transactions per month
  • 500 million read requests per month
  • 10:1 write to read ratio
    • Write-heavy, users make transactions daily, but few visit the site daily

Calculate usage

Clarify with your interviewer if you should run back-of-the-envelope usage calculations.

  • Size per transaction:
    • user_id - 8 bytes
    • created_at - 5 bytes
    • seller - 32 bytes
    • amount - 5 bytes
    • Total: ~50 bytes
  • 250 GB of new transaction content per month
    • 50 bytes per transaction * 5 billion transactions per month
    • 9 TB of new transaction content in 3 years
    • Assume most are new transactions instead of updates to existing ones
  • 2,000 transactions per second on average
  • 200 read requests per second on average

Handy conversion guide:

  • 2.5 million seconds per month
  • 1 request per second = 2.5 million requests per month
  • 40 requests per second = 100 million requests per month
  • 400 requests per second = 1 billion requests per month

Step 2: Create a high level design

Outline a high level design with all important components.

Imgur

Step 3: Design core components

Dive into details for each core component.

Use case: User connects to a financial account

We could store info on the 10 million users in a relational database. We should discuss the use cases and tradeoffs between choosing SQL or NoSQL.

  • The Client sends a request to the Web Server, running as a reverse proxy
  • The Web Server forwards the request to the Accounts API server
  • The Accounts API server updates the SQL Database accounts table with the newly entered account info

Clarify with your interviewer how much code you are expected to write.

The accounts table could have the following structure:

id int NOT NULL AUTO_INCREMENT created_at datetime NOT NULL last_update datetime NOT NULL account_url varchar(255) NOT NULL account_login varchar(32) NOT NULL account_password_hash char(64) NOT NULL user_id int NOT NULL PRIMARY KEY(id) FOREIGN KEY(user_id) REFERENCES users(id)

We'll create an index on id, user_id , and created_at to speed up lookups (log-time instead of scanning the entire table) and to keep the data in memory. Reading 1 MB sequentially from memory takes about 250 microseconds, while reading from SSD takes 4x and from disk takes 80x longer.1

We'll use a public REST API:

$ curl -X POST --data '{ "user_id": "foo", "account_url": "bar", \ "account_login": "baz", "account_password": "qux" }' \ https://mint.com/api/v1/account

For internal communications, we could use Remote Procedure Calls.

Next, the service extracts transactions from the account.

Use case: Service extracts transactions from the account

We'll want to extract information from an account in these cases:

  • The user first links the account
  • The user manually refreshes the account
  • Automatically each day for users who have been active in the past 30 days

Data flow:

  • The Client sends a request to the Web Server
  • The Web Server forwards the request to the Accounts API server
  • The Accounts API server places a job on a Queue such as Amazon SQS or RabbitMQ
    • Extracting transactions could take awhile, we'd probably want to do this asynchronously with a queue, although this introduces additional complexity
  • The Transaction Extraction Service does the following:
    • Pulls from the Queue and extracts transactions for the given account from the financial institution, storing the results as raw log files in the Object Store
    • Uses the Category Service to categorize each transaction
    • Uses the Budget Service to calculate aggregate monthly spending by category
      • The Budget Service uses the Notification Service to let users know if they are nearing or have exceeded their budget
    • Updates the SQL Database transactions table with categorized transactions
    • Updates the SQL Database monthly_spending table with aggregate monthly spending by category
    • Notifies the user the transactions have completed through the Notification Service:
      • Uses a Queue (not pictured) to asynchronously send out notifications

The transactions table could have the following structure:

id int NOT NULL AUTO_INCREMENT created_at datetime NOT NULL seller varchar(32) NOT NULL amount decimal NOT NULL user_id int NOT NULL PRIMARY KEY(id) FOREIGN KEY(user_id) REFERENCES users(id)

We'll create an index on id, user_id , and created_at.

The monthly_spending table could have the following structure:

id int NOT NULL AUTO_INCREMENT month_year date NOT NULL category varchar(32) amount decimal NOT NULL user_id int NOT NULL PRIMARY KEY(id) FOREIGN KEY(user_id) REFERENCES users(id)

We'll create an index on id and user_id .

Category service

For the Category Service, we can seed a seller-to-category dictionary with the most popular sellers. If we estimate 50,000 sellers and estimate each entry to take less than 255 bytes, the dictionary would only take about 12 MB of memory.

Clarify with your interviewer how much code you are expected to write.

class DefaultCategories(Enum): HOUSING = 0 FOOD = 1 GAS = 2 SHOPPING = 3 ... seller_category_map = {} seller_category_map['Exxon'] = DefaultCategories.GAS seller_category_map['Target'] = DefaultCategories.SHOPPING ...

For sellers not initially seeded in the map, we could use a crowdsourcing effort by evaluating the manual category overrides our users provide. We could use a heap to quickly lookup the top manual override per seller in O(1) time.

class Categorizer(object): def __init__(self, seller_category_map, seller_category_crowd_overrides_map): self.seller_category_map = seller_category_map self.seller_category_crowd_overrides_map = \ seller_category_crowd_overrides_map def categorize(self, transaction): if transaction.seller in self.seller_category_map: return self.seller_category_map[transaction.seller] elif transaction.seller in self.seller_category_crowd_overrides_map: self.seller_category_map[transaction.seller] = \ self.seller_category_crowd_overrides_map[transaction.seller].peek_min() return self.seller_category_map[transaction.seller] return None

Transaction implementation:

class Transaction(object): def __init__(self, created_at, seller, amount): self.created_at = created_at self.seller = seller self.amount = amount

Use case: Service recommends a budget

To start, we could use a generic budget template that allocates category amounts based on income tiers. Using this approach, we would not have to store the 100 million budget items identified in the constraints, only those that the user overrides. If a user overrides a budget category, which we could store the override in the TABLE budget_overrides.

class Budget(object): def __init__(self, income): self.income = income self.categories_to_budget_map = self.create_budget_template() def create_budget_template(self): return { DefaultCategories.HOUSING: self.income * .4, DefaultCategories.FOOD: self.income * .2, DefaultCategories.GAS: self.income * .1, DefaultCategories.SHOPPING: self.income * .2, ... } def override_category_budget(self, category, amount): self.categories_to_budget_map[category] = amount

For the Budget Service, we can potentially run SQL queries on the transactions table to generate the monthly_spending aggregate table. The monthly_spending table would likely have much fewer rows than the total 5 billion transactions, since users typically have many transactions per month.

As an alternative, we can run MapReduce jobs on the raw transaction files to:

  • Categorize each transaction
  • Generate aggregate monthly spending by category

Running analyses on the transaction files could significantly reduce the load on the database.

We could call the Budget Service to re-run the analysis if the user updates a category.

Clarify with your interviewer how much code you are expected to write.

Sample log file format, tab delimited:

user_id timestamp seller amount

MapReduce implementation:

class SpendingByCategory(MRJob): def __init__(self, categorizer): self.categorizer = categorizer self.current_year_month = calc_current_year_month() ... def calc_current_year_month(self): """Return the current year and month.""" ... def extract_year_month(self, timestamp): """Return the year and month portions of the timestamp.""" ... def handle_budget_notifications(self, key, total): """Call notification API if nearing or exceeded budget.""" ... def mapper(self, _, line): """Parse each log line, extract and transform relevant lines. Argument line will be of the form: user_id timestamp seller amount Using the categorizer to convert seller to category, emit key value pairs of the form: (user_id, 2016-01, shopping), 25 (user_id, 2016-01, shopping), 100 (user_id, 2016-01, gas), 50 """ user_id, timestamp, seller, amount = line.split('\t') category = self.categorizer.categorize(seller) period = self.extract_year_month(timestamp) if period == self.current_year_month: yield (user_id, period, category), amount def reducer(self, key, value): """Sum values for each key. (user_id, 2016-01, shopping), 125 (user_id, 2016-01, gas), 50 """ total = sum(values) yield key, sum(values)

Step 4: Scale the design

Identify and address bottlenecks, given the constraints.

Imgur

Important: Do not simply jump right into the final design from the initial design!

State you would 1) Benchmark/Load Test, 2) Profile for bottlenecks 3) address bottlenecks while evaluating alternatives and trade-offs, and 4) repeat. See Design a system that scales to millions of users on AWS as a sample on how to iteratively scale the initial design.

It's important to discuss what bottlenecks you might encounter with the initial design and how you might address each of them. For example, what issues are addressed by adding a Load Balancer with multiple Web Servers? CDN? Master-Slave Replicas? What are the alternatives and Trade-Offs for each?

We'll introduce some components to complete the design and to address scalability issues. Internal load balancers are not shown to reduce clutter.

To avoid repeating discussions, refer to the following system design topics for main talking points, tradeoffs, and alternatives:

We'll add an additional use case: User accesses summaries and transactions.

User sessions, aggregate stats by category, and recent transactions could be placed in a Memory Cache such as Redis or Memcached.

  • The Client sends a read request to the Web Server
  • The Web Server forwards the request to the Read API server
    • Static content can be served from the Object Store such as S3, which is cached on the CDN
  • The Read API server does the following:
    • Checks the Memory Cache for the content
      • If the url is in the Memory Cache, returns the cached contents
      • Else
        • If the url is in the SQL Database, fetches the contents
          • Updates the Memory Cache with the contents

Refer to When to update the cache for tradeoffs and alternatives. The approach above describes cache-aside.

Instead of keeping the monthly_spending aggregate table in the SQL Database, we could create a separate Analytics Database using a data warehousing solution such as Amazon Redshift or Google BigQuery.

We might only want to store a month of transactions data in the database, while storing the rest in a data warehouse or in an Object Store. An Object Store such as Amazon S3 can comfortably handle the constraint of 250 GB of new content per month.

To address the 200 average read requests per second (higher at peak), traffic for popular content should be handled by the Memory Cache instead of the database. The Memory Cache is also useful for handling the unevenly distributed traffic and traffic spikes. The SQL Read Replicas should be able to handle the cache misses, as long as the replicas are not bogged down with replicating writes.

2,000 average transaction writes per second (higher at peak) might be tough for a single SQL Write Master-Slave. We might need to employ additional SQL scaling patterns:

We should also consider moving some data to a NoSQL Database.

Additional talking points

Additional topics to dive into, depending on the problem scope and time remaining.

NoSQL

Caching

Asynchronism and microservices

Communications

Security

Refer to the security section.

Latency numbers

See Latency numbers every programmer should know.

Ongoing

  • Continue benchmarking and monitoring your system to address bottlenecks as they come up
  • Scaling is an iterative process

作者与出处
原作者: donnemartin
来源:donnemartin
许可证:CC BY 4.0
整理: 灏天文库整理
由灏天文库结构化整理,提供目录导航、全文检索与在线阅读,便于系统化学习
发布者: 作者: donnemartin 转发
评论区 (0)
U