devttl/xiaozhi-esp32
1
0
Fork 0
forked from xiaozhi/xiaozhi-esp32
Code Pull Requests Activity

Add camera support for Otto Robot board (#1520)

Browse Source 
* otto v1.4.0 MCP

1.使用MCP协议控制机器人
2.gif继承lcdDisplay，避免修改lcdDisplay

* otto v1.4.1 gif as components

gif as components

* electronBot v1.1.0 mcp

1.增加electronBot支持
2.mcp协议
3.gif 作为组件
4.display子类

* 规范代码

1.规范代码
2.修复切换主题死机bug

* fix(ota): 修复 ottoRobot和electronBot OTA 升级崩溃问题 bug

* 1.增加robot舵机初始位置校准
2.fix(mcp_sever) 超出范围异常捕获类型  bug

* refactor: Update Electron and Otto emoji display implementations

- Removed GIF selection from Kconfig for Electron and Otto boards.
- Updated Electron and Otto bot versions to 2.0.4 in their respective config files.
- Refactored emoji display classes to utilize EmojiCollection for managing emojis.
- Enhanced chat label setup and status display functionality in both classes.
- Cleaned up unused code and improved initialization logging for emoji displays.

* Rename OTTO_ICON_FONT.c to otto_icon_font.c

* Rename OTTO_ICON_FONT.c to otto_icon_font.c

* refactor: Update Otto emoji display configurations and functionalities

- Changed chat label text mode to circular scrolling for both Otto and Electron emoji displays.
- Bumped Otto robot version to 2.0.5 in the configuration file.
- Added new actions for Otto robot including Sit, WhirlwindLeg, Fitness, Greeting, Shy, RadioCalisthenics, MagicCircle, and Showcase.
- Enhanced servo sequence handling and added support for executing custom servo sequences.
- Improved logging and error handling for servo sequence execution.

* refactor: Update chat label long mode for Electron and Otto emoji displays

- Changed chat label text mode from wrap to circular scrolling for both Electron and Otto emoji displays.
- Improved consistency in chat label setup across both implementations.

* Update Otto robot README with new actions and parameters

* Update Otto controller parameters for oscillation settings

- Changed default oscillation period from 500ms to 300ms.
- Increased default steps from 5.0 to 8.0.
- Updated default amplitude from 20 degrees to 0 degrees.
- Enhanced documentation with new examples for oscillation modes and sequences.

* Fix default amplitude initialization in Otto controller to use a single zero instead of two digits.

* chore: update txp666/otto-emoji-gif-component version to 1.0.3 in idf_component.yml

* Refactor Otto controller
- Consolidated movement actions into a unified tool for the Otto robot, allowing for a single action command with various parameters.
- Removed individual movement tools (walk, turn, jump, etc.) and replaced them with a more flexible action system.

* Enhance Otto robot functionality by adding WebSocket control server and IP address retrieval feature. Updated config to support WebSocket, and revised README to include new control options and usage examples.

* Add camera support for Otto Robot board

- Introduced configuration option to enable the Otto Robot camera in Kconfig.
- Updated config.h to define camera-related GPIO pins and settings.
- Modified config.json to include camera configuration.
- Enhanced otto_robot.cc to initialize I2C and camera components when the camera is enabled.
- Adjusted power_manager.h to manage battery updates during camera operations.
- Removed unused SetupChatLabel method from OttoEmojiDisplay class.
This commit is contained in:
小鹏
2025-12-05 23:05:44 +08:00
committed by GitHub
parent d7c1aef77a
commit f9de29519b
8 changed files with 207 additions and 40 deletions
Download Patch File Download Diff File Expand all files Collapse all files

7
main/Kconfig.projbuild
View File

@@ -690,6 +690,13 @@ config RECEIVE_CUSTOM_MESSAGE
help
Enable custom message reception, allow the device to receive custom messages from the server (preferably through the MQTT protocol)
config OTTO_ROBOT_USE_CAMERA
bool "Enable Otto Robot Camera"
default n
depends on BOARD_TYPE_OTTO_ROBOT
help
Enable Otto Robot Camera
menu "Camera Configuration"
depends on !IDF_TARGET_ESP32

59
main/boards/otto-robot/config.h
View File

@@ -3,6 +3,60 @@
#include <driver/gpio.h>
#if CONFIG_OTTO_ROBOT_USE_CAMERA
#define POWER_CHARGE_DETECT_PIN GPIO_NUM_NC
#define POWER_ADC_UNIT ADC_UNIT_1
#define POWER_ADC_CHANNEL ADC_CHANNEL_1
#define RIGHT_LEG_PIN GPIO_NUM_43
#define RIGHT_FOOT_PIN GPIO_NUM_44
#define LEFT_LEG_PIN GPIO_NUM_5
#define LEFT_FOOT_PIN GPIO_NUM_6
#define LEFT_HAND_PIN GPIO_NUM_4
#define RIGHT_HAND_PIN GPIO_NUM_7
#define AUDIO_INPUT_SAMPLE_RATE 16000
#define AUDIO_OUTPUT_SAMPLE_RATE 16000
#define AUDIO_I2S_GPIO_WS GPIO_NUM_40
#define AUDIO_I2S_GPIO_BCLK GPIO_NUM_42
#define AUDIO_I2S_GPIO_DIN GPIO_NUM_41
#define AUDIO_I2S_GPIO_DOUT GPIO_NUM_39
#define DISPLAY_BACKLIGHT_PIN GPIO_NUM_38
#define DISPLAY_MOSI_PIN GPIO_NUM_45
#define DISPLAY_CLK_PIN GPIO_NUM_48
#define DISPLAY_DC_PIN GPIO_NUM_47
#define DISPLAY_RST_PIN GPIO_NUM_1
#define DISPLAY_CS_PIN GPIO_NUM_NC
/* Camera PINs*/
#define I2C_SDA_PIN GPIO_NUM_15
#define I2C_SCL_PIN GPIO_NUM_16
#define CAMERA_XCLK (GPIO_NUM_3)
#define CAMERA_PCLK (GPIO_NUM_10)
#define CAMERA_VSYNC (GPIO_NUM_17)
#define CAMERA_HSYNC (GPIO_NUM_18)
#define CAMERA_D0 (GPIO_NUM_12)
#define CAMERA_D1 (GPIO_NUM_14)
#define CAMERA_D2 (GPIO_NUM_21)
#define CAMERA_D3 (GPIO_NUM_13)
#define CAMERA_D4 (GPIO_NUM_11)
#define CAMERA_D5 (GPIO_NUM_9)
#define CAMERA_D6 (GPIO_NUM_46)
#define CAMERA_D7 (GPIO_NUM_8)
#define CAMERA_PWDN (GPIO_NUM_NC)
#define CAMERA_RESET (GPIO_NUM_NC)
#define CAMERA_XCLK_FREQ (16000000)
#define LEDC_TIMER (LEDC_TIMER_0)
#define LEDC_CHANNEL (LEDC_CHANNEL_0)
#define CAMERA_SIOD (GPIO_NUM_NC)
#define CAMERA_SIOC (GPIO_NUM_NC)
#else
#define POWER_CHARGE_DETECT_PIN GPIO_NUM_21
#define POWER_ADC_UNIT ADC_UNIT_2
#define POWER_ADC_CHANNEL ADC_CHANNEL_3
@@ -32,6 +86,8 @@
#define DISPLAY_RST_PIN GPIO_NUM_11
#define DISPLAY_CS_PIN GPIO_NUM_12
#endif
#define LCD_TYPE_ST7789_SERIAL
#define DISPLAY_WIDTH 240
#define DISPLAY_HEIGHT 240
@@ -47,6 +103,5 @@
#define BOOT_BUTTON_GPIO GPIO_NUM_0
#define OTTO_ROBOT_VERSION "2.0.5"
#endif // _BOARD_CONFIG_H_
#endif // _BOARD_CONFIG_H_

10
main/boards/otto-robot/config.json
View File

@@ -6,6 +6,16 @@
"sdkconfig_append": [
"CONFIG_HTTPD_WS_SUPPORT=y"
]
},
{
"name": "otto-robot-camera",
"sdkconfig_append": [
"CONFIG_OTTO_ROBOT_USE_CAMERA=y",
"CONFIG_CAMERA_OV2640=y",
"CONFIG_CAMERA_OV2640_AUTO_DETECT_DVP_INTERFACE_SENSOR=y",
"CONFIG_CAMERA_OV2640_DVP_YUV422_240X240_25FPS=y",
"CONFIG_HTTPD_WS_SUPPORT=y"
]
}
]
}

4
main/boards/otto-robot/otto_controller.cc
View File

@@ -13,6 +13,7 @@
#include "config.h"
#include "mcp_server.h"
#include "otto_movements.h"
#include "power_manager.h"
#include "sdkconfig.h"
#include "settings.h"
#include <wifi_station.h>
@@ -75,6 +76,7 @@ private:
while (true) {
if (xQueueReceive(controller->action_queue_, &params, pdMS_TO_TICKS(1000)) == pdTRUE) {
ESP_LOGI(TAG, "执行动作: %d", params.action_type);
PowerManager::PauseBatteryUpdate(); // 动作开始时暂停电量更新
controller->is_action_in_progress_ = true;
if (params.action_type == ACTION_SERVO_SEQUENCE) {
// 执行舵机序列（自编程）- 仅支持短键名格式
@@ -427,6 +429,7 @@ private:
}
}
controller->is_action_in_progress_ = false;
PowerManager::ResumeBatteryUpdate(); // 动作结束时恢复电量更新
vTaskDelay(pdMS_TO_TICKS(20));
}
}
@@ -712,6 +715,7 @@ public:
action_task_handle_ = nullptr;
}
is_action_in_progress_ = false;
PowerManager::ResumeBatteryUpdate(); // 停止动作时恢复电量更新
xQueueReset(action_queue_);
QueueAction(ACTION_HOME, 1, 1000, 1, 0);

23
main/boards/otto-robot/otto_emoji_display.cc
View File

@@ -14,8 +14,8 @@
OttoEmojiDisplay::OttoEmojiDisplay(esp_lcd_panel_io_handle_t panel_io, esp_lcd_panel_handle_t panel, int width, int height, int offset_x, int offset_y, bool mirror_x, bool mirror_y, bool swap_xy)
: SpiLcdDisplay(panel_io, panel, width, height, offset_x, offset_y, mirror_x, mirror_y, swap_xy) {
InitializeOttoEmojis();
SetupChatLabel();
SetupPreviewImage();
SetTheme(LvglThemeManager::GetInstance().GetTheme("dark"));
}
void OttoEmojiDisplay::SetupPreviewImage() {
@@ -84,22 +84,6 @@ void OttoEmojiDisplay::InitializeOttoEmojis() {
ESP_LOGI(TAG, "Otto GIF表情初始化完成");
}
void OttoEmojiDisplay::SetupChatLabel() {
DisplayLockGuard lock(this);
if (chat_message_label_) {
lv_obj_del(chat_message_label_);
}
chat_message_label_ = lv_label_create(container_);
lv_label_set_text(chat_message_label_, "");
lv_obj_set_width(chat_message_label_, width_ * 0.9); // 限制宽度为屏幕宽度的 90%
lv_label_set_long_mode(chat_message_label_, LV_LABEL_LONG_SCROLL_CIRCULAR);
lv_obj_set_style_text_align(chat_message_label_, LV_TEXT_ALIGN_CENTER, 0); // 设置文本居中对齐
lv_obj_set_style_text_color(chat_message_label_, lv_color_white(), 0);
SetTheme(LvglThemeManager::GetInstance().GetTheme("dark"));
}
LV_FONT_DECLARE(OTTO_ICON_FONT);
void OttoEmojiDisplay::SetStatus(const char* status) {
auto lvgl_theme = static_cast<LvglTheme*>(current_theme_);
@@ -133,14 +117,13 @@ void OttoEmojiDisplay::SetStatus(const char* status) {
lv_obj_set_style_text_font(status_label_, text_font, 0);
lv_label_set_text(status_label_, "");
lv_obj_clear_flag(status_label_, LV_OBJ_FLAG_HIDDEN);
lv_obj_clear_flag(network_label_, LV_OBJ_FLAG_HIDDEN);
lv_obj_clear_flag(battery_label_, LV_OBJ_FLAG_HIDDEN);
return;
}
lv_obj_set_style_text_font(status_label_, text_font, 0);
lv_label_set_text(status_label_, status);
lv_obj_clear_flag(status_label_, LV_OBJ_FLAG_HIDDEN);
lv_obj_clear_flag(network_label_, LV_OBJ_FLAG_HIDDEN);
lv_obj_clear_flag(battery_label_, LV_OBJ_FLAG_HIDDEN);
}
void OttoEmojiDisplay::SetPreviewImage(std::unique_ptr<LvglImage> image) {

1
main/boards/otto-robot/otto_emoji_display.h
View File

@@ -19,6 +19,5 @@ class OttoEmojiDisplay : public SpiLcdDisplay {
private:
void InitializeOttoEmojis();
void SetupChatLabel();
void SetupPreviewImage();
};

96
main/boards/otto-robot/otto_robot.cc
View File

@@ -4,6 +4,8 @@
#include <esp_lcd_panel_ops.h>
#include <esp_lcd_panel_vendor.h>
#include <esp_log.h>
#include <freertos/FreeRTOS.h>
#include <freertos/task.h>
#include <wifi_station.h>
#include "application.h"
@@ -18,6 +20,7 @@
#include "power_manager.h"
#include "system_reset.h"
#include "wifi_board.h"
#include "esp32_camera.h"
#include "websocket_control_server.h"
#define TAG "OttoRobot"
@@ -30,6 +33,10 @@ private:
PowerManager* power_manager_;
Button boot_button_;
WebSocketControlServer* ws_control_server_;
#if CONFIG_OTTO_ROBOT_USE_CAMERA
i2c_master_bus_handle_t i2c_bus_;
Esp32Camera *camera_;
#endif
void InitializePowerManager() {
power_manager_ =
new PowerManager(POWER_CHARGE_DETECT_PIN, POWER_ADC_UNIT, POWER_ADC_CHANNEL);
@@ -115,10 +122,79 @@ private:
InitializeWebSocketControlServer();
}
#if CONFIG_OTTO_ROBOT_USE_CAMERA
void InitializeI2c() {
// Initialize I2C peripheral
i2c_master_bus_config_t i2c_bus_cfg = {
.i2c_port = I2C_NUM_0,
.sda_io_num = I2C_SDA_PIN,
.scl_io_num = I2C_SCL_PIN,
.clk_source = I2C_CLK_SRC_DEFAULT,
.glitch_ignore_cnt = 7,
.intr_priority = 0,
.trans_queue_depth = 0,
.flags =
{
.enable_internal_pullup = 1,
},
};
ESP_ERROR_CHECK(i2c_new_master_bus(&i2c_bus_cfg, &i2c_bus_));
}
void InitializeCamera() {
// DVP pin configuration
static esp_cam_ctlr_dvp_pin_config_t dvp_pin_config = {
.data_width = CAM_CTLR_DATA_WIDTH_8,
.data_io = {
[0] = CAMERA_D0,
[1] = CAMERA_D1,
[2] = CAMERA_D2,
[3] = CAMERA_D3,
[4] = CAMERA_D4,
[5] = CAMERA_D5,
[6] = CAMERA_D6,
[7] = CAMERA_D7,
},
.vsync_io = CAMERA_VSYNC,
.de_io = CAMERA_HSYNC,
.pclk_io = CAMERA_PCLK,
.xclk_io = CAMERA_XCLK,
};
// 复用 I2C 总线
esp_video_init_sccb_config_t sccb_config = {
.init_sccb = false, // 不初始化新的 SCCB，使用现有的 I2C 总线
.i2c_handle = i2c_bus_, // 使用现有的 I2C 总线句柄
.freq = 100000, // 100kHz
};
// DVP configuration
esp_video_init_dvp_config_t dvp_config = {
.sccb_config = sccb_config,
.reset_pin = CAMERA_RESET,
.pwdn_pin = CAMERA_PWDN,
.dvp_pin = dvp_pin_config,
.xclk_freq = CAMERA_XCLK_FREQ,
};
// Main video configuration
esp_video_init_config_t video_config = {
.dvp = &dvp_config,
};
camera_ = new Esp32Camera(video_config);
// camera_->SetHMirror(true);
camera_->SetVFlip(true);
}
#endif
public:
OttoRobot() : boot_button_(BOOT_BUTTON_GPIO) {
InitializeSpi();
InitializeLcdDisplay();
#if CONFIG_OTTO_ROBOT_USE_CAMERA
InitializeI2c();
InitializeCamera();
#endif
InitializeButtons();
InitializePowerManager();
InitializeOttoController();
@@ -126,11 +202,18 @@ public:
GetBacklight()->RestoreBrightness();
}
virtual AudioCodec* GetAudioCodec() override {
static NoAudioCodecSimplex audio_codec(AUDIO_INPUT_SAMPLE_RATE, AUDIO_OUTPUT_SAMPLE_RATE,
AUDIO_I2S_SPK_GPIO_BCLK, AUDIO_I2S_SPK_GPIO_LRCK,
AUDIO_I2S_SPK_GPIO_DOUT, AUDIO_I2S_MIC_GPIO_SCK,
AUDIO_I2S_MIC_GPIO_WS, AUDIO_I2S_MIC_GPIO_DIN);
virtual AudioCodec *GetAudioCodec() override {
#ifdef AUDIO_I2S_METHOD_SIMPLEX
static NoAudioCodecSimplex audio_codec(
AUDIO_INPUT_SAMPLE_RATE, AUDIO_OUTPUT_SAMPLE_RATE,
AUDIO_I2S_SPK_GPIO_BCLK, AUDIO_I2S_SPK_GPIO_LRCK,
AUDIO_I2S_SPK_GPIO_DOUT, AUDIO_I2S_MIC_GPIO_SCK, AUDIO_I2S_MIC_GPIO_WS,
AUDIO_I2S_MIC_GPIO_DIN);
#else
static NoAudioCodecDuplex audio_codec(
AUDIO_INPUT_SAMPLE_RATE, AUDIO_OUTPUT_SAMPLE_RATE, AUDIO_I2S_GPIO_BCLK,
AUDIO_I2S_GPIO_WS, AUDIO_I2S_GPIO_DOUT, AUDIO_I2S_GPIO_DIN);
#endif
return &audio_codec;
}
@@ -146,6 +229,9 @@ public:
level = power_manager_->GetBatteryLevel();
return true;
}
#if CONFIG_OTTO_ROBOT_USE_CAMERA
virtual Camera *GetCamera() override { return camera_; }
#endif
};
DECLARE_BOARD(OttoRobot);

47
main/boards/otto-robot/power_manager.h
View File

@@ -12,7 +12,7 @@ private:
static constexpr struct {
uint16_t adc;
uint8_t level;
} BATTERY_LEVELS[] = {{2150, 0}, {2450, 100}};
} BATTERY_LEVELS[] = {{2050, 0}, {2450, 100}};
static constexpr size_t BATTERY_LEVELS_COUNT = 2;
static constexpr size_t ADC_VALUES_COUNT = 10;
@@ -25,12 +25,23 @@ private:
size_t adc_values_count_ = 0;
uint8_t battery_level_ = 100;
bool is_charging_ = false;
inline static bool battery_update_paused_ = false; // 静态标志：是否暂停电量更新
adc_oneshot_unit_handle_t adc_handle_;
void CheckBatteryStatus() {
// 如果电量更新被暂停（动作进行中），则跳过更新
if (battery_update_paused_) {
return;
}
ReadBatteryAdcData();
if (charging_pin_ == GPIO_NUM_NC) {
is_charging_ = false;
} else {
is_charging_ = gpio_get_level(charging_pin_) == 0;
ReadBatteryAdcData();
}
}
void ReadBatteryAdcData() {
@@ -71,6 +82,8 @@ public:
PowerManager(gpio_num_t charging_pin, adc_unit_t adc_unit = ADC_UNIT_2,
adc_channel_t adc_channel = ADC_CHANNEL_3)
: charging_pin_(charging_pin), adc_unit_(adc_unit), adc_channel_(adc_channel) {
if (charging_pin_ != GPIO_NUM_NC) {
gpio_config_t io_conf = {};
io_conf.intr_type = GPIO_INTR_DISABLE;
io_conf.mode = GPIO_MODE_INPUT;
@@ -78,6 +91,10 @@ public:
io_conf.pull_down_en = GPIO_PULLDOWN_DISABLE;
io_conf.pull_up_en = GPIO_PULLUP_ENABLE;
gpio_config(&io_conf);
ESP_LOGI("PowerManager", "充电检测引脚配置完成: GPIO%d", charging_pin_);
} else {
ESP_LOGI("PowerManager", "充电检测引脚未配置，不进行充电状态检测");
}
esp_timer_create_args_t timer_args = {
.callback =
@@ -97,18 +114,20 @@ public:
}
void InitializeAdc() {
adc_oneshot_unit_init_cfg_t init_config = {
.unit_id = adc_unit_,
.ulp_mode = ADC_ULP_MODE_DISABLE,
};
ESP_ERROR_CHECK(adc_oneshot_new_unit(&init_config, &adc_handle_));
adc_oneshot_unit_init_cfg_t init_config = {
.unit_id = adc_unit_,
.clk_src = ADC_RTC_CLK_SRC_DEFAULT,
.ulp_mode = ADC_ULP_MODE_DISABLE,
};
ESP_ERROR_CHECK(adc_oneshot_new_unit(&init_config, &adc_handle_));
adc_oneshot_chan_cfg_t chan_config = {
.atten = ADC_ATTEN_DB_12,
.bitwidth = ADC_BITWIDTH_12,
};
adc_oneshot_chan_cfg_t chan_config = {
.atten = ADC_ATTEN_DB_12,
.bitwidth = ADC_BITWIDTH_12,
};
ESP_ERROR_CHECK(adc_oneshot_config_channel(adc_handle_, adc_channel_, &chan_config));
ESP_ERROR_CHECK(
adc_oneshot_config_channel(adc_handle_, adc_channel_, &chan_config));
}
~PowerManager() {
@@ -124,5 +143,9 @@ public:
bool IsCharging() { return is_charging_; }
uint8_t GetBatteryLevel() { return battery_level_; }
// 暂停/恢复电量更新（用于动作执行时屏蔽更新）
static void PauseBatteryUpdate() { battery_update_paused_ = true; }
static void ResumeBatteryUpdate() { battery_update_paused_ = false; }
};
#endif // __POWER_MANAGER_H__